u/chrisvdweth

A small milestone: SELENE has reached 200 starts on GitHub!

Since GitHub stars are currently my main source of feedback, and I'm sure that many of those come form this community: Thanks a lot!

Just as a quick reminder: SELENE is a public repository of Jupyter notebooks covering topics around AI, ML/DL, NLP, data mining, data science. What started out as interactive lecture notes for my courses as NUS, has slowly grown to something the might be useful for anyone starting to learn about these topics.

The current focus is on the fundamentals, so the target audience are beginners but who are comfortable with basic math (linear algebra, calculus, probability theory). Here is a crude overview to some of the topics (the links go to the HTML version of the notebooks)

• Traditional models: Linear Regression [1,2,3], Logistic Regression [1,2], Multinomial Naive Bayes, Decision Trees / CART [1,2], Random Forests, Boosting Methods (AdaBoost, Gradient Boosted Machines, XGBoost, LightGBM, CatBoost)
• Neural network models: basics / MLPs (incl Backpropagation [1,2]), RNNs (incl. Backpropagation Through Time), Training a NumPy-only MLP
• Neural network components: linear layer, residual connections, layer normalization, dropout, mixture-of-experts
• Transformers: attention mechanism, transformer architecture, positional encodings [1,2,3], masking
• LLMs: language models [1,2], RAG [1,2], fine-tuning [1,2], training an LLM from scratch, efficiency strategies, data preparation
• Optimizers: Gradient Descent with Momentum, RMSProp, AdaGrad, Adam
• NLP basics: tokenization (incl. Byte-Pair Encoding and WordPiece), normalization, lemmatization & stemming, embeddings (overview, Word2Vec [1,2])

There is an overview page for all topics with links to the HTML version, the GitHub repo, as well to open each notebook directly in Google Colab. We are also in the process of building a web interface to help navigate topics and suggest learning paths.

SELENE will continue to grow. Right now, I'm working on time series analysis and classical statistical models (e.g., AR, ARMA, ARIMA) – I want to cover this topic in my data mining course in the upcoming semester :).

A few days ago I created a post to share our public SELENE repo. In a nutshell, SELENE is a growing collection of Jupyter notebooks covering fundamentals in AI/ML/NLP, data mining, etc. In this post, I already added a screenshot of a website we are building to navigate the topic and suggest mastery/learning paths, kind of as a teaser.

Well, we now have a live prototype for people to have a look and maybe share some feedback – there are many things we're not really sure not to best approach it. First a few comments for things that might not be obvious:

• You can right-click or tap+hold a graph node to access the context.
• Master paths shows all the paths from the root (START) to the selected topic.
• The topic graph has 2 types of edges: normal edges represent "subtopics"; dashed edges represent "(recommended) prerequisites".
• The site is currently running my own (very) small VPS; we have no idea how stable it is :)
• The current topic graph does not cover all the content in the repo.

Some things we are still contemplating about or struggle with:

• Defining the topic graph is surprisingly difficult; I'm sure there are edges in the current graph some of you will disagree with.
• Not sure if showing the full graph is actually meaningful, particularly if it keeps growing.
• Notebooks can be rather theoretical or very practical – or anything in between. Right now, you cannot really tell from the topic nodes in the graph.
• general, we already support that topics and notebooks can have a "type" – like "subtopic" or "prerequisite" for edges – but don't implement this yet.

In short, many of the visualization and navigation bits are far from fleshed out yet. So we are open to any feedback.

In the long run: we plan and already explore things like self-assessment (e.g., quizzes) and tracking user's progress, as well as personalized recommendation. These features would require user accounts; right now we focus on the public part, mainly browsing, searching, and navigating the content.

Since I just uploaded the 75th Jupyter notebook to my public GitHub repo I built to maintain and grow interactive lecture notes, I thought about sharing it with the community. I teach AI/ML/NLP and related university courses and provide Jupyter notebooks as lecture notes for a long time with my students. The SELENE repo is the next iteration: consolidating the notebooks across all my courses and improving them towards an open, large-scale, interactive textbook.

The current focus is on the fundamentals, so the target audience are beginners but who are comfortable with basic math (linear algebra, calculus, probability theory). Here is a crude overview to some of the topics (the links go to the HTML version of the notebooks)

• Traditional models: Linear Regression [1,2,3], Logistic Regression [1,2], Multinomial Naive Bayes, Decision Trees / CART [1,2], Random Forests, Boosting Methods (AdaBoost, Gradient Boosted Machines, XGBoost, LightGBM, CatBoost)
• Neural network models: basics / MLPs (incl Backpropagation [1,2]), RNNs (incl. Backpropagation Through Time), Training a NumPy-only MLP
• Neural network components: linear layer, residual connections, layer normalization, dropout, mixture-of-experts
• Transformers: attention mechanism, transformer architecture, positional encodings [1,2,3], masking
• LLMs: language models [1,2], RAG [1,2], fine-tuning [1,2], training an LLM from scratch, efficiency strategies, data preparation
• Optimizers: Gradient Descent with Momentum, RMSProp, AdaGrad, Adam
• NLP basics: tokenization (incl. Byte-Pair Encoding and WordPiece), normalization, lemmatization & stemming, embeddings (overview, Word2Vec [1,2])

There is an overview page for all topics with links to the HTML version, the GitHub repo, as well to open each notebook directly in Google Colab.

Feel free to check it out; hopefully useful to some of you who want to get started. I would be curious what other people other than my students think. I do have a Discord server for the latest updates and handling questions and other issues.

We are also in the process of building a web interface to help navigate topics and suggest learning paths. A first prototype is almost done; see the screenshot for a teaser.