Manifold hypothesis

Manifold hypothesis is a very interesting topic and kind of a high-level inspiration of explainable AI. It has the power of generalization both in image modality and in NLP.

In both universes, this hypothesis suggests that the enormous dimensional space in which images, for example, exist is completely sparse, except for a very, very tiny space in which all of our visuals exist.

So the probability of drawing a sample from all possible high-dimensional images and finding that sample looking like any possible known image, or even a non-complete noise image, is extremely low.

That idea suggests that all known images are kind of a manifold that the deep learning model tries to unfold.

Just like when you have a sheet of paper, which is 2D, and you write text on it, which is also 2D. But suppose you crumple that paper; then the text appears to be in 3-dimensional space, while it is not.

The role of generative deep learning is to learn this crumpled high-dimensional modality and generate meaningful samples from it.

reddit.com
u/Logical_Respect_2381 — 1 month ago
▲ 11 r/deeplearning+1 crossposts

Manifold hypothesis

Manifold hypothesis is a very interesting topic and kind of a high-level inspiration of explainable AI. It has the power of generalization both in image modality and in NLP.

In both universes, this hypothesis suggests that the enormous dimensional space in which images, for example, exist is completely sparse, except for a very, very tiny space in which all of our visuals exist.

So the probability of drawing a sample from all possible high-dimensional images and finding that sample looking like any possible known image, or even a non-complete noise image, is extremely low.

That idea suggests that all known images are kind of a manifold that the deep learning model tries to unfold.

Just like when you have a sheet of paper, which is 2D, and you write text on it, which is also 2D. But suppose you crumple that paper; then the text appears to be in 3-dimensional space, while it is not.

The role of generative deep learning is to learn this crumpled high-dimensional modality and generate meaningful samples from it.

reddit.com
u/Logical_Respect_2381 — 1 month ago

[Project] I mapped out the entire GPT-3 forward pass matrix-by-matrix (44-page visual architecture blueprint)

>

Go ahead and drop this right into the group. It’s a brand-new day, a fresh crowd is scrolling, and this post is perfectly optimized to respect the community rules while driving traffic straight to your profile!

u/Logical_Respect_2381 — 1 month ago
▲ 0 r/learnmachinelearning+1 crossposts

Visualizing the Math: Complete GPT-3 Forward Pass mapped out matrix-by-matrix (44 pages)

Hi everyone,

When I was first learning the mathematical foundations of transformers, I struggled with how abstract the equations felt compared to the actual tensor shapes moving through memory.

To bridge that gap, I spent a few weeks drawing out a comprehensive, step-by-step visual map of the entire GPT-3 forward pass—tracking every single matrix multiplication, dimension shift, and layer normalization from raw token inputs to final vocabulary logits.

It spans 44 highly detailed visual pages designed specifically for engineering students and independent researchers who learn better by seeing the actual linear algebra layouts.

How to download the full PDF: The complete document is available on a 'Pay-What-You-Want' ($0+) structure so it stays accessible to any student who needs it.

Because direct digital storefront links are filtered by Reddit site-wide, I have placed the copy-paste text link inside a pinned post at the top of my personal profile. Just click my username (u/Logical_Respect_2381) to grab it!

Let me know if this visual style helps clarify the multi-head attention subspace projections or the MLP dimensions for your studies!

u/Logical_Respect_2381 — 1 month ago

I made a 32-page visual guide on what happens after LLM pretraining — looking for feedback on the pipeline

A lot of beginner explanations make the journey sound like:

train a huge Transformer → release a ChatGPT-like assistant.

But a real assistant needs many layers after the base model:

base model → SFT → preference data → reward model → RLHF/DPO → safety training → chat formatting → tools → RAG → multimodality → evaluation → serving infrastructure → UX.

The attached image is one roadmap page from a 32-page visual guide I made to organize this journey in one place. The full guide also includes explanations, glossary pages, and a recommended learning path with courses/resources for each major part.

I’m mainly looking for feedback on the pipeline:

Does this look accurate for beginners?
Would you add, remove, or rename any stage?

https://preview.redd.it/jsdix48c3v2h1.png?width=1672&format=png&auto=webp&s=41388e2b21d8225f1e5f4711ba936d565d77638d

reddit.com
u/Logical_Respect_2381 — 2 months ago

I made a 32-page visual guide on what happens after LLM pretraining — looking for feedback on the pipeline

A lot of beginner explanations make the journey sound like:

train a huge Transformer → release a ChatGPT-like assistant.

But a real assistant needs many layers after the base model:

base model → SFT → preference data → reward model → RLHF/DPO → safety training → chat formatting → tools → RAG → multimodality → evaluation → serving infrastructure → UX.

The attached image is one roadmap page from a 32-page visual guide I made to organize this journey in one place. The full guide also includes explanations, glossary pages, and a recommended learning path with courses/resources for each major part.

I’m mainly looking for feedback on the pipeline:

Does this look accurate for beginners?
Would you add, remove, or rename any stage?

https://preview.redd.it/rj8grmt3zu2h1.png?width=1672&format=png&auto=webp&s=2c9a6c51cfd435443c84764643c63be3baf440db

reddit.com
u/Logical_Respect_2381 — 2 months ago

I recently tried to make a beginner-friendly visual explanation of how Stable Diffusion works, because I noticed many newcomers hear terms like diffusion, U-Net, latent space, cross-attention, and embeddings, but often struggle to see how the full system connects together.

So I put together a YouTube video using narrated slides that walks through the process step by step — from adding noise during training, to denoising, text conditioning, and newer transformer-based models.

I’m still learning myself, so I’m sure there are places that can be improved or explained better.

If anyone here is willing to watch and give honest feedback, I’d genuinely appreciate it — especially from people with stronger technical understanding of diffusion models.

Constructive criticism is very welcome. If something is inaccurate, oversimplified, or unclear, please tell me so I can improve future videos.

I’ll place the link in the comments. Thank you.

reddit.com
u/Logical_Respect_2381 — 2 months ago