u/Udbhav96

i wanna build a text diffusion model which interpret text and convert it into ascii images

so like

Text : build a cat

Output :

/\\\_/\\

( o.o )

\> \^ <

So , i have a decent background of ml algo ( completed cs229 , cs230 , Ml architecture and basic CNN and diffusion model )

ik making a project like this is tricky and making diffusion model like that from scratch is hard but i wanna try it because that's wot make me excited lol ...

I am currently reading GANs research paper , can u guys help me in finding more papers which helps me in making this project or guide me through this good title for this

Thx in adv

i wanna build a text diffusion model which interpret text and convert it into ascii images

so like

Text : build a cat

Output :

/\\\_/\\

( o.o )

\> \^ <

So , i have a decent background of ml algo ( completed cs229 , cs230 , Ml architecture and basic CNN and diffusion model )

ik making a project like this is tricky and making diffusion model like that from scratch is hard but i wanna try it because that's wot make me excited lol ...

I am currently reading GANs research paper , can u guys help me in finding more papers which helps me in making this project or guide me through this good title for this

Thx in adv

i wanna build a text diffusion model which interpret text and convert it into ascii images

so like

Text : build a cat

Output :

/\\\_/\\

( o.o )

\> \^ <

So , i have a decent background of ml algo ( completed cs229 , cs230 , Ml architecture and basic CNN and diffusion model )

ik making a project like this is tricky and making diffusion model like that from scratch is hard but i wanna try it because that's wot make me excited lol ...

I am currently reading GANs research paper , can u guys help me in finding more papers which helps me in making this project or guide me through this good title for this

Thx in adv

Just finished reading Attention Is All You Need and I genuinely can't stop thinking about it.

I knew Transformers power modern LLMs, but reading the original paper felt different.

The craziest part?

The authors looked at a world dominated by RNNs and basically said:

"What if we remove recurrence entirely?"

And somehow that worked.

No recurrence.
No convolutions.
Just attention.

The paper isn't even trying to be flashy. It's mostly a series of elegant engineering decisions:

• Scaled Dot-Product Attention
• Multi-Head Attention
• Residual Connections
• Layer Normalization
• Positional Encodings
• Massive Parallelization

Individually, none of these ideas feel magical.

Together, they changed AI.

One thing that really stood out to me was the discussion about path lengths between tokens. In an RNN, information might need to travel through many sequential steps. In self-attention, every token can directly interact with every other token in a single layer.

Such a simple idea.

Such massive consequences.

It's funny reading this paper in 2026 knowing that GPT, Claude, Gemini, Llama, Mistral, DeepSeek, and basically every major LLM can trace their roots back to these 15 pages.

Sometimes progress isn't adding more complexity.

It's removing the thing everyone assumed was necessary.

Attention really was all they needed.

Question:

• What's your favorite ML paper of all time and why?
• Do you think the Transformer will eventually be replaced, or are we still in the early chapters of its story?

Just finished reading Attention Is All You Need and I genuinely can't stop thinking about it.

I knew Transformers power modern LLMs, but reading the original paper felt different.

The craziest part?

The authors looked at a world dominated by RNNs and basically said:

"What if we remove recurrence entirely?"

And somehow that worked.

No recurrence.
No convolutions.
Just attention.

The paper isn't even trying to be flashy. It's mostly a series of elegant engineering decisions:

• Scaled Dot-Product Attention
• Multi-Head Attention
• Residual Connections
• Layer Normalization
• Positional Encodings
• Massive Parallelization

Individually, none of these ideas feel magical.

Together, they changed AI.

One thing that really stood out to me was the discussion about path lengths between tokens. In an RNN, information might need to travel through many sequential steps. In self-attention, every token can directly interact with every other token in a single layer.

Such a simple idea.

Such massive consequences.

It's funny reading this paper in 2026 knowing that GPT, Claude, Gemini, Llama, Mistral, DeepSeek, and basically every major LLM can trace their roots back to these 15 pages.

Sometimes progress isn't adding more complexity.

It's removing the thing everyone assumed was necessary.

Attention really was all they needed.

QUESTION:

• What's your favorite ML paper of all time and why?
• Do you think the Transformer will eventually be replaced, or are we still in the early chapters of its story?

A few hours ago, I posted about embeddings and tokenization.

&#x200B;

After spending time understanding the theory, I wanted to see what happens when you actually build part of the pipeline yourself.

&#x200B;

So I spent the few hrs building a Byte Pair Encoding (BPE) tokenizer pipeline from scratch.

&#x200B;

The project: • Extracts Wikipedia data • Trains a custom BPE tokenizer • Evaluates it on WikiText-103 and Penn Treebank • Compares outputs against GPT-2's tokenizer • Includes a web UI for visualizing tokenization in real time

&#x200B;

One thing I didn't fully appreciate before building it was how much tokenization influences everything downstream. Context usage, compression efficiency, vocabulary design, and even training costs all start here.

&#x200B;

Demo: https://mini-bpe-udbhav96s-projects.vercel.app/

&#x200B;

My long-term goal is to understand and build the major components behind modern AI systems from scratch.

&#x200B;

I'm thinking the next project might be a web crawler and data collection pipeline so I can continue moving backward through the LLM stack.

&#x200B;

For those who have built LLM infrastructure:

&#x200B;

• What would you build next after a tokenizer? • What mistakes do beginners usually make when building data pipelines? • Are there any tokenizer evaluation metrics you think deserve more attention?

&#x200B;

Would love feedback, criticism, or suggestions.

&#x200B;

&#x200B;

A few hours ago, I posted about embeddings and tokenization.

&#x200B;

After spending few hrs understanding the theory, I wanted to see what happens when you actually build part of the pipeline yourself.

&#x200B;

So I spent the last few weeks building a Byte Pair Encoding (BPE) tokenizer pipeline from scratch.

&#x200B;

The project: • Extracts Wikipedia data • Trains a custom BPE tokenizer • Evaluates it on WikiText-103 and Penn Treebank • Compares outputs against GPT-2's tokenizer • Includes a web UI for visualizing tokenization in real time

&#x200B;

One thing I didn't fully appreciate before building it was how much tokenization influences everything downstream. Context usage, compression efficiency, vocabulary design, and even training costs all start here.

&#x200B;

Demo: https://mini-bpe-udbhav96s-projects.vercel.app/

&#x200B;

My long-term goal is to understand and build the major components behind modern AI systems from scratch.

&#x200B;

I'm thinking the next project might be a web crawler and data collection pipeline so I can continue moving backward through the LLM stack.

&#x200B;

For those who have built LLM infrastructure:

&#x200B;

• What would you build next after a tokenizer? • What mistakes do beginners usually make when building data pipelines? • Are there any tokenizer evaluation metrics you think deserve more attention?

&#x200B;

Would love feedback, criticism, or suggestions.

&#x200B;

&#x200B;

A few hours ago, I posted about embeddings and tokenization.

After spending time understanding the theory, I wanted to see what happens when you actually build part of the pipeline yourself.

So I spent the last few hrs building a Byte Pair Encoding (BPE) tokenizer pipeline from scratch.

The project:
• Extracts Wikipedia data
• Trains a custom BPE tokenizer
• Evaluates it on WikiText-103 and Penn Treebank
• Compares outputs against GPT-2's tokenizer
• Includes a web UI for visualizing tokenization in real time

One thing I didn't fully appreciate before building it was how much tokenization influences everything downstream. Context usage, compression efficiency, vocabulary design, and even training costs all start here.

Demo:
https://mini-bpe-udbhav96s-projects.vercel.app/

My long-term goal is to understand and build the major components behind modern AI systems from scratch.

I'm thinking the next project might be a web crawler and data collection pipeline so I can continue moving backward through the LLM stack.

For those who have built LLM infrastructure:

• What would you build next after a tokenizer?
• What mistakes do beginners usually make when building data pipelines?
• Are there any tokenizer evaluation metrics you think deserve more attention?

Would love feedback, criticism, or suggestions.

I've been diving into tokenization lately and finally feel like I understand why modern LLMs use BPE instead of simple character or word tokenization.

My understanding so far:

• Character-level tokenization → tiny vocabulary but very long sequences.
• Word-level tokenization → shorter sequences but massive vocabularies and OOV issues.
• Unicode (ord) tokenization → supports all languages but ends up with both a huge vocabulary and long sequences.
• UTF-8 byte tokenization → fixed vocabulary of 256 bytes but sequences become extremely long.

This is where Byte Pair Encoding (BPE) becomes interesting.

Instead of treating every byte individually, BPE repeatedly merges the most frequent byte pairs into new tokens, gradually building a larger vocabulary while reducing sequence length.

A simple example:

aaabdaaabac

→ ZabdZabac (Z = aa)

→ ZYdZYac (Y = ab)

→ XdXac (X = ZY)

What I found interesting is that BPE seems to hit a sweet spot between vocabulary size and sequence length, which is why many modern LLM tokenizers are based on it.

I'm planning to implement a BPE tokenizer from scratch next to understand the training process more deeply.

For those who have built tokenizers before:

• Did implementing BPE from scratch help you understand LLMs better?
• Any common mistakes or edge cases I should watch out for?

Reference:
https://medium.com/thedeephub/all-you-need-to-know-about-tokenization-in-llms-7a801302cf54

Post drafted with assistance from ChatGPT.

I've been learning about embeddings recently and finally found an explanation that made the concept click for me.

Imagine these sentences:

• "When the worker left..."
• "When the fisherman left..."
• "When the dog left..."

Even if we don't know what the words mean, we can see that they appear in very similar contexts.

The core idea behind word embeddings is that if two words appear in similar contexts across a massive corpus, their meanings are probably related. Instead of storing words as strings, we map them to vectors in a high-dimensional space (often hundreds of dimensions).

What I found interesting is that the model isn't explicitly taught what "cat" or "dog" means. During training, it learns tasks like predicting context words, and meaningful embeddings emerge as a byproduct.

Another thing I learned is that embedding matrices are huge. A vocabulary of 50,000 words with 300-dimensional embeddings already requires around 15 million parameters. Yet during a training step, only a small subset of word vectors gets updated, which creates some interesting distributed-systems challenges around sparse communication and synchronization.

The famous example:

King − Queen ≈ Man − Woman

isn't magic—it's a consequence of the geometric relationships learned in the embedding space.

For people who work with LLMs regularly:

What's the intuition or explanation that finally made embeddings "click" for you?

Source:
https://petuum.medium.com/embeddings-a-matrix-of-meaning-4de877c9aa27

Post drafted with ChatGPT and reviewed by me.

Hey everyone!!

I'm currently researching how technology is stepping in to handle repetitive and time-consuming tasks across different industries and I want to hear from real employees like you!

**What I'm exploring:**

-->How technology is taking over routine tasks so employees can focus on more meaningful work

Which fields are seeing the biggest shift in how work gets done

-->How workers are evolving alongside technology in their roles

--> The opportunities technology is opening up for employees across industries

**I want to hear YOUR experience:**

Has technology in your workplace freed up your time to focus on bigger, more creative responsibilities? What tasks have been automated and how has it changed your day-to-day?

Whether you're in healthcare, finance, education, retail, logistics, tech or any other field — your story is valuable!

**Want to share more?**

If you'd like to have a deeper conversation about how technology is shaping your field, feel free to DM me! I'd love to connect one-on-one.

Drop a comment or DM me , let's have a great conversation!!

Hey everyone!!

I'm currently researching how technology is stepping in to handle repetitive and time-consuming tasks across different industries and I want to hear from real employees like you!

**What I'm exploring:**

-->How technology is taking over routine tasks so employees can focus on more meaningful work

Which fields are seeing the biggest shift in how work gets done

-->How workers are evolving alongside technology in their roles

--> The opportunities technology is opening up for employees across industries

**I want to hear YOUR experience:**

Has technology in your workplace freed up your time to focus on bigger, more creative responsibilities? What tasks have been automated and how has it changed your day-to-day?

Whether you're in healthcare, finance, education, retail, logistics, tech or any other field — your story is valuable!

**Want to share more?**

If you'd like to have a deeper conversation about how technology is shaping your field, feel free to DM me! I'd love to connect one-on-one.

Drop a comment or DM me , let's have a great conversation!!

Hey everyone!!

I'm currently researching how technology is stepping in to handle repetitive and time-consuming tasks across different industries and I want to hear from real employees like you!

\*\*What I'm exploring:\*\*

\-->How technology is taking over routine tasks so employees can focus on more meaningful work

Which fields are seeing the biggest shift in how work gets done

\-->How workers are evolving alongside technology in their roles

\--> The opportunities technology is opening up for employees across industries

\*\*I want to hear YOUR experience:\*\*

Has technology in your workplace freed up your time to focus on bigger, more creative responsibilities? What tasks have been automated and how has it changed your day-to-day?

Whether you're in healthcare, finance, education, retail, logistics, tech or any other field — your story is valuable!

\*\*Want to share more?\*\*

If you'd like to have a deeper conversation about how technology is shaping your field, feel free to DM me! I'd love to connect one-on-one.

Drop a comment or DM me , let's have a great conversation!!

Hey everyone!!

I'm currently researching how technology is stepping in to handle repetitive and time-consuming tasks across different industries and I want to hear from real employees like you!

What I'm exploring:

-->How technology is taking over routine tasks so employees can focus on more meaningful work

Which fields are seeing the biggest shift in how work gets done

-->How workers are evolving alongside technology in their roles

--> The opportunities technology is opening up for employees across industries

I want to hear YOUR experience:

Has technology in your workplace freed up your time to focus on bigger, more creative responsibilities? What tasks have been automated and how has it changed your day-to-day?

Whether you're in healthcare, finance, education, retail, logistics, tech or any other field — your story is valuable!

Want to share more?

If you'd like to have a deeper conversation about how technology is shaping your field, feel free to DM me! I'd love to connect one-on-one.

Drop a comment or DM me , let's have a great conversation!!