u/Ok_Second2105

A few weeks ago, I built a neural network from scratch to understand what was happening behind the scenes.

I manually implemented:

• Forward propagation
• Backpropagation
• Gradient calculations
• Weight updates
• Activation functions

That project taught me a lot about the calculus and mathematics that make neural networks work.

After understanding the fundamentals, I decided to recreate the same MNIST handwritten digit classifier using PyTorch.

This time, instead of implementing everything manually, I used:

• torch.nn
• torch.nn.functional
• Built-in optimizers

One thing that surprised me was how dramatically the code complexity decreased. What previously required implementing dozens of lines of mathematical operations could now be expressed in just a few layers and a training loop.

At the same time, I feel like I appreciate PyTorch much more now because I understand what those functions are actually doing under the hood.

For those who learned deep learning:

Do you think building a neural network from scratch is still worth the effort today?

After doing both projects, my current opinion is that building one from scratch helped me understand why PyTorch works.

I'm curious whether more experienced practitioners agree with that perspective or think the time would be better spent elsewhere.

GitHub:
https://github.com/HelloSamved/learning-neural-network/blob/master/mnist_prediction/mnist_prediction_pytorch.ipynb

Writing topics on Excali:
https://excalidraw.com/#json=-R2-NuPIsipANT5l9tXW_,w1qUhg3vyl644_OC3o81pA

Over the past week, I've been building a neural network from scratch as a learning project.

The original goal was to understand what was happening behind libraries like TensorFlow and PyTorch—forward propagation, backpropagation, gradient descent, and the calculus that makes the whole thing work.

After a few iterations, I improved the model to about 94.1% accuracy on MNIST and decided to take the next step: deployment.

So I built a small Streamlit app where you can draw a digit and let the network predict what number it is.

Try it here:

https://predictdigits.streamlit.app/

What I found interesting is that building the model was only part of the challenge. Deploying it forced me to think about things I hadn't considered before:

• Converting user drawings into the same format as the training data
• Preprocessing inputs consistently
• Handling prediction confidence
• Making the model usable by someone who has never seen the code

For people who have built ML projects before:

At what point do you consider a project "complete"?

Is training the model enough, or do you think deployment and creating a usable interface is an essential part of the learning process?

I'd love any feedback on the app, the user experience, or the overall project.

A few days ago, I built a neural network from scratch to better understand forward propagation, backpropagation, and the calculus behind training.

While working on that project, I realized that I was spending all my time understanding what happens inside the network, but almost no time understanding what happens before the data even reaches the network.

That led me down a rabbit hole of tokenization.

So for my next learning project, I built a simple tokenizer from scratch.

The goal wasn't to create something as sophisticated as GPT's tokenizer, but to understand how text is transformed into numerical representations that neural networks can actually process.

Some things I explored:

• Building a vocabulary from text
• Converting tokens into IDs
• Encoding and decoding text
• Handling unknown words
• Understanding why tokenization is necessary before text can be fed into a neural network

One thing I found interesting is that we often spend a lot of time discussing neural network architectures, activation functions, and optimization techniques, but the quality of the tokenization step can have a huge impact on what information the model is actually able to learn.

Repository:
https://github.com/HelloSamved/learning-neural-network/tree/master/human-conversation

For those who have worked with NLP models or LLMs:

Am I thinking about this correctly? It seems like tokenization is effectively the bridge between human language and the numerical world that neural networks operate in. If that's true, what would be the next logical concept to learn after building a basic tokenizer?

Would you recommend:

• Byte Pair Encoding (BPE)
• WordPiece
• Embeddings
• Attention mechanisms
• Something else?

I'd love to hear what path more experienced practitioners would take from here.
Can anyone please help me by telling me is it really possible to create a whole neural network from very scratch just by using numpy and maths and create a chatbot based on human conversation.
If yes then can you please tell me how can we use our own tokenizer in that.

Again sorry for the handwritten notes but while learning I prefer to make handwritten notes and I didn't got enough time to make them in Latex

Yesterday I shared a neural network that I built from scratch to better understand what happens behind frameworks like TensorFlow and PyTorch.

Today I spent some time redesigning the network and digging deeper into the calculus behind backpropagation. While implementing the changes, I added an additional hidden layer, bringing the network to 3 trainable layers in total.

The model is still trained on the MNIST handwritten digit dataset, but the test accuracy increased from roughly 92% to 94.1%.

What I learned:

• How gradients flow through multiple layers
• Applying the chain rule across the network
• Why backpropagation works mathematically rather than just conceptually
• How deeper architectures affect learning

One thing I'm trying to understand:

Adding an extra layer increased the accuracy by only about 2%. Is that roughly what you would expect on a dataset like MNIST, or does it suggest that the added complexity isn't contributing much?

My intuition is that MNIST is already a relatively simple dataset, so adding more layers may not provide huge gains. But I'm still learning, so I'd love to know whether that reasoning is correct or if I'm missing something important.

Repository:
https://github.com/HelloSamved/learning-neural-network

Any feedback on the architecture, learning process, or my understanding of the results would be greatly appreciated.
Sorry for handwritten notes but I didn't got enough time to make my notes in LateX