u/teee0512

this is what ive learnt:

still building on linear regression

I thought logistic regression was just “linear regression, but for classification.” Turns out I was missing a pretty important detail.

v good example which helped me understand better:

Say you’re building a spam detector.

A linear regression model might happily predict:

1.7 → spam

-0.4 → not spam

The math works.

The answers don’t.

Because unlike predicting house prices or temperatures, classification isn’t asking us “what number?” righttt!

It’s asking:

“How confident are you that this belongs to class A?”

That confidence has to be between 0 and 1.

That’s where the sigmoid finally made sense to me.

It takes any real number…

-100

0

5

1000

…and squishes it into a value strictly between 0 and 1.

The coolest part?

It never actually reaches 0 or 1.

It gets ridiculously close, but never all the way there.

Which basically means the model is never saying,

“I’m infinitely certain.” There’s always some uncertainty left.

That felt surprisingly elegant to me :)

Also made me realize logistic regression isn’t really replacing linear regression..

It’s taking the same linear equation…

z = wx + b

…and asking a completely different question.

Instead of: What’s the output?

it’s asking: Given this score, what’s the probability this belongs to class 1?

that tiny shift changes everything.

if I’ve oversimplified anything, please call it out. I’d genuinely love corrections from people who’ve worked with ML longer than I have.

Still building on linear regression, equation is just the half part of thestory and the other half is how the model learns.

this is all what i already knew ok :)

I knew gradient descent minimized the loss function.

I knew linear algebra helped scale the equation.

I never connected the two in my head!!

What finally clicked today after reading more is that they’re solving two completely different problems.

Linear algebra lets the equation scale from one feature to thousands.

Calculus lets the model figure out which parameters minimize the error

Then my brain did a complete 180 on linear regression lol,

The loss function for ordinary least squares is convex.

which means gradient descent isn’t just “trying its best” to find a good solution. Right?

it can actually converge to the global minimum because the loss surface is convex, so there aren’t any local minima to get trapped in.

Then I found out something even cooler.

Gradient descent isn’t even necessary.

There’s an exact analytical solution:

θ = (XᵀX)⁻¹Xᵀy

No learning rate.

No iterations.

No waiting for convergence.

Just the optimal parameters.

The reason we still use gradient descent is because inverting huge matrices gets ridiculously expensive as the number of features grows.

So we intentionally trade an exact solution for a computationally cheaper approximation.

That was such an “ohhhh” moment for me.

This is basically me learning ML in public at this point :p

If I’ve messed something up or oversimplified anything, please roast the explanation, not me. I’d genuinely appreciate corrections from people who’ve been in the field longer :)

Yesterday I learned why y = wx + b is the foundation of ML. Today I learned that the equation is actually the easy part.

The hard part is teaching the model when it’s wrong.

That starts with something called a residual.

Residual = Actual - Predicted

Sounds ridiculously simple.

If the model predicts 8 and the answer is 10, the residual is +2.

Predict 12 instead?

Residual becomes -2.

My first thought was… Cool. Just add up all the residuals.

Except that completely breaks.

Positive and negative errors cancel each other out, so a model making huge mistakes in both directions could still end up with a total error close to zero.

So instead of summing residuals directly, we define a loss function.

The two that finally clicked for me today:

MAE (Mean Absolute Error)

Take the absolute value of every residual.

Every mistake is treated equally.

A mistake of 10 hurts exactly 10× more than a mistake of 1.

MSE (Mean Squared Error)

Square every residual before averaging.

Now a residual of 10 contributes 100.

A residual of 1 contributes 1.

Large mistakes suddenly become really expensive.

That means the model gets pushed much harder to fix its worst predictions.

The coolest part...?

I always assumed MSE became the default because it measures error “better.”

Turns out that’s not really the reason.

One big reason is that squaring gives you a smooth, differentiable function, which makes gradient descent much easier to optimize.

Tiny mathematical decision.

Massive impact on how models actually learn.

Attaching the animation that made this click for me because seeing the residuals change visually explained it way better than reading another formula.

What should I dive into next? I'm lovin it so far!!!

Genuine QUESTION!!!: what’s the distinction between a residual, a loss function, a cost function, and an objective function? Different books seem to use these terms differently. im p confused. is there a convention practitioners actually follow, or is it mostly context-dependent??????

Today I learned something that I think I rushed through when I first started learning ML.

The equation y = wx + b looks almost too simple, so I never paid much attention to it.

What finally clicked for me is that this isn’t just the equation of a line.

With one feature, you’re fitting a line.

With two features, you’re fitting a plane.

With n features, you’re fitting a hyperplane in n-dimensional space.

The equation barely changes: y = w₁x₁ + w₂x₂ + … + wₙxₙ + b

Another thing I didn’t know until today:

“Linear” doesn’t necessarily mean the relationship between x and y is a straight line. It means the model is linear in its parameters (the weights). So you can use features like x² or log(x) and it’s still linear regression.

That also helped me understand why linear models are still widely used in production—they’re simple, interpretable, and every weight has a meaning.

Kind of funny that I spent more time trying to understand transformers than the equation almost every supervised ML model builds on.

For people who’ve been doing ML for a while: I’m working through ML from first principles. What topic should I dive into next?