Proby, the Probastack mascotprobastack

Teach a Line to Predict

The simplest model in the world is a straight line. Fit one by hand, watch the error shrink, then let the machine find the best line itself. That hill-climbing toward less error is exactly how models learn.

Beginner9 min read·Linear Regression · Loss / Error · Best Fit

The hook

Ten students told us how many hours they studied and what they scored. The dots clearly drift upward — more hours, higher marks. So here’s a question: if a friend studied for 7 hours, what would you bet they scored?

To answer, you’d mentally draw a line through the cloud of dots and read off the height at 7. That line is a model — a simple rule that turns an input (hours) into a prediction (score). The whole game is finding the right line.

Fit it by hand

Try it. Use the two sliders to tilt and lift the line until it slices through the dots as snugly as you can. The red lines show how far off each prediction is; the total error adds up their squares.

hours studied →score
Total error: 4,732

The red lines are the gaps between the line’s guess and reality. “Total error” adds up their squares — smaller is better.

Hunt for the smallest total error you can find by hand.

What 'best' means

Notice you weren’t fitting by vibes — you had a target: make the total error as small as possible. That single number turns “draw a good line” into a precise, solvable problem. The best line is simply the one where the error can’t shrink any further.

The reveal

That’s machine learning, stripped to its bones: pick a rule with some knobs, define what “wrong” means, and turn the knobs to be less wrong.

The downhill nudging has a name — gradient descent — and it’s the engine underneath almost everything, from this two-knob line to a language model with billions of knobs. The numbers get astronomically bigger; the idea stays exactly this small.

The math, gently

None of this needs scary symbols, but a peek under the hood is worth it. The line you dragged is just y^=b+mx\hat{y} = b + m x: pick an intercept bb (where the line starts) and a slope mm (how steeply it climbs), feed in an xx (hours), and out comes y^\hat{y}, your prediction.

For any real point, the residual is the vertical gap between what actually happened and what the line guessed: yi−y^iy_i - \hat{y}_i. Those are the little red lines in the lab. Some points sit above the line, some below — so the errors come with plus and minus signs.

To score a whole line with one number, we use Mean Squared Error. We square every residual, then average them. Squaring does two nice things at once: it throws away the sign (an error of −3 and +3 both count the same), and it punishes big misses far more than small ones.

But what if you couldn’t solve it directly — too many knobs, too messy a rule? Then you’d do what the lab’s “Let it learn” button does: start anywhere, check which way tilting mm and lifting bb makes the error drop, and take a small step that way. Repeat until you reach the bottom of the error surface. That “roll downhill” move is gradient descent — the exact same idea that trains models with billions of knobs.

Check yourself

When a model 'learns,' what is it actually doing?

In ŷ = b + m·x, what does the slope m actually tell you?

Why does Mean Squared Error square the residuals instead of just adding them up?

Where it shows up

This one idea — fit a rule by minimizing error — predicts house prices from square footage, demand from price, risk from history, and a thousand other things. Master the line and you’ve got the seed of every model that follows: the only real difference is how many knobs there are, and how bendy the rule is allowed to be.

← All stories