Strip away the mystique and a neuron is one small formula:

out = activation(w₁·x₁ + w₂·x₂ + b)

It takes its inputs, multiplies each by a weight (how much it cares), adds a bias (its baseline mood), and passes the total through an activation function (how sharply it commits to an answer).

The panel shows one neuron’s opinion about every possible input at once — amber on one side, blue on the other. Drag the knobs:

• The weights tilt the dividing line. Big w₁ means “input 1 decides.”
• The bias slides the line without turning it.
• The activation sets the edge: step is a hard verdict, sigmoid and tanh are graded confidence, ReLU only speaks up on one side.

Two things to hold onto. First, those knobs — the weights and bias — are the neuron’s parameters. When we say a model learns, we mean exactly this: its parameters change. Nothing else about it does.

Second, notice what one neuron can’t do: its boundary is always a straight line. However you set the knobs, it can only split the world in two with one cut. Keep that limitation in mind for the next panel.

A frontier LLM is made of the same ingredient. The difference is quantity: your browser is drawing one neuron’s three knobs; GPT-4-class models have on the order of a trillion.

The panel’s data is the classic XOR problem: blue lives in two opposite quadrants, amber in the other two. No single straight line can separate them — which means the neuron from last panel, alone, cannot solve this. This exact limitation stalled neural-net research for years.

The fix is embarrassingly simple: use more than one, in layers.

Your browser just trained a network with four hidden neurons feeding one output neuron. Look at the little thumbnails — each hidden neuron is still just a straight-line splitter, exactly like the last panel. Click one to see its view of the world, full size.

Then look at the combined map. The output neuron takes those four straight opinions, weighs them (with its own learned weights!), and blends them into a bent boundary that carves out all four quadrants correctly.

That’s the whole trick, and it’s the deepest fact in this track:

• One neuron: a line.
• One layer of neurons + a blend: corners and curves.
• Layers upon layers: any shape at all. This is (informally) the universal approximation property.

Depth composes. Early neurons learn simple cuts; later neurons combine cuts into shapes; still-later ones combine shapes into concepts. In the LLM tracks you saw the same story at scale — attention layers stacking 80+ deep, each refining the last layer’s work.

“Learn” implies “improve,” and “improve” implies a score. In machine learning that score is the loss: a single number that says how wrong the model currently is. Zero would be perfect; big is bad. The entire training process exists to push this one number down.

The panel gives you the simplest possible model — a straight line with two knobs, slope and offset — and eleven data points it should pass through. Every dashed pink bar is one prediction’s miss (the residual). The loss here is mean squared error: square every miss, average them.

Drag the knobs and watch the meter. Two things worth feeling out:

• Squaring punishes big misses hard. One awful prediction hurts more than five slightly-off ones. Get the line badly wrong for the rightmost point and watch the loss spike.
• There’s a floor you can’t reach. The points are noisy, so even the best line can’t hit zero. The meter turns teal when you’re near the best achievable — good is defined relative to the data, not as perfection.

One number to summarize millions of predictions may sound crude, and it is — choosing what to measure is one of the most consequential design decisions in all of AI. An LLM’s loss is “how badly did you predict the next token, averaged over the internet.” Everything ChatGPT is flows from pushing that number down.

Now: you can feel that some knob-directions lower the loss and some raise it. You just discovered the gradient — next panel makes it precise.

Our tiny neuron from Step 1, fit to twelve data points, has exactly two knobs: a weight w and a bias b. That means we can do something the textbooks usually fake: plot the true loss for every possible setting of both knobs at once. That’s the landscape in the panel — dark valley, amber ridges. It is not an artist’s impression; it’s computed live from the actual loss.

Click anywhere to drop the ball. At every point, the gradient says which way is uphill — so the ball steps the opposite way. That’s the entire algorithm:

knob ← knob − learning_rate × gradient

Repeat until the valley. That’s the “learning” every tab on this site keeps mentioning. It has a name you’ve now earned: gradient descent.

The learning rate slider sets the step size, and it’s the most temperamental dial in all of deep learning:

• Too small: the ball creeps; training takes forever.
• Just right: a brisk, confident roll to the bottom.
• Too big (try > 3): each step overshoots the valley and the ball rattles itself right out of the bowl. That failure is real, not staged — it’s the same divergence that makes real training runs explode into NaNs.

Honest asterisk: a real model has billions of knobs, so its landscape is a billion-dimensional object nobody can picture — saddle points, plateaus, strange winding valleys. The rolling ball is the true mechanism; a 2D bowl is usually the lie-to-children. Ours happens to be a true lie-to-children, because this neuron really does have just two knobs.

Gradient descent needs the gradient — a blame number for every knob. With two knobs you could find it by wiggling each one and re-measuring. With a billion, wiggle-and-check would take geologic time. Backpropagation computes all the blames in one backward sweep, and it’s why deep learning is possible at all.

The panel lays one neuron out as a computational graph — every multiply, add, and squash as its own node. Step through it:

• Forward (blue numbers): the input flows left to right, exactly the arithmetic from Step 1, ending in a loss.
• Backward (pink ∇ numbers): the loss asks the node before it, “how much did you move me?” — and that node asks its inputs, and so on, right to left. Each answer is the node’s local effect multiplied by the blame arriving from ahead of it. Calculus calls this the chain rule; you can read it as blame flows backward, scaled at every step.

Notice the division of labor when you reach the knobs:

• The add node passes blame through untouched.
• The multiply node scales blame by the other input — so a knob that was amplifying a big signal gets more blame than one riding a small signal.

When the sweep finishes, press apply the nudge: every knob moves against its gradient, and the loss drops. Forward, backward, nudge — you just did one lap of the training loop, by hand.

A real training step is this exact walk over a graph with billions of nodes, run by autograd machinery (PyTorch, JAX) instead of your thumb — millions of laps, batched thousands of examples at a time. Same walk. Bigger graph.

Time to let go of the crank. The panel is the whole loop — guess, measure, blame, nudge — running about 1,200 times per second, in your browser, on a network of 105 knobs. The painted map is what the network currently believes; the dots are the truth it’s trying to match.

Press train and watch the belief map organize itself out of nothing:

• XOR falls in a blink — you watched a 4-neuron net do it in Step 2.
• Circle takes a beat longer: the net has to bend a line into a ring.
• Spiral is the honest one. Watch the early epochs flail — the loss curve stalling, the boundary sloshing — before it suddenly catches, and the arms sharpen. That stretch of apparent futility is what most of training feels like at every scale.

Things worth trying:

• Reset and retrain. Same starting weights, same data, same result — training is deterministic here. (Real runs randomize; we seed everything so you and every other reader watch the identical learning curve.)
• Switch datasets mid-training and watch it adapt what it already built.

There is no model file behind this panel, no video, no server — just the arithmetic from the last five panels, repeated fast. If the loop can sculpt a spiral out of 105 knobs while you watch, you can believe the same loop, given 10¹¹ knobs and months of GPU time, sculpts a ChatGPT.

The loss curve’s shape — fast drop, long grind, occasional sudden improvement — is the same silhouette you saw in the reasoning track’s RL training scene. Different reward, same rolling-downhill machinery.

There’s a cheat that beats every training loss: memorize the answers. A big enough network can wrap a perfect little boundary around every training point — noise, mistakes and all — and drive its training loss to nearly zero while understanding nothing.

The defense is beautifully simple, and it’s the ritual behind every model you use: hide some data. In the panel, the network trains only on the solid dots; the hollow dots are the held-out test set it never sees. Two scores, two very different meanings:

• Train loss — how well it fits what it studied.
• Test loss — how well it generalizes to what it didn’t.

Now break it on purpose. Push the noise slider up and give the huge network its head. Watch the boundary grow ragged islands around individual mislabeled dots — and watch the two losses come apart: train keeps falling, test doesn’t follow. That gap is overfitting. The model isn’t learning the circle; it’s learning the noise.

Then try the same noise with the tiny network. It physically can’t memorize 75 points with 13 knobs, so it’s forced to learn the broad shape — often scoring better on the hollow dots than the huge net does.

The tension is real at every scale: frontier LLMs have enough capacity to memorize their training data verbatim, and a whole discipline — regularization, held-out evals, deduplication — exists to keep them generalizing instead. When you hear “the model was evaluated on held-out benchmarks,” this panel is what that sentence means.