Reasoning LLM.

The model can’t read characters the way you do. Before anything else, your text is split into tokens — chunks that are often whole words, sometimes word-pieces, sometimes single characters.

This is done with byte-pair encoding (BPE): start from raw bytes and repeatedly merge the most common adjacent pair into a new token. Frequent sequences like " the" become a single token; rare ones get fragmented.

Try it → type in the panel. A few things worth discovering:

• Try “strawberry”, with and without a leading space. The split depends on the tokenizer. Tokens do not usually correspond one-to-one with letters; that can make character-level tasks awkward, but it is not a complete explanation of every counting error.
• A leading space is part of the token: " Paris" and "Paris" are different tokens.
• Numbers, code, emoji, and other languages tokenize very differently. Toggle GPT-4 (~100k vocab) vs GPT-4o (~200k) and compare the counts. A larger vocabulary need not use fewer tokens for every string, and token count alone does not determine capability or cost.

Each token is really just an integer — its row number in the model’s dictionary. These IDs select token embeddings; the model also uses positional information and, in some systems, other inputs.

A bare token ID like 2746 means nothing on its own. So the model looks it up in a giant table and swaps it for a long list of numbers — a vector. For GPT-2 small that’s 768 numbers per token; modern models use thousands.

These vectors are learned (in the gradient-descent sense — the How Models Learn track shows the machinery). Relationships can appear in learned representations, but distance depends on the model, layer, and comparison. In this illustrative panel, each dot is a word; click one to light up its nearest neighbors — dog sits by puppy and kitten, far from algorithm.

Because meaning lives in directions, you can sometimes do arithmetic on words. Toggle the analogy: king − man + woman points toward queen in the authored example.

Honest asterisk: this famous example is hand-picked. It only works cleanly when you exclude the input words, and it describes static word vectors — a real LLM’s internal representations are contextual and messier. It’s a beautiful hint that meaning is geometric, not a hard law.

One thing embeddings don’t carry yet is order — “dog bites man” and “man bites dog” would look the same. Models incorporate positional information; some add position embeddings, while others encode relative positions within attention.

To understand a word you need context. In “The animal didn’t cross the street because it was too tired,” what does “it” refer to? Self-attention lets each token look at the other tokens and decide how much to pull from each.

Mechanically, every token produces three vectors: a Query (“what am I looking for?”), a Key (“what do I offer?”), and a Value (“what I’ll hand over”). A token’s query is compared against every key; the scores go through a softmax into weights that sum to 1; the output is that weighted blend of Values.

Do it for every pair and you get the attention matrix in the panel — row i is how much token i attends to each other token. Notice the greyed upper triangle: a token may only look backward, never at words it hasn’t generated yet. That causal mask is what makes the model a left-to-right predictor. Hover any cell to read the weight.

Two things to keep in mind as we simplify:

• Real models run many attention heads in parallel (GPT-2 small: 12), each learning a different kind of relationship, plus a feed-forward layer that applies further learned transformations. Knowledge is not neatly isolated in just one component.
• This whole block is stacked many times (12 in GPT-2 small; a model-dependent number in larger systems). The token’s vector gets refined at every layer.

After the last block, the model is ready to turn that vector back into a word.

After the last block, the model takes the final token’s vector and scores every token in its vocabulary — about 50,000 scores in GPT-2, and a different count in other models. Those scores are called logits. A softmax squashes them into probabilities that sum to 100%. The panel computes a distribution from authored example scores, not a live model prediction: “87% chance the next token is Paris, 3% the, …”

How do we actually pick one? That’s what the controls in the panel do:

• Greedy — always take the top token. Deterministic for fixed scores, but not guaranteed correct or safe.
• Temperature — rescales the logits. Drag it down and one bar dominates (conservative, near-deterministic at 0). Drag it up and the bars flatten (more diverse, riskier). Temperature 0 does not mean “more true” — just more predictable.
• Top-p (nucleus) — keep only the smallest set of tokens whose probability adds up to p, then sample from those. The struck-through bars are the tail it’s ignoring.

Press Sample! a few times to feel the randomness — then set temperature to 0 and watch it become deterministic.

Then the crucial move: whatever token we pick gets appended to the sequence, and the whole pipeline runs again to produce the next one. This autoregressive loop — one token at a time — is how a whole paragraph gets written.

And it’s exactly this loop that a reasoning model turns into something more.

A model can be prompted or trained to produce intermediate steps before the final answer. Ordinary language models can do this too; it is not exclusive to specialized reasoning models.

Chain-of-thought is one approach: the model generates intermediate reasoning steps before the final answer. Research found benefits on selected multi-step tasks; results depend on the model, prompt, and evaluation.

Toggle it in the panel and watch two things happen at once:

• The scripted example reaches a correct answer with intermediate steps. It illustrates a scratchpad; it does not prove the same result on a new task.
• The token counter climbs. That’s the real mechanism: every token the model writes is another full pass through the network. A longer trace literally buys the model more computation to spend before committing.

That second point is the whole game. If thinking longer helps… what happens if we let it think much longer?

Training compute changes the model. Test-time compute is work spent while solving a particular task: generating a longer trace, sampling several candidate solutions, revising an answer, or checking intermediate steps.

More work can help. It can also be wasted, amplify a mistake, or yield diminishing returns. The best allocation depends on the model, task difficulty, strategy, and quality of the feedback. A correct-answer checker is useful when available; it is not a universal requirement for any benefit from extra computation.

Drag the budget slider. The curve is invented for teaching—it illustrates a possible trade-off, not a measured accuracy forecast. The verifier toggle is a simplification of how feedback quality can matter, not an experimental comparison.

Keep two questions separate:

• What changed during training? DeepSeek-R1-Zero’s reported improvement on AIME 2024—from 15.6% to 71.0% pass@1—occurred during reinforcement learning.
• What happens if inference gets a bigger budget? That needs a comparison holding the trained model and evaluation conditions fixed.

The test-time compute study investigates adaptive strategies. Its results are much more interesting than “turn the knob and correctness goes up forever.”

The important move is simple: generated intermediate tokens become part of the context for later tokens. A model can produce a calculation, inspect a candidate answer, or generate text that revisits a previous step before its final response.

This uses the autoregressive machinery we’ve already explored. It does not require a new architecture solely because we call it reasoning; specific models can still differ in architecture, training, and inference systems.

Press Run both. These are two scripted teaching examples, not live measurements. One answers incorrectly; the other uses intermediate steps and answers correctly. That contrast explains the mechanism, not a rule that ordinary chat models fail and reasoning models succeed.

A generated trace can contain useful calculations and also misleading rationalizations. “Reads its thoughts” is an informal way to describe reusing generated context. It does not establish consciousness, intention, or a faithful record of all the computation inside the network.

Some systems expose reasoning text; others expose summaries or no trace at all. Interfaces, budgets, and billing depend on the particular product and version. The durable idea is the extra computation—not one vendor’s current setting names.

One approach is reinforcement learning from verifiable rewards: generate candidate solutions, score outcomes that can be checked, and update the model to favor better-scoring behavior. The parameter updates connect back to How Models Learn, although an RL objective is not simply the raw reward treated as a differentiable loss.

The original DeepSeek-R1 report distinguishes two systems. R1-Zero applied RL to an already pretrained base model without an initial supervised fine-tuning stage. R1 used cold-start examples and a multistage pipeline combining supervised and reinforcement learning.

The slider illustrates R1-Zero’s reported training trend; it is not training that model here. Accuracy and response length changed together, but that alone does not isolate the effect of longer responses.

The authors observed generated text that revisits an earlier step. Their “aha moment” label describes an output pattern, not evidence of subjective insight. The procedure encourages behavior through rewards, not a hand-written rule for every solution.

GRPO compares sampled responses within a group without PPO’s separate value-model critic. Verifiable rewards can make assessment easier, but checks can be incomplete and rewards can still be exploited. Distillation can transfer useful behavior to smaller models, with results that depend on the training and evaluation—not a promise of identical capability.

Once a model can reason, there are many ways to use that ability at inference time — and each has a different shape. Flip through them in the panel:

• Greedy chain of thought — one line of reasoning, straight through.
• Self-consistency — sample many independent chains and take the majority vote. Different paths, same destination.
• Tree of thoughts — branch at each step, evaluate partial solutions, prune dead ends, and backtrack. On the “Game of 24” puzzle, GPT-4 solved 4% with plain chain-of-thought versus 74% with tree search.
• Reflection — a genuine loop: generate an answer, critique it, revise, repeat.

Which is best depends on the problem: research suggests you should revise on easy problems and search wider on hard ones. Reasoning models bake a lot of this behavior directly into their single long chain of thought — but the same ideas power the scaffolds built around them.

A larger inference budget can mean more generated tokens, more candidate solutions, or more checking. It can also mean more time and cost. Whether the extra work improves the result depends on the task and the system.

The meters in this panel are an illustration, not prices, timings, or benchmark predictions. Treat them as a way to explore a trade-off. A hard problem may benefit from a larger budget; a simple one may not.

Some APIs offer an effort level or token budget. The supported controls change across models and releases. This guide focuses on the concept rather than promising a portable set of parameter names.

The s1 study explored “budget forcing,” including appending “Wait” to extend a response in its experimental setup. That is evidence about a particular method, not a universal spell for better answers. Try the panel’s scripted version and notice what it leaves out: measurement, repeated trials, and checking the answer.

Next, the question hiding behind every neat-looking trace: how much of the reasoning can we trust?

It’s tempting to read a model’s chain of thought as the reason it gave an answer. Be careful — it often isn’t.

In the panel, a model is given a question with a planted hint. Its chain of thought produces an eloquent, confident rationale… that never mentions the hint — even though the hint is what actually drove the answer. Reveal what really happened.

Anthropic measured this directly. When given hints, models mentioned the decisive hint in their reasoning only about 25% of the time in that evaluation (Claude 3.7 Sonnet) and 39% (DeepSeek-R1). In a reward-hacking setup, models exploited the hack in over 99% of cases but admitted it less than 2% of the time — fabricating plausible-sounding rationales instead.

The chain of thought is a genuinely useful, generated artifact — great for catching errors and for oversight. But it is not a guaranteed transcript of the model’s real computation. Keeping it readable enough to monitor is an open safety question.

These rates describe the tested models, hints, and evaluation conditions. They are not universal odds that a model is honest. The panel is a written illustration; the linked research contains the actual experiment.

You’ve now seen the whole pipeline — tokenize, embed, attention, sample — and the intermediate-token loop used in our reasoning examples. Time to see the real thing.

Click Load GPT-2 in the panel and a genuine 124-million-parameter model downloads into your browser (a substantial download, often around 100 MB or more depending on the backend, with browser caching) and runs entirely on your machine — your prompt is processed locally. Download requests still contact the CDN and model-file hosts; local inference is not the same as no network traffic. Then type a prompt and watch it generate.

GPT-2 is small and from 2019, so its output is charmingly rough compared to a frontier model. But it is a concrete decoder-transformer example of the ideas you just learned, running for real — every token it emits is the exact tokenize → embed → attention → sample loop, one step at a time.

The other tracks reuse useful ideas while showing their limits: chat adds post-training, mixture-of-experts changes selected feed-forward layers, and image diffusion uses a different generation process. These are learning paths, not a claim that every model is GPT-2 plus one extra part.