The model can’t read characters the way you do. Before anything else, your text is split into tokens — chunks that are often whole words, sometimes word-pieces, sometimes single characters.

This is done with byte-pair encoding (BPE): start from raw bytes and repeatedly merge the most common adjacent pair into a new token. Frequent sequences like " the" become a single token; rare ones get fragmented.

Try it → type in the panel. A few things worth discovering:

• “strawberry” splits into pieces (straw + berry). The model never sees the individual r’s — which is exactly why “how many R’s in strawberry?” is weirdly hard for it. Counting letters isn’t a native operation.
• A leading space is part of the token: " Paris" and "Paris" are different tokens.
• Numbers, code, emoji, and other languages tokenize very differently. Toggle GPT-4 (~100k vocab) vs GPT-4o (~200k) and watch the same text use fewer tokens with the bigger vocabulary — that’s cheaper and better at other languages.

Each token is really just an integer — its row number in the model’s dictionary. That list of integers is the only thing the network receives.

A bare token ID like 2746 means nothing on its own. So the model looks it up in a giant table and swaps it for a long list of numbers — a vector. For GPT-2 small that’s 768 numbers per token; modern models use thousands.

These vectors are learned (in the gradient-descent sense — the How Models Learn track shows the machinery). Tokens used in similar contexts end up near each other in this space, so distance and direction encode meaning. In the panel, each dot is a word; click one to light up its nearest neighbors — dog sits by puppy and kitten, far from algorithm.

Because meaning lives in directions, you can sometimes do arithmetic on words. Toggle the analogy: king − man + woman lands right on queen.

Honest asterisk: this famous example is hand-picked. It only works cleanly when you exclude the input words, and it describes static word vectors — a real LLM’s internal representations are contextual and messier. It’s a beautiful hint that meaning is geometric, not a hard law.

One thing embeddings don’t carry yet is order — “dog bites man” and “man bites dog” would look the same. Models fix that by adding positional information to each vector before the next step.

The prompt you just tokenized and embedded now flows through a Transformer text encoder. In Stable Diffusion that’s CLIP; newer models (SD3, FLUX) add a big T5 encoder too.

It’s the same self-attention you met in the LLM track — tokens looking at each other to build context‑aware meaning. (The panel shows attention on an example sentence; a prompt works exactly the same way.) The one difference is what comes out the end: instead of a probability over the next word, the encoder produces a sequence of conditioning vectors — a compact “meaning of your prompt” that the image engine will follow at every step.

For SD 1.5 that’s 77 vectors, each 768 numbers. Hold onto them — in a few steps they’ll reach into the image through cross-attention.

Now we leave familiar territory. The next steps are the image engine — and they’re nothing like predicting a token.

A 512×512 color image is 786,432 numbers. Running dozens of denoising passes directly on that many pixels would be brutally expensive.

The trick that made Stable Diffusion practical is latent diffusion: a variational autoencoder (VAE) compresses the image into a tiny 64×64×4 grid — just 16,384 numbers, about 48× smaller. All the denoising happens in that little space. Only at the very end does a VAE decoder blow the finished latent back up into full pixels.

Watch the encode/decode in the panel. The whole expensive loop we’re about to build runs on the small block in the middle — that’s the entire reason a modern image model can run in seconds instead of minutes.

(Newer models like SD3 and FLUX use a richer 16‑channel latent for more detail.)

This is the signature move of a diffusion model — and the payoff of the whole track. Grab the slider in the panel and drag from noise to image, or hit Denoise ▶.

During training, the model learns by doing the opposite: take a real image, add noise step by step until it’s pure static, and learn to predict the noise that was added. At generation time you run that in reverse — start from random noise and, step by step, predict-and-subtract the noise until a clean image is left.

How many steps? That’s the scheduler:

• DDPM — the original, up to ~1000 stochastic steps.
• DDIM and modern schedulers — deterministic, great quality in 20–50 steps.
• LCM / Turbo — distilled for 1–4 steps: watch the tradeoff in the panel, fast but a little rougher.

Every step needs one thing to know what to remove — and that’s where your prompt comes back in. Two questions remain: what network does the predicting, and how do your words steer it?

At every denoising step, one network does the work: it takes the noisy latent, the timestep (how far along we are), and the text conditioning, and predicts the noise to remove. Toggle between the two designs in the panel:

• U-Net (Stable Diffusion 1.x/2.x, SDXL): a convolutional encoder–decoder with skip connections — it shrinks the latent down to capture the big picture, then builds it back up with fine detail, ResNet and attention blocks throughout.
• DiT / MMDiT (SD3, PixArt, FLUX): chop the latent into patches, treat them as tokens, and run Transformer blocks — literally the same engine as an LLM, repurposed. The timestep is injected by adaptively scaling each layer (adaLN).

The field is steadily moving from U-Nets toward Transformers, for the same reason LLMs did: they scale beautifully. Either way, the steering comes from your prompt — next.

Remember those conditioning vectors from the text encoder? Here’s where they do their job. At every denoising step, the image talks to your prompt through cross-attention.

In the LLM track, self-attention had tokens attending to other tokens — Queries, Keys, and Values all came from one sequence. Cross-attention is the same math (Q·Kᵀ → softmax → weighted V) with one twist:

• the image latent provides the Queries (“what should this region look like?”), and
• the text embeddings provide the Keys and Values (“here’s what the words mean”).

So each patch of the image “looks up” which words are relevant and pulls in their meaning. Hover the prompt words in the panel: “balloon” lights up the balloon, “hills” the hills. (The regions here are illustrative, but this is genuinely how the prompt controls where things go.)

One knob remains: how hard the model should listen to your prompt.

At each step the model actually predicts the noise twice: once with your prompt (conditional) and once with an empty prompt (unconditional). Then it extrapolates between them:

ε = ε_uncond + w · (ε_cond − ε_uncond)

That w is the guidance scale, and it’s the single most important knob you touch as a user. Drag it in the panel:

• w = 0 — ignores your prompt entirely.
• w = 1 — follows it, softly.
• w ≈ 7 — the usual sweet spot: strong adherence, still natural.
• w ≈ 20 — over-cooked: rigid, oversaturated, fewer surprises.

A negative prompt is the same trick, aimed backwards: swap the “empty” branch for things you don’t want (“blurry, extra fingers”), and the image is actively pushed away from them.

And that’s a text-to-image model, end to end: the same tokenize → embed → Transformer front end as an LLM, handed to a different engine that denoises a compressed image, steered by your words.