Image Diffusion.

The model can’t read characters the way you do. Before anything else, your text is split into tokens — chunks that are often whole words, sometimes word-pieces, sometimes single characters.

This is done with byte-pair encoding (BPE): start from raw bytes and repeatedly merge the most common adjacent pair into a new token. Frequent sequences like " the" become a single token; rare ones get fragmented.

Try it → type in the panel. A few things worth discovering:

• Try “strawberry”, with and without a leading space. The split depends on the tokenizer. Tokens do not usually correspond one-to-one with letters; that can make character-level tasks awkward, but it is not a complete explanation of every counting error.
• A leading space is part of the token: " Paris" and "Paris" are different tokens.
• Numbers, code, emoji, and other languages tokenize very differently. Toggle GPT-4 (~100k vocab) vs GPT-4o (~200k) and compare the counts. A larger vocabulary need not use fewer tokens for every string, and token count alone does not determine capability or cost.

Each token is really just an integer — its row number in the model’s dictionary. These IDs select token embeddings; the model also uses positional information and, in some systems, other inputs.

A bare token ID like 2746 means nothing on its own. So the model looks it up in a giant table and swaps it for a long list of numbers — a vector. For GPT-2 small that’s 768 numbers per token; modern models use thousands.

These vectors are learned (in the gradient-descent sense — the How Models Learn track shows the machinery). Relationships can appear in learned representations, but distance depends on the model, layer, and comparison. In this illustrative panel, each dot is a word; click one to light up its nearest neighbors — dog sits by puppy and kitten, far from algorithm.

Because meaning lives in directions, you can sometimes do arithmetic on words. Toggle the analogy: king − man + woman points toward queen in the authored example.

Honest asterisk: this famous example is hand-picked. It only works cleanly when you exclude the input words, and it describes static word vectors — a real LLM’s internal representations are contextual and messier. It’s a beautiful hint that meaning is geometric, not a hard law.

One thing embeddings don’t carry yet is order — “dog bites man” and “man bites dog” would look the same. Models incorporate positional information; some add position embeddings, while others encode relative positions within attention.

The prompt you just tokenized and embedded now flows through a Transformer text encoder. In Stable Diffusion that’s CLIP; newer models (SD3, FLUX) add a big T5 encoder too.

It’s the same self-attention you met in the LLM track — tokens looking at each other to build context‑aware meaning. (The panel shows attention on an example sentence; the shared heatmap is illustrative, not a measured text-encoder run.) An important difference is the output: instead of a probability over the next word, the encoder produces a sequence of conditioning vectors — a compact “meaning of your prompt” that the image engine will follow at every step.

For SD 1.5 that’s 77 vectors, each 768 numbers. Hold onto them — in a few steps they’ll reach into the image through cross-attention.

Now we leave familiar territory. The next steps are the image engine — and they’re nothing like predicting a token.

A 512×512 color image is 786,432 numbers. Running dozens of denoising passes directly on that many pixels would be brutally expensive.

The trick that made Stable Diffusion practical is latent diffusion: a variational autoencoder (VAE) compresses the image into a tiny 64×64×4 grid — just 16,384 numbers, about 48× smaller. All the denoising happens in that little space. Only at the very end does a VAE decoder blow the finished latent back up into full pixels.

Watch the encode/decode in the panel. The whole expensive loop we’re about to build runs on the small block in the middle — that reduces work; runtime also depends on model size, hardware, resolution, and sampling choices.

(Newer models like SD3 and FLUX use a richer 16‑channel latent for more detail.)

This is the signature move of a diffusion model — and the payoff of the whole track. Grab the slider in the panel and drag from noise to image, or hit Denoise ▶.

During training, the model learns by doing the opposite: take a real image, add noise step by step until it’s pure static, and learn to predict the noise that was added. At generation time you run that in reverse — start from random noise and, step by step, predict-and-subtract the noise until a clean image is left.

How many steps? That’s the scheduler:

• DDPM — the original, up to ~1000 stochastic steps.
• DDIM can use deterministic sampling and fewer steps; results depend on the model and settings.
• LCM / Turbo-style distillation can enable very few steps in compatible models. The panel sketches the speed-quality trade-off; it is not a benchmark.

Every step needs one thing to know what to remove — and that’s where your prompt comes back in. Two questions remain: what network does the predicting, and how do your words steer it?

At every denoising step, one network does the work: it takes the noisy latent, the timestep (how far along we are), and the text conditioning, and predicts the noise to remove. Toggle between the two designs in the panel:

• U-Net (Stable Diffusion 1.x/2.x, SDXL): a convolutional encoder–decoder with skip connections — it shrinks the latent down to capture the big picture, then builds it back up with fine detail, ResNet and attention blocks throughout.
• DiT / MMDiT (SD3, PixArt, FLUX): chop the latent into patches, treat them as tokens, and run Transformer blocks — reusing transformer ideas, with different conditioning, masking, and architecture choices. The timestep is injected by adaptively scaling each layer (adaLN).

Both families have been useful. Scaling behavior, task, and implementation affect the choice; neither diagram covers every image generator. Either way, the steering comes from your prompt — next.

Remember those conditioning vectors from the text encoder? Here’s where they do their job. In the latent-diffusion design illustrated here, cross-attention connects image features to text conditioning.

In the LLM track, self-attention had tokens attending to other tokens — Queries, Keys, and Values all came from one sequence. Cross-attention is the same math (Q·Kᵀ → softmax → weighted V) with one twist:

• the image latent provides the Queries (“what should this region look like?”), and
• the text embeddings provide the Keys and Values (“here’s what the words mean”).

So each patch of the image “looks up” which words are relevant and pulls in their meaning. Hover the prompt words in the panel: “balloon” lights up the balloon, “hills” the hills. (The regions here are illustrative, not measured maps or a guarantee of exact word-to-region control.)

One knob remains: how hard the model should listen to your prompt.

In this classic classifier-free guidance setup, the model predicts noise under two conditioning settings: once with your prompt (conditional) and once with an empty prompt (unconditional). Then it extrapolates between them:

ε = ε_uncond + w · (ε_cond − ε_uncond)

That w is the guidance scale, and it is one useful control among several. Drag it in the panel:

• w = 0 — ignores your prompt entirely.
• w = 1 — follows it, softly.
• w ≈ 7 — a moderate illustration here, not a universal recommended value.
• w ≈ 20 — high guidance can produce artifacts in some setups.

A negative prompt is the same trick, aimed backwards: swap the “empty” branch for things you don’t want (“blurry, extra fingers”), and the update changes relative to that alternative conditioning. It does not guarantee that an unwanted feature disappears.

That is our Stable Diffusion-style example, end to end: a text encoder turns the prompt into representations that condition repeated image updates. You have met some familiar transformer machinery, used for a different job. Other image models can use different encoders, prediction targets, and guidance methods.