Inside a language model.
Follow a prompt from text to the next token. Turn the 3D model, open its blocks and try the calculation with your own text. Unfamiliar words are explained in the AI glossary.
Drag to turn the model. Click it once, then scroll to zoom. On a phone, use two fingers. Click a part or its label to learn what it does. The shapes are a simplified picture. The number lab below runs a real calculation.
Number lab
What the model calculates for your text
A small Transformer runs here in your browser. Before the page was built, it learned its weights from 25 short sentences, so it continues texts like the examples well and anything else poorly. Nothing you type leaves your device.
Pick an example or type your own text, then ask for the next token.
Next-token probabilities
A separate training experiment
A classifier with one weight learns to predict a target of 1. Each step measures the error with cross-entropy and moves the weight by gradient descent. These buttons do not train the Transformer above.
Check yourself
Six short questions
Pick an answer to see a short explanation. Nothing is scored or saved.
Terms
Words used in this lab
No term matches. Try another word.
Further reading
From this model to real ones
Papers and documentation behind the lesson.