TokenTown: A visual way to understand how LLMs work

A browser city sim shows one token's round trip through a transformer.

A backend developer who calls LLM APIs but has no feel for how tokens are processed inside the model

A transformer drawn as a city
- Each district is one stage of the model.
- A convoy moves through tokenizing, embedding, positional encoding, the layer ring and the sampler.
The layer ring repeats per layer
- One lap is attention, residual, feed-forward, residual.
- At the end it makes a probability distribution, picks one token and loops back.
Parts are really computed
- Tokenizer split, embedding lookup, causal attention, GELU feed-forward and temperature/top-p sampling run live in the browser.
- Attention runs over a real growing KV cache.
- Prefill processes all prompt tokens at once while decode processes one.
Scale is shrunk
- 12 dimensions instead of thousands, 2 heads instead of dozens.
- 2–12 layers instead of 80, a few hundred words instead of 100k+.

Other visualizers preferred
- A commenter said Brendan Bycroft's LLM visualiser (bbycroft.net/llm) helped them most.
- Another wanted an interactive 3Blue1Brown-style version.
Criticism of the explanation text
- Commenters said the copy reads like generic Claude output heavy with jargon.
- Another asked to state the prerequisites.
A factual inconsistency pointed out
- One comment noted the text says it is not a loop but a stack of blocks, yet the visual loops 6 times.
Usability complaints
- One asked for mobile support.
- Others called it unhelpful or asked to filter vibe-coded projects from the homepage.

Watch prefill vs decode in TokenTown
- Enter a prompt and press Run to see prefill process all tokens at once.
- Compare with decode, which handles one token at a time.
- Use it to explain why long prompts delay the first response.
Feel sampling with the Temperature and Top-p sliders
- Run the same prompt while changing Temperature.
- See how output variety shifts before setting API defaults.
Use it for onboarding with a backup resource
- Show TokenTown first.
- Pair it with bbycroft.net/llm since the explanations are thin.

KV cacheA memory of the keys/values of tokens already read. The model reuses it instead of recomputing earlier text.

prefill / decodePrefill reads the whole prompt at once. Decode produces answer tokens one by one.

residual streamThe shared lane that carries values through layers. Each block adds its result to it.

top-p samplingKeep the likeliest candidates until their probability sums to p, then sample among them.

attention sinkThe tendency of attention to pile onto the first token.