Transformers Explained Visually

TL;DR Highlight

Run GPT-2 in your browser to see Transformer internals and temperature in action

Who Should Read

An LLM API developer who tunes temperature and top-p by gut feel and wants to see what they actually do

Core Mechanics

Text generation is next-token prediction
- The model predicts the most probable next token, a word or part of a word.
- Its core idea is self-attention, which looks at the whole sequence and captures long-range links.
Input goes through four embedding steps
- The steps are tokenization, token embedding, positional encoding, and summing them into the final embedding.
- GPT-2 has a 50,257-token vocabulary and 768-dimensional vectors per token.
Attention works like a web search
- Query is the search text, Key is the page title, and Value is the page content.
- Query-Key dot products give scores, and a mask hides future tokens.
Attention and MLP do different jobs
- Attention moves information between tokens.
- The MLP refines each token separately, expanding 768 to 3072 and back to 768.
Temperature just divides the logits

Community Reactions

Commenters flagged differences from modern models
- One commenter noted GPT-2's absolute positional encoding is no longer used.
- They argued such details shape what representations a model can learn, so it shouldn't stand in for modern models.
Pushback on calling low temperature safe
- One commenter said safety is the wrong word in the temperature explanation.
- They said temperature 0 text lacks surprise and feels artificial.
A view of attention as a dynamic layer
- One commenter said multiplying the attention matrix by Value acts like a Dense layer whose weights come from the attention matrix.
- So each head builds a small single-layer network at inference time from Key and Query.
Reports of heavy performance cost
- One user said the page crashed their Chromebook.
- Another said the tab used 2.2 GB of RAM within 10 seconds and slowed their laptop.

How to Apply

Compare temperatures on one prompt
- Today, enter a prompt in Transformer Explainer and set temperature to 0.2, 0.8, and 1.5 to watch the distribution change.
- Then vary top-k and top-p to see candidate tokens shrink.
Pick sampling values per use case
- A/B test low temperature for extraction and classification calls and higher values for text generation.
- Don't assume low temperature is safe; check whether the output gets too flat.
Add the visualization to onboarding
- Show the site to teammates new to LLMs to align on token, embedding, and attention terms.
- Tell them it is GPT-2 based and explain separately where modern models differ.

Try It

copy & run

# Paste into Claude or ChatGPT (a simple experiment based on the article's concepts)

You are an LLM sampling explainer. Generate 3 answers to the same question under each setting below.

Question: {{question}}
Setting A: temperature 0.2 (sharp distribution, predictable output)
Setting B: temperature 0.8
Setting C: temperature 1.5 (flat distribution, more randomness)

Compare how similar the answers are within each setting. Then tell me which value suits my service ({{service description}}) and why.

Next step: repeat the comparison with top-p set to 0.5 and 0.95.

Terminology

TokenThe smallest unit a model reads. It can be a whole word or a piece of one.

EmbeddingA token converted into a numeric vector. Tokens with similar meaning sit close together.

Self-attentionA way for each token to compute how much to look at other tokens, like deciding whose words to listen to in a meeting.

LogitA per-token score before softmax. It is converted into a probability to pick the next token.

Top-p samplingKeeps tokens from most probable downward until their cumulative probability passes p.