Show HN: LLM Attention Visualization

Who Should Read

An LLM API or RAG developer who lacks intuition for why models copy source text so accurately.

Core Mechanics

• Hover shows which past tokens fed each generated token
  • Hovering a generated token highlights the past tokens that influenced it.
  • The value is attention weight scaled by value vector magnitude, summed across all heads and layers.
  • The largest value gets opacity 1 and the rest are interpolated.
• It is a heavily simplified view
  • Much information was discarded to keep one number per past token.
  • The author first doubted it would be comprehensible, but it produced interesting patterns.
• It shows why copying is accurate
  • For verbatim text like addresses and dates, the source data stands out strongly.
  • The model doesn't predict the whole sequence from limited internal state but draws from past tokens directly.
  • So the chance of copy errors can be very low.
• A 600M-parameter model copies a whole function
  • In the 'Debugging an Average Function' example it reproduced a JS function except for the intended change.

Community Reactions

• Praised as a teaching tool
  • A commenter teaching this topic said the weighting scheme alone doesn't build intuition and the visualization helps a lot.
  • Another said it was the clearest example after reading books and watching videos.
• UX complaints about playback controls
  • One report says pause resets the animation on Chromium/Linux instead of pausing.
  • Commenters suggested slow playback or single-step buttons.
• Doubts about the method
  • One commenter questioned equating high vector magnitude with high influence.
  • Another asked whether summing across layers lets the more numerous early layers drown out later ones.
• Compared with Bertviz
  • One commenter found it simpler than Bertviz and good for beginners.

How to Apply

• Ask for verbatim copying explicitly
  • Spend 30 minutes on the demo comparing copied spans with summarized ones.
  • Tell the model to quote exact values like addresses, dates, and IDs verbatim.
• Put exact values directly in context
  • In RAG, pass raw chunks for numbers or code that must not be wrong instead of summaries.
  • This lets the model draw straight from past tokens.
• Check the repo if you need internal visualization
  • Look at the onnx modification approach in the GitHub repo.
  • Host the instrumented model separately since stock onnx models hide internals.

Try It

# Paste into Claude or ChatGPT (simplified version of the idea: copy-accuracy check)

Read the {{document}} below and answer {{question}}.

Rules:
1. Copy addresses, dates, numbers, and IDs exactly from the document and put them in quotes.
2. For anything inferred by combining sentences, say which sentences you combined.

[Document]
{{document}}

[Question]
{{question}}

Next: diff the quoted parts against the source to confirm they match character for character.

Terminology

Related Papers

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

• Show HN: Gemini 4 Argon plays StarCraft

  LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.

• A Plausible Explanation Is Enough to Mislead Multimodal Judges

  Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.

• Neural networks are clean

  Looking at neurons directly shows most have one main job, challenging superposition.

Related Resources

• https://ishamf.dev/p/llm-attention-visualizer/