Neural networks are clean

TL;DR Highlight

Looking at neurons directly shows most have one main job, challenging superposition.

Who Should Read

LLM app developers curious about how models work inside and where interpretability research is heading

Core Mechanics

Superposition was the wrong explanation
- The idea that neurons fold unrelated concepts together (superposition) was wrong for InceptionV1, the author says.
- The author points to recursive generalization instead.
- Neurons grow like tree branches into adjacent concepts.
99-100% of InceptionV1's last layer is interpretable
- Earlier work found almost no interpretable neurons in that layer.
- Ordinary architecture improvements made 99-100% easily interpretable, with no change to model size or training data.
- Versions trained across a 300× size range stay broadly legible.
The cat/car neuron was misread
- The famous cat/car neuron is ordinary generalization and composition, not arbitrary mixing.
- The shared pattern it reads is a V shape.
- You must visualize what the neuron's inputs have in common, not the neuron alone.
Qwen3-32B preliminary audit
- About 90% of neuron poles have one main job.

Community Reactions

Pushback on overreach
- One commenter said the claims go beyond what is proven.
- They noted progressive feature complexity is long known and not mutually exclusive with parameter overloading.
Emergence may hide in the remainder
- One commenter argued the unexplained 2-4% could hold the hubs of emergent cognition.
- Then understanding 96-98% would be worth less than the author thinks.
Call for a correction of ZoomIn
- A commenter said calling the V neuron a dog-and-car neuron was a plain error.
- They argued it may have misled researchers for years and warrants a correction or retraction.
TLDR from a commenter
- The summary was that the idea of neurons bundling unrelated concepts was a methodological error.
- Grouping related concepts was right all along.
Questions on method and readability

How to Apply

Spend 30 minutes in the original microscope
- Open the InceptionV1 microscope at already-clean.goodfire.com and inspect a few neurons.
- Check where each fires strongly and write a one-line description.
Separate evidence levels when reading interpretability claims
- Check model size, measurement criteria, and sample size together.
- Note that the 90% figure comes from a preliminary Qwen3-32B audit.
State your weak-activation threshold up front
- Decide whether you count only strong activations or weak ones too before analysis.
- Document the threshold so results can be compared.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the essay's method: verifying neuron explanations)

Below are text examples where a neural network neuron activates most strongly.
{{activation examples}}

Answer in this order.
1. Write a one-sentence hypothesis for this neuron's job.
2. Find examples the hypothesis fails to explain.
3. Check whether the unexplained ones come from context such as next-word prediction or punctuation position.
4. Rewrite the hypothesis in a more general form.

Next to try: add weak-activation examples {{weak activation examples}} and compare how the hypothesis changes.

Terminology

polysemanticityA neuron that seems to have several unrelated jobs. Like one switch that turns on both a light and a fan.

superpositionThe hypothesis that a network folds unrelated concepts into one neuron when it runs out of room. This essay argues it's wrong.

recursive generalizationA neuron grows offshoots into adjacent concepts the way a branch sprouts sub-branches.

sparse autoencoderA second model trained to approximate the network in an unfolded form so features can be studied.

attribution graphA graph showing how data flows through neurons to influence an output.