The 'Astra can read and write weights to improve Qwen' post is misleading

Who Should Read

Developers weighing whether LLMs editing model weights directly is realistic for their own work

Core Mechanics

• Only 3 experiments used raw weights alone
  • Experiments 1, 2 and 4 put raw weights in context with no tools and no running the network.
  • Those networks have 17K to 172K weights.
• Every real-LLM experiment used Python
  • All experiments involving a real language model use Python.
  • The headline Qwen result also ran the model on up to 50 batches of practice problems.
• Qwen went from 7.9% to 48.1%
  • Qwen3.5-0.8B's MATH-500 rose from 7.9% to 48.1% at the same sampling settings.
  • Edits touched 0.016% of weights, and IFEval rose 3x.
• It failed on Gemma 3 270M
  • Spotting the sycophantic Gemma with Python alone failed.
  • Adding attention patterns gave 3/3 correct.
  • The Qwen-style maths fix also failed on Gemma.
• It shows Astra can do ML engineer work
  • Human ML engineers also write code to analyse and edit weights.
  • Doing this well matters, but it was not a single technique doing both.
• Game network results had caveats
  • On MinAtar, it beat PPO-500M only on Breakout (517 vs 25).
  • It lost on Seaquest (53 vs 89) and Asterix (53 vs 55).

Community Reactions

• A mech interp researcher pushed back
  • They said the key result is fewer reasoning loops, not better maths.
  • They said steering vectors are an intro-level interp project that takes 4-8 hours for someone who can code and knows LLM internals.
• A quoted Neel Nanda comment
  • The quote says Astra is good at old-school mech interp research, not anything spooky.
  • It adds that this style of analysis is hard to scale to real LLMs.
• A user said it worked in practice
  • They built a 17M reranker from zero weights with Astra and replaced a 68M model.
  • They report stylus tool recognition accuracy going from 88% to 96%.
• Some said the original post already said all this
  • Commenters argued the original explained these limits, so it was not misleading.
  • One said the clickbait around it caused the confusion.

How to Apply

• Ask an agent to diagnose a small model's loops today
  • Pick a model with repetition issues and give Claude Code its weights and failure samples.
  • Start with the prompt below.
• Check experiment tables before trusting AI claims
  • Split the headline into per-experiment conditions: tools, model runs, target size.
  • Check whether the method worked on other models under the same conditions.
• Try weight edits before training small models
  • For small classifiers or rerankers, let an agent try weight edits instead of GPU training.
  • Compare before and after on the same eval set.

Try It

# Paste into Claude Code (simplified version of the post's method: analysis via code execution)

You are an ML engineer. My model is at {{model path}}, and the problem is {{e.g. reasoning never terminates and loops}}.
Failure samples are in {{failure output file}}.

Write Python code yourself and do the following.
1. Contrast normal outputs with failing outputs.
2. Find the layers or directions that likely cause it.
3. Propose how much to change the weights.
4. Compare metrics before and after on {{eval set}}.

Next step: ask for a report on the fraction of weights changed and any side effects.

Terminology

Related Papers

• Qwen 3.8 27B on my laptop can interpret model weights, too

  Local Qwen 3.8 27B identified 3 of 4 games from raw network weights alone

• Google will turn off search for schools that reject AI

  Reports say schools turning off AI also lose Google Search, and commenters are split

• Launch HN: Momentic (YC W24) – Agents that catch bugs by using your app

  Mo, an AI QA agent that explores your app, found about 269 labeled bugs on QA Bench versus 86 for the next best.

• We're releasing the weights of Mistral Large 4

  Mistral released Large 4 weights, claiming top open-weight scores on coding benchmarks

• Gemini 4 Carbon is the new #1 on the Artificial Analysis Intelligence Index v5

  Gemini 4 Carbon takes #1 on Intelligence Index v5 with 59 and is second in coding agents.

• Gemini 4 Carbon

  Google unveils Gemini 4 Carbon, a cybersecurity-focused model, but access is limited to select cyber defenders.

• Dayjob: Tax – evaluating AI agents on 50 professional tax assignments

  On 50 professional tax tasks, the best AI agent passed only 10.8%

• Anthropic S-1

  Anthropic filed an S-1 to list Class A shares on Nasdaq as 'ANT'.

Related Resources

• https://honnibal.dev/blog/astra-read-write-misleading
• https://news.ycombinator.com/item?id=50341174
• https://news.ycombinator.com/item?id=50344861
• https://arxiv.org/abs/2406.11717
• https://arxiv.org/abs/2503.22048
• https://arxiv.org/abs/2506.18167
• https://arxiv.org/abs/2512.05648
• https://quickthoughts.ca/projects/welsh-asr/