GPT-6 Astra in code review: Gains, privacy, and cost
TL;DR Highlight
In CodeRabbit's eval, Astra caught 4% more bugs than Sol, and 20% more on hard cross-file reviews
Who Should Read
A backend developer choosing an LLM for automated code review who must weigh performance against cost
Core Mechanics
- Astra caught 4% more bugs
- On CodeRabbit's actionable bug coverage, it beat GPT-5.6 Sol by about 4% and Opus 5 by 22%.
- Easy reviews in the mix made the overall gap look modest.
- The gap widens on hard cross-file reviews
- On the harder cross-file subset, Astra led Sol by 20% and Opus 5 by 33%.
- They read this as value in linking a change's intent to effects across the codebase.
- This does not establish an overall ranking
- It is an early directional result and does not predict a team's defect rate.
- They could not isolate whether context size caused the gain.
- Token price is 2.5x Sol
- Astra costs $10 input and $50 output per 1M tokens, same as Fable 5.1.
- For 100K input and 10K output tokens, a task costs $1.50 vs Sol $0.60, Terra $0.32, Luna $0.032, i.e. 2.5x, about 4.7x, about 47x.
- Cost per task may differ from price
Community Reactions
- A commenter found Astra about 30% faster than Sol at similar cost
- In toy benchmarks, token efficiency offset the 2.5x sticker price for the same outcome.
- A commenter found Astra slower in practice
- They said it was clearly slower than 5.6 Sol on the same tasks.
- They suggested it may be reading more context.
- A commenter said the comparison is irrelevant outside CodeRabbit's tool
- A former security researcher said the tool's noise and friction in CI review weren't worth it.
- A commenter questioned the comparison set
- They asked whether Fable, as a Sol-class model, should be compared with Mythos instead.
How to Apply
- Run 10 identical PRs on Astra and your current model
- Today, take 10 recently merged cross-file PRs and give both models the same review prompt.
- Record bugs found, verification time, and token usage in a table.
- Compare models by total cost per success
- Track tokens and retries per successful outcome, not just unit price.
- Include caching, tool calls, and retry costs.
- Route only hard reviews to Astra
- Send multi-file changes with scattered evidence to Astra.
- Leave routine changes to a cheaper model.
- Check data retention terms before signing
- Confirm ZDR eligibility per model before sending customer code.
Try It
# Paste into Claude or ChatGPT: quick cross-file review test (simplified version of the article's method)
You are a code reviewer. Given the diff and related files below, find places outside the changed lines that could break.
[diff]
{{diff}}
[related files]
{{related file contents}}
For each finding, give: 1) the location that breaks (file:line) 2) the evidence linking it to the change 3) a fix the developer can apply right away. If there is no evidence, do not guess; write "insufficient evidence".
Next step: run the same prompt on two models and compare bugs found and verification time.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.