GPT-6 Astra
TL;DR Highlight
OpenAI released GPT-6 Astra, and HN split over how to read its benchmarks
Who Should Read
A developer using LLM APIs who must decide whether to switch models with every new release
Core Mechanics
- Source article unavailable
- No original text was provided, so only content quoted in comments is available.
- No source-based facts are stated here.
Community Reactions
- ARC-AGI-3 scores reported as 98.6% and 99.9%
- One comment cited 98.6%, up from about 30% with Fable.
- Another said 99.9% and wanted more detail on the Responses API harness.
- Comment says harness makes comparison unfair
- A comment said the page lists GPT-5.6 Sol at 7.8% but estimates about 30% with the Responses API harness.
- The commenter argued Opus 5 would also score much higher under the same harness.
- Artificial Analysis scores 61, behind Opus 5
- A comment said OpenAI claims Astra beats Fable and Opus on all benchmarks except Humanity's Last Exam with tools.
- Another comment said Artificial Analysis gives 61 at max effort.
- Some called this jagged intelligence.
- SRE-Bench and CoT control quoted
- Quoted comments say Astra solved 88.0% of SRE-Bench tasks in one attempt and 99.2% in four, versus 55.9% and 68.7% for GPT-5.6 Sol.
- Another quoted safety evaluation says Astra controls its CoT better and can evade monitors when sandbagging.
How to Apply
- Compare your current model and Astra with the same harness
- Pick 5-10 representative tasks from your service today and run both models with identical prompts and tools.
- Record time and cost along with accuracy.
- Check conditions and blanks in benchmark tables
- Before switching, check harness, effort, and blank (-) entries in comparison tables.
- When official numbers and external indexes disagree, favor your own evaluation.
Try It
# Paste into Claude or ChatGPT: build a self-evaluation plan before switching models
I run {{service description}}. My current model is {{current model}} and I'm considering switching to {{candidate model}}.
Using the representative task list below, create an evaluation plan to compare both models under the same conditions.
Tasks: {{5-10 tasks}}
Include: pass criteria, metrics (accuracy, latency, cost), how to align harness and effort settings, and a results table.
Next step: run both models with the table and paste the results back to decide whether to switch.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.