OpenAI's GPT-6 Astra on ARC-AGI-3

TL;DR Highlight

Switching Astra to the Provider Adapter harness lifted ARC-AGI-3 from 62.7% to 99.9%.

Who Should Read

An LLM backend developer deciding how an agent loop should manage context and reasoning effort

Core Mechanics

Harness swap took 62.7% to 99.9%
- With the Standard harness, Astra (max) scored 62.7% on ARC-AGI-3 Semi-Private for $26K.
- With the Provider Adapter harness, Astra (high) scored 99.9% for $19K.
The harnesses handle context differently
- Standard carries forward only the notes the model chooses to keep.
- Provider Adapter preserves opaque reasoning state between requests and compacts longer conversations.
Provider Adapter was faster and used fewer tokens
- Across the 167 game-reasoning pairs both harnesses solved, it was about 3.66x faster in aggregate elapsed time.
- It used 49% fewer total tokens.
Astra's notes looked like compact code
- It tracked object positions, rules and unfinished plans in an algebraic shorthand (e.g. extend8 to3; retract10 to2).
- The post says this is an on-the-fly shorthand, not a full programming language.
Fewer actions than the human median

Community Reactions

Opinion that goalposts will keep moving
- One commenter predicted that after 99.9%, the argument will shift to humans being cheaper.
- Another argued anything verifiable will be saturated by models.
Opinion that Erdos problems show a long tail remains
- A commenter quoting Epoch AI said Astra solved at least once only 5 of 68 problems.
- Reaching those five took over $220,000 of compute.
Question about possible harness tuning
- A commenter asked whether OpenAI could have run ARC-AGI-3 beforehand and tuned a custom harness or used RL for these tests.
Point that no-reasoning mode did well
- A commenter noted the none setting scored 35.2% Standard and 96.7% Provider Adapter.
- Another calculated about $360 per puzzle, more than minimum-wage human labor.

How to Apply

Audit how your agent loop passes notes
- Spend 30 minutes checking how your agent carries prior-turn information.
- Run the same task with model-chosen notes versus full history and compare results.
Add a compaction step to long conversations
- Summarize earlier turns every N turns in long-running tasks.
- Log token counts and success rate before and after compaction.
Measure total cost per effort level
- Run one task set from low to max and record calls and total token cost.
- Compare by cost per completed task, not unit price.
Record harness conditions with every score
- State the context-handling setup, such as Standard or provider-native, in your model comparison reports.

Try It

copy & run

# Paste into a Claude or ChatGPT chat (simplified version of the method: only model-chosen compact notes carry over between turns)

You are an agent exploring a turn-based environment whose rules are unknown. Each turn, pick an action, then write notes to carry to the next turn in this short shorthand format.
STATE: current state compressed with symbols and coordinates
RULES: rules confirmed so far
PLAN: ordered list of next actions

Environment description: {{environment description}}
Previous turn notes: {{previous notes}}
This turn's observation: {{observation}}

Next to try: run the same task with full conversation history and compare action count and tokens.

Terminology

HarnessThe execution wrapper that connects a model to an environment and manages inputs, outputs and memory. Think of it as the exam room's rules and toolkit.

CompactionShrinking a long conversation so it can continue. Like summarizing meeting minutes for the next meeting.

Action efficiencyHow many actions it took to solve an environment. It measures attempts needed, not cost.

ARC-AGI-3A benchmark testing exploration, modeling, goal-setting and planning in turn-based environments with no stated rules.