What is Missing from AI Post-Training AI: An Empirical Analysis | AI Paper Digest

TL;DR Highlight

Agents execute well but switch strategy only 2.1% of the time; one mid-run instruction gained up to 17.44 points.

Who Should Read

A developer who hands long-running experiments or fine-tuning to coding agents like Claude Code or Codex and is frustrated that they repeat the same approach.

Core Mechanics

Agents are reliable executors
- Across 1,338 trajectories they finished the full pipeline and beat the base model on every benchmark.
- They handle bug fixes, hyperparameter tuning, and data formatting well.
Strategy locks in at the start
- Claude Code mostly starts with full-parameter SFT, while Codex CLI mostly starts with PEFT.
- The default follows the agent, not the task.
Experience does not change strategy
- An experiment journal, skill library, and evaluator agent improved execution.
- The agent adopted all 22 execution-level suggestions but none of the 21 strategy-level ones.
More tokens only refine
- Easier tasks gain mostly early, with small returns afterwards.
- On AIME 2025, about 14M extra tokens bought roughly one extra problem.
One human mid-run instruction works

Evidence

Strategy switches are rare
- Only 74 of 3,557 adjacent training pairs (2.1%) changed strategy.
- Per agent: Claude Code 4.5%, Codex CLI 1.6%, OpenCode 0.4%.
Experience improved execution
- On Qwen3-1.7B-Base it closed 75.1% of the gap to the official instruct model, versus 48.6% for the same agent running autonomously.
- Removing the evaluator agent dropped it to 56.8%.
Switching mid-run raised scores
- Forked from the same checkpoint, GSM8K went 47.76%→65.20% (+17.44pp).
- HumanEval went 52.0%→62.8% (+10.8), and AIME 2025 went 0/30→2/30 (pass@8).
Reconsider-only did not help
- The reconsider fork scored 48.0% on GSM8K, close to the recorded 47.76%.
- It scored 31.7% on HumanEval versus 52.0%, and all three forks stayed with SFT.

How to Apply

Add a continue-or-switch question after each eval
- Today, add to your agent's system prompt that after every evaluation it must state whether to continue or switch approach.
- Require a rationale and one concrete alternative.
Name a concrete alternative when progress stalls
- If 3-5 runs of the same approach plateau, step in yourself.
- Write a specific instruction like 'switch from SFT to GRPO' rather than 'reconsider'.
Track adoption of strategy-level suggestions
- Split evaluator or log suggestions into execution-level and strategy-level and count adoptions.
- If strategy adoption is zero, design a human intervention point.
Fork checkpoints to compare strategies
- From a mid-run checkpoint, run a continue branch and a switch branch under the same budget.
- Record the number of strategy switches along with the score.

Try It

copy & run

# Paste at the end of your Claude Code or agent system prompt (simplified version of the paper's method)

## Strategy check rule
After every evaluation, before the next action, answer in this format.
1. Current strategy: {{e.g., full SFT}}
2. Score change over the last 3 runs: {{list scores}}
3. Decision: CONTINUE or SWITCH
4. If SWITCH: one concrete alternative (which of algorithm, data source, or training stages changes) and why
5. If CONTINUE: one sentence on why the score should still improve

If improvement is below {{threshold, e.g., 1 point}} for 2 consecutive runs, prefer considering SWITCH.

Next step: from the same checkpoint, run one branch with this rule and one without, under the same budget, and compare scores.

Terminology

RSI (recursive self-improvement)A loop where AI improves AI itself: propose a strategy, run experiments, interpret results, and revise the strategy.

SFTSupervised fine-tuning: further training a model on example answers, like studying by copying model solutions.

PEFT/LoRATraining only a small set of adapter parameters instead of the whole model, like adding sticky notes rather than rewriting the book.

GRPOA reinforcement learning method that trains with rewards such as answer correctness, by sampling many attempts and learning from their scores.

pass@kA metric that counts a problem as solved if any of k generated attempts is correct. AIME has only 30 problems, so they used pass@8.

Strategy vs execution levelStrategy is choosing the training algorithm, data source, or stages. Execution is work within a chosen strategy, like tuning learning rate or fixing formats.

Related Papers

Original Abstract (Expand)

Large language model (LLM) agents can now post-train an LLM end-to-end, raising the prospect of recursive self-improvement (RSI). Yet this progress is measured by aggregate benchmark scores, which cannot tell whether an agent executes a fixed plan well or strategically revises the plan when it fails. We separate these two capabilities: execution-level capability, iterating within an established training strategy, and strategy-level capability, revising that strategy as experimental evidence accumulates. Analyzing 1,338 post-training trajectories of frontier agents, we find that agents reliably execute post-training but lock into a default strategy, which follows the agent rather than the task, and only 2.1% of transitions between adjacent training runs ever change strategy. We then test whether the agent lacks experience, reasoning, or the decision to switch. (1) Experience improves execution but not the strategy. (2) Additional reasoning compute yields front-loaded gains on easier tasks but refines, rather than revises, the committed strategy. (3) Human review before training changes which strategy the agent locks into, not whether it locks in, whereas a single mid-run instruction outperforms the agent's own continuation by up to 17.44 points under the same budget. In conclusion, what the agent lacks is the decision to reopen a committed strategy and try another one. Realizing RSI therefore calls for interaction protocols and training signals that make strategy revision an explicit, rewarded decision.