Termination Lag: Coding Agents Solve Early but Stop Late
TL;DR Highlight
Coding agents are correct on the first edit in 38–82% of solved runs, yet spend 35–60% of steps after.
Who Should Read
Backend or platform developers running coding agents like OpenHands who are struggling with long runs and high token costs.
Core Mechanics
- The first edit is often already correct
- Among runs that reach a correct patch, the first source edit is already correct in 38–82% of cases.
- Qwen3-Coder hit 82% on SWE-bench Verified.
- 35–60% of steps come after the fix
- This is termination lag: the steps between the first correct patch and the end of the run.
- In successful runs, the median was 35–37% for Qwen3-Coder and 54–60% for Laguna-XS.2.
- Most extra steps are verify and explore
- Verification (running tests, writing checks, building) is 48–63% of post-fix steps.
- Exploration is 15–29%.
- Further source edits are mostly just 2–6%.
- Extra work sometimes breaks a correct fix
- In 1–8% of runs that reached a correct patch, later edits broke it and the run ended incorrect.
- Stopping at the first correct patch would have avoided these failures.
- Final score alone hides the waste
- Two agents fixing at step 10 get the same score even if one runs 40 more steps.
- The authors treat producing a fix and recognizing it is done as separate abilities.
Evidence
- Qwen3-Coder median lag 16 steps, 35%
- On SWE-bench Verified, solved tasks only; Multilingual shows 18 steps and 37%.
- Context grew +30% on Verified and +28% on Multilingual after the fix.
- Laguna-XS.2 lagged longer
- Median 57 steps, 60% on Verified.
- Median 56 steps, 54% on Multilingual.
- Same pattern in commercial models
- Claude Haiku 5.5 on Verified: median 10 steps, 54%.
- GPT-6 Luna on Verified: median 6 steps, 36%.
- Oracle stopping at first correct patch
- It removes a median 35% (Qwen3-Coder Verified) and 37% (Multilingual) of steps in solved runs.
- No originally solved task is lost.
How to Apply
- Find the first passing point in your agent logs
- Today, save the repo diff at every step and run tests offline to compute TTFCP and TL.
- Measure only; give the agent no feedback.
- Classify post-fix steps by type
- Bucket tool calls into verify, explore, edit, and wrap-up and compute shares.
- If verify plus explore exceeds half, make it a target for stopping-rule design.
- Cap redundant verification with a policy
- Tell the agent in the prompt to submit once tests pass and the diff is rechecked once.
- The paper only measured the savings ceiling and did not test such rules, so check your own benchmark for score drops.
- Add an efficiency metric to evaluation
- Put NTL (share of steps after the fix) on your dashboard next to solve rate.
- Use both when comparing models or frameworks.
Try It
# Paste into Claude Code or an agent system prompt (simplified version of the paper's idea: a stopping-rule prompt)
You are a code-fixing agent. Follow this stopping rule.
1. If a test that reproduces the issue or the relevant existing tests pass, treat it as a solution candidate.
2. Once you have a candidate, check git diff exactly once.
3. Then submit immediately, with no extra exploration, new test writing, or refactoring.
4. Do not edit already-passing code again without new evidence.
Task: {{issue description}}
Next to try: run {{number of tasks}} of the same tasks with and without the rule and compare solve rate and total steps.Terminology
Related Papers
LLM assisted reverse engineering
Pointing an agent at binaries with skills and MD notes made years-long hacks feasible
Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
Injecting per-turn complexity reports into coding agents cut code erosion without changing correctness
Why is Gemini 4 Carbon taking so long?
Carbon took nearly an hour on bug fixes that Astra and Opus finished in minutes, mostly from extra unrequested work.
How the Claude Code team thinks about verification
E2E, property and differential tests plus verification rules let Claude run in parallel.
Running coding agents without handing them my credentials
Credentials stay on the laptop; agents run commands there via Touch ID approval
Windows 11 Is Collecting Data on Which Coding Agents and Developer Tools You Use
Reverse engineering shows Windows 11 logs which coding agents you use and reports it as optional diagnostics
The Silent Cost of Conversation History in LLM Reasoning
Keeping full conversation history dropped accuracy from 0.6 to 0.1 within 20 turns; a verified state summary prevented it.
Anthropic upgrades Claude Code Projects to run parallel AI workers
Claude Code Projects now has a coordinator that splits work into parallel threads and opens PRs.
Original Abstract (Expand)
Coding agents can solve a task well before they stop working on it. We characterize \emph{termination lag}--the number of steps between an agent's first correct patch and the end of its run--across five models, two agent frameworks, and three SWE-bench datasets by recording and testing every distinct repository patch generated during execution. Our analysis reveals that 38--82% of successful runs achieve a correct solution on their initial source-code edit. However, agents continue to expend substantial effort after correct patches, with post-solution steps accounting for a median of 35--37% for Qwen3-Coder and 54--60% for Laguna-XS.2 in successful runs on SWE-bench Verified and Multilingual. Agents spend most of these additional steps verifying their work and exploring the repository. In a small fraction of cases (1--8%), this continued activity leads agents to undo their earlier correct patch, leaving the final solution incorrect.