Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
TL;DR Highlight
Injecting per-turn complexity reports into coding agents cut code erosion without changing correctness
Who Should Read
A developer whose coding agent runs long tasks and leaves huge, complex functions that worry them about maintainability
Core Mechanics
- Complexity report injected every turn
- After each code-changing turn, functions with cyclomatic complexity (independent paths through a function) above 6 are flagged.
- It is a plugin (MITL) needing no training, reference solution, or hidden tests.
- Erosion dropped significantly
- Erosion (share of complexity mass in functions with complexity above 10) fell significantly for all four models on Commit0 and SlopCodeBench.
- On SWE-EVO the effect was smaller and significant for only two of the four models.
- Correctness did not improve
- Every per-model held-out correctness interval included zero.
- The authors say a small cost of combined feedback on SlopCodeBench cannot be ruled out.
- Agents acted on the reports
- They re-edited 54–88% of flagged functions on Commit0 and 87–94% on SlopCodeBench.
- Agents saw at least 98.2% of the offered messages.
- Not just function splitting
- After inlining single-use helpers, 3–46% of the erosion difference remained on Commit0 and 8–18% on SlopCodeBench.
- This check does not isolate the causal effect of splitting.
- Coverage feedback did not help
- Coverage feedback alone did not lower erosion.
- On SlopCodeBench it raised E10 for DeepSeek, GLM and Luna.
Evidence
- Commit0 pooled erosion drop
- Complexity feedback (C) lowered E10 by −0.112 [−0.138, −0.087] versus the reminder-only arm R.
- Complexity plus coverage (CT) gave −0.078 [−0.097, −0.059] versus R.
- SlopCodeBench pooled erosion drop
- C lowered E10 by −0.246 [−0.261, −0.230] versus R.
- C and CT kept E10 at or below 0.022 at every checkpoint.
- Correctness changes include zero
- Pooled Commit0 CT−R correctness was +3.1 points [−0.1, +6.3].
- SlopCodeBench was −1.7 [−3.6, +0.1] and SWE-EVO fix rate was +2.7 [−0.2, +5.6].
- Reminder alone also helped
- On SlopCodeBench the reminder (R) lowered E10 by 0.133 to 0.383 versus the no-feedback arm B0.
- For Haiku, R−B0 was −0.383, much larger than C−R at −0.045.
How to Apply
- Run a complexity check after agent work
- Run a tool such as radon or lizard on code your coding agent wrote today and list functions with complexity above 6.
- Paste that list back to the agent and ask it to refactor.
- Inject a report on each code-changing turn
- In your agent harness, measure only the changed functions at the end of each code-changing turn.
- Put functions over the limit (e.g., 6) into the next turn's context as a short message, highest first.
- Separate report effect from reminder effect
- Create a control arm that gets only the instruction text without measurements.
- Compare correctness on hidden tests and erosion together.
- Don't use coverage feedback alone
- Add complexity feedback first and measure coverage feedback separately.
- In the paper coverage alone did not reduce erosion.
Try It
# Paste into Claude Code or ChatGPT — simplified version of the paper's MITL method
Below is code you just modified. Find functions whose cyclomatic complexity (number of independent paths through a function) exceeds 6.
{{recently modified code}}
Instructions:
1. List functions above 6, highest first, with your estimate of each complexity.
2. Refactor each one to 6 or below without changing behavior.
3. Tell me how to verify that existing tests still pass after refactoring.
Next step: repeat this prompt every time the agent changes files, and vary the {{limit}}.Terminology
Related Papers
LLM assisted reverse engineering
Pointing an agent at binaries with skills and MD notes made years-long hacks feasible
Why is Gemini 4 Carbon taking so long?
Carbon took nearly an hour on bug fixes that Astra and Opus finished in minutes, mostly from extra unrequested work.
How the Claude Code team thinks about verification
E2E, property and differential tests plus verification rules let Claude run in parallel.
Running coding agents without handing them my credentials
Credentials stay on the laptop; agents run commands there via Touch ID approval
Windows 11 Is Collecting Data on Which Coding Agents and Developer Tools You Use
Reverse engineering shows Windows 11 logs which coding agents you use and reports it as optional diagnostics
The Silent Cost of Conversation History in LLM Reasoning
Keeping full conversation history dropped accuracy from 0.6 to 0.1 within 20 turns; a verified state summary prevented it.
Anthropic upgrades Claude Code Projects to run parallel AI workers
Claude Code Projects now has a coordinator that splits work into parallel threads and opens PRs.
Show HN: AgentRun – open-source, Self-improving Managed Agents
Agents save notes and skills after each run, cutting code review cost to 18 cents
Original Abstract (Expand)
Coding agents now write whole codebases in long autonomous runs, guided mainly by whether their tests pass. Yet passing tests alone does not reveal accumulating complexity or gaps in test coverage, which then persist as agents extend their own code. We introduce metrics-in-the-loop, an inference-time intervention that gives agents in-flight feedback: cheap, automatically computed code metrics injected into the agent's context during the trajectory. After each code-changing agent turn, we insert short feedback messages naming changed functions whose cyclomatic complexity exceeds a limit (complexity feedback) or changed lines that are skipped by tests (coverage feedback), giving the agent the opportunity to refactor code or strengthen tests before continuing. We run extensive experiments and ablations comparing several interventions across three long-horizon coding benchmarks and four strong models. We find that complexity feedback significantly reduces erosion, a measure of complexity concentration of code, without affecting held-out correctness of the code. Broadly, our experiments investigate a practical form of agent "re-railing", where timely, actionable measurements redirect agents before weaknesses become the foundation they keep building on. As agents scale to codebases with millions of lines, such interventions become critical to sustaining effective development and long-term code maintainability.