Line-Anchored Feedback Cuts Token Costs and Improves Correctness in AI Code Editing

Who Should Read

A developer feeding review feedback to an LLM code-editing agent who is worried about output token cost and latency

Core Mechanics

• Same requests, different format
  • The requested changes were identical and only the delivery differed.
  • One arm was a holistic prompt, the other a FileMark export with line spans and quoted code.
• Big files saw big token savings
  • Claude Sonnet generated 58% fewer tokens overall and Opus 22.5% fewer.
  • On files of 100+ lines the drop was 24-80%.
• Small files have nothing to save
  • Under 100 lines Claude's token change ranged from -9% to +12%.
  • Locating the flaw is easy there, so both formats performed alike.
• Local models got more correct
  • Three of five local models gained 5-7 points.
  • Silent wrong-output runs dropped sharply and turned into visible errors.
• Sonnet lost a little accuracy
  • Over-terse edits dropped correctness from 0.994 to 0.949.

Evidence

• Claude token reduction
  • Sonnet's mean output tokens fell from 1,282 to 545 (-57.5% vs. holistic prompt).
  • Opus fell 22.5%.
• Bigger drop at 100+ lines
  • At T4, Sonnet fell 80.3% and Opus 38.4%.
  • The median T4 output across all 7 models fell from 1,736 to 681 tokens.
• Small overall correctness gain
  • Pooled correctness across 7 models rose from 0.630 to 0.650 (+2.0 points, McNemar p=8.7e-6).
  • qwen3-coder:30b gained 7.3 points, qwen2.5-coder:14b 6.0 and llama3.1 5.1.
• Larger effect with function patching
  • When the harness applied edits, correctness for the five local models on 100+ line files rose from 0.097 to 0.279, about 3x (exploratory).
  • The T4 score effect grew from +0.048 to +0.191.

How to Apply

• Add line spans to edit requests
  • Rewrite your review comments as line range, quoted code and request, then run both formats on the same file and compare output tokens.
  • Start with files of 100+ lines.
• Let tooling apply the edits
  • With local models, don't make the model match a diff grammar; have it return only revised functions and let the harness swap them in.
  • Use this when files are organized into functions.
• Measure edit-format compliance first
  • Run your model about 20 times to see whether it emits applicable SEARCH/REPLACE blocks.
  • If it can't, fix the format problem before anchoring.
• Track tokens and tests together
  • After switching to anchored feedback, record test pass rate alongside token counts.
  • Check whether overly terse edits start failing validation, as with Sonnet.

Try It

# Paste into Claude or ChatGPT (simplified version of the paper's method: mimicking FileMark's line anchors by hand)

Please edit the code below. Each request has a line range and the quoted code.
Change only the requested lines and output only the changed parts as SEARCH/REPLACE blocks.

[Code]
{{code}}

[Feedback]
- Lines {{start}}-{{end}} `{{quoted code}}`: {{requested change}}
- Lines {{start}}-{{end}} `{{quoted code}}`: {{requested change}}

Next, try this: send the same requests as one paragraph without line numbers and compare the output token counts.

Terminology

Related Papers

• Mercury 3: Diffusion at scale

  Scaling diffusion LLM 40x gives Mercury 3: 4–8x faster than speed-tier models, similar quality

• Timestral: Mistral's time-series foundation models

  A 209M time-series model ranks first on three benchmarks and runs on CPU.

• DeepSeek V4.1 Flash 1017 (Max)

  DeepSeek V4.1 Flash 1017 Max scores 45 on the Intelligence Index at 211 tok/s and $0.30 input.

• If you don't run your own LLM proxy, how do you know whats going on?

  Running your own LLM proxy shows which keys, models, and cache misses drive your usage.

• DeepSeek: DeepSeek v4.1 Flash 1017

  DeepSeek V4.1 Flash 1017: $0.05/M input, 1M context, built for agents

• DeepSeek-v4.1-Flash-1017

  DeepSeek V4.1 Flash minor update lifts coding and agent benchmarks while cutting run time and output length

• Show HN: Llmspeed – feel how fast a local LLM runs before you buy the hardware

Related Resources

• https://github.com/billyl320/filemark usage