CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents | AI Paper Digest

TL;DR Highlight

Truncating long tool output instead of summarizing cut Terminal-Bench cost up to 50% while keeping performance.

Who Should Read

A developer running long coding-agent sessions in Claude Code or OpenHands who is struggling with context cost and quality decay.

Core Mechanics

It only truncates and never summarizes
- When the token threshold is reached, tool results over 500 characters are dropped and shorter ones kept verbatim.
- Tool calls are reduced to signatures such as the tool name and path, and thoughts are cut to 300 characters.
- The system prompt, the first user message and the K most recent turns stay in full.
It never compacts a compaction
- Each new compaction discards the previous compacted block and compresses only the current live session.
- This prevents the context drift that builds up in summaries of summaries.
- The price is losing direct memory after two compactions.
Summaries make agents skip re-reading
- A plausible summary convinces the model it already has the information, so it stops revisiting docs and code.
- Truncation leaves visible gaps, so the agent re-reads the source.
Fewer cache breaks make it cheaper
- Context is left untouched between compactions, so the KV-cache (reused computation for earlier tokens) stays valid.

Evidence

Kimi K2.6 success rate on Terminal-Bench 2.0
- At 32K and 16K it reaches 61.42%, 2.26 points above the full-context 59.16%.
- At 16K it scores 61.42% versus 55.45% for Terminus-2's native summarization.
GLM 5.3 Flash under Claude Code on Terminal-Bench 2.1
- At a 45K compaction budget it reaches 76.69%, above Claude Code auto-compaction (70.97%) and the default 200K setup (73.03%).
Three rollouts plus SGV, cost versus accuracy
- Kimi K2.6 at 16K with three rollouts scores 69.7% at $58.01, 5.7 points above the uncompacted three-rollout run (64.0%, $91.65) at 37% lower cost.
- Its +10.5 points over a single full-context run costs 1.9x, matching Opus 4.7 (69.4%) and beating GPT 5.3 Codex (64.7%) and Opus 4.6 (62.9%).
Speedup on KernelBench Level 3
- Kimi K2.7 with 128K compaction reaches 2.23x at 200 steps and 3.58x at 400 steps versus 1.30x for no compaction at 256K.
- GPT-5-mini at 200 steps gets 2.09x versus 1.78x for AdaExplore, and at equal cost 2.02x versus 1.30x.
- At 400 steps the compared methods reach 2.86x for sliding window ($21.52), 3.47x for summarization ($8.10), 3.33x for microcompaction ($12.84) and 3.58x for CliffCompaction ($8.32).

How to Apply

Measure token shares in your agent logs
- Take one session you ran today and count what percent of tokens are tool results, tool calls and thoughts.
- The paper found 56% tool results and 28% tool calls on Terminal-Bench with GLM 5.1, so compare your numbers.
Add a rule that drops tool results over 500 characters
Try a 16K to 32K compaction threshold in your own harness
Attach it to closed tools like Claude Code via an API proxy

Try It

copy & run

# Simplified version of the paper's method: paste into a long coding chat in Claude Code, ChatGPT, etc.

Now we will shrink the context. Build a compacted block of this conversation's earlier content using the rules below. No summarizing or rewording; only truncate or drop.
- If a tool result is over 500 characters, drop it entirely. If it is 500 or fewer, keep it verbatim.
- Reduce each tool call to the tool name, target file path and key arguments. Drop long content written into files.
- Keep only the first 300 characters of each of your earlier thoughts or plans.
- Keep the system instructions and the first task description in full.
- If a previous compacted block exists, discard it and compact only what came after.

Conversation to compact:
{{conversation log to compact}}

Try next: continue the task using only the compacted block, and have the agent re-call tools to fetch anything that is missing.

Terminology

CompactionShrinking earlier conversation before the context window fills up. It is like clearing filed papers off a full desk.

KV-cacheMemory that stores the model's computation for tokens it has already read so it can reuse them. If the prefix changes, it must recompute everything (re-prefill), which costs money.

Test-time scalingSolving the same problem several times and picking the best answer. It is like taking an exam repeatedly and reporting the top score.

Compaction precision/recallPrecision is how faithful the kept information is to the original, and recall is how much past information can still be retrieved. CliffCompaction trades recall for precision.

SGV (Soft Group Verification)A selector that uses features such as how much candidate solutions overlap on the same task and a LightGBM model to pick the best candidate.

Related Papers

Related Resources

https://github.com/nguyenvuthientrang/cliffcompaction

Original Abstract (Expand)

Agents often work on complex problems that require millions of tokens of context, which necessitates compacting across sessions due to limited context windows. We develop CliffCompaction, an autocompaction technique that reduces cost by up to 50% under a bounded context while maintaining or improving performance on Terminal-Bench and achieving new levels of efficiency for test-time scaling and state-of-the-art results on KernelBench. The per-rollout savings of CliffCompaction make the performance--cost trade-off of test-time scaling more efficient, adding over 10 percentage points on Terminal-Bench for less than the cost of two full-context runs. Under parallel test-time scaling, CliffCompaction lets Kimi K2.6 match Opus 4.7, and exceed Opus 4.6 and GPT-5.3 Codex at lower cost. The key to CliffCompaction's effectiveness is that it keeps compacted information faithful by only truncating or dropping content, never rephrasing or rewriting it. We never compact a compaction---each pass operates only on original content, and prior compacted output is discarded, preventing context drift from accumulating. These properties sustain continual learning over sessions exceeding a million tokens: on KernelBench, CliffCompaction reaches CUDA kernel speedups of $2.23\times$ after 200 steps and $3.58\times$ after 400 steps, surpassing specialized search algorithms and trained agents despite being a general-purpose compaction technique. We open-source a scaffold-agnostic API-proxy implementation of CliffCompaction usable with Claude Code, Codex and other harnesses.