Agent-Controlled Forgetting for Tool-Using Agents: Reversible Context Curation in Practice | AI Paper Digest

TL;DR Highlight

Letting the agent swap tool results for notes with archived originals cut final prompt size by 75%.

Who Should Read

A backend developer running tool-using agents whose long tool outputs make context costs balloon

Core Mechanics

The agent curates its own tool results
- After reading, the model picks tool results and replaces each with a short note.
- Originals stay in an archive and can be recovered with a tool.
User and assistant messages stay untouched
- Only tool-result occurrences (by result ID) can be selected.
- If any ID in a batch is invalid, the whole batch is rejected with no partial change.
Big savings in a noisy debugging run
- It worked in an OpenTelemetry debugging task followed by an unrelated C++ task.
- Both arms passed the two primary behavioral tests.
Savings aren't free
- Requests rose from 121 to 185 and wall time was 17% longer.
- Completion tokens more than doubled.
Workload decides whether it helps
- The Planroom app-development task showed no savings.

Evidence

Final prompt tokens down 75%
- OpenTelemetry: 231,951 with the method vs 912,492 with retained history.
- At primary submission it was 124,746 vs 847,674, an 85% reduction.
Cumulative input down 50%, cost about 67–71% lower
- Cumulative input went from 45,766,146 to 22,988,240.
- Estimated API cost was $1.283–1.436 vs $4.378–4.379.
No savings on Planroom
- Final prompt was 6% larger: 283,105 vs 265,965.
- Estimated cost was $3.510–3.577 vs $3.461–3.465.
A quality drop appeared in one case
- On a browser workflow-design continuation, total tokens fell 63% but the manual score was 4/12 vs 6/12 for retained.
- On the C++ follow-up's mixed grader, method scored 3/6 vs 2/6 for retained.

How to Apply

Log tool-result sizes first
- Today, record per-request prompt tokens and tool-result sizes in your agent.
- Look for results of hundreds of thousands of characters, like browser_find.
- Without such outputs, expect little gain.
Design a context_apply / context_recover tool pair
- Give tool results unique IDs and accept batches of (result_id, note).
- Archive originals first, then swap in notes, and reject the whole batch if any ID is invalid.
- Recovery should append the original as a new tool result at the end.
Block user and assistant messages from curation
- Make user instructions and assistant messages non-selectable.
- Tell the model in the prompt to keep unresolved evidence and archive only finished diagnostics.
A/B against retained history
- Run the same task once without curation and once with the tools.
- Compare final prompt, cumulative input, requests, time, and quality separately.

Try It

copy & run

# Prompt to paste into Claude Code or ChatGPT (simplified version of the paper's method: notes without tools)

You are an agent managing your context during a long task. Below is the list of tool results seen so far.
{{tool result list: IDs and summaries}}

Rules:
- Select only results whose diagnosis is finished or that are no longer needed.
- Keep evidence that is still unresolved.
- For each selected result, output an (ID, note) pair. The note should hold a conclusion, source locations, or why it is no longer needed.
- Never select user instructions or assistant messages.

Instruction: Following these rules, output the list of (ID, note) pairs.

Next to try: check whether the notes alone are enough to answer {{next question}}, and if not, have the model say which IDs to recover.

Terminology

retained historyThe default approach of keeping every tool result verbatim in the conversation. It is the paper's baseline.

reversible curationReplacing tool results with notes while storing the originals so they can be restored. Like filing papers away with a sticky note.

stubThe short replacement left where the tool result used to be. It holds the note and an archive reference.

prompt tokensInput tokens sent to the model on each request. They accumulate, so longer conversations cost more.

oracleAn automated test that judges whether the output is correct. Here it is two Cypress cases.

Related Papers

Original Abstract (Expand)

Tool-using agents repeatedly carry observations whose useful content can be much smaller than their original payload. We study agent-controlled forgetting: the acting model selects previously observed tool results, replaces each with a short note at its original position, and retains the exact original in a recoverable archive. A Python harness exposes batch archival and explicit recovery without task-specific model training, while protecting user instructions and assistant messages from these operations. In an exploratory OpenTelemetry debugging case followed by an unrelated implementation task, the method ended with 231,951 provider-reported prompt tokens versus 912,492 under retained history, used 50% fewer cumulative input tokens, and had an estimated API cost of USD 1.28-1.44 versus approximately USD 4.38. Both arms passed the two-case primary behavioral oracle; neither fully satisfied the follow-up evaluation. The method made more requests and took 17% longer. A contrasting application-development pair produced no context or cost saving, and an earlier continuation exhibited lower manually assessed quality despite reduced context. These observations demonstrate substantial resource savings in noisy tool-use trajectories and identify workload dependence as a central consideration for reversible context management.