Context Language Models

TL;DR Highlight

Letting models edit their own context file beat SOTA context management with less compute

Who Should Read

Backend developers building long-running agents who hand-code context pruning and summarization in their harness

Core Mechanics

Context is treated as a file
- A Context Language Model (CLM) treats its context as a single file that the model can update without restriction.
- The goal is to let the model learn what is most important to keep.
It extends to multi-agent systems
- Each agent's context lives as its own file.
Zero-shot CLMs on existing models won
- Built zero-shot from existing models, CLMs beat SOTA context management strategies.
- On BrowseComp-Plus, accuracy was 11.4% higher with 21.5% fewer FLOPs.
Long tasks used far less compute
- On 12-hour EdgeBench, scores were 5% higher with 59% fewer FLOPs.
- On a 24-hour multi-repo agent-swarm task, improvement was 65% greater at the same compute.
Context management can be learned via instructions and RL
- Natural-language instructions evolved by a skill-optimization loop raised held-out accuracy by up to 35.9 points while reducing compute.

Community Reactions

Cache hit rate is the main obstacle, one commenter says
- Frequently editing the context or prefix lowers cache hit rates, so it is hard to implement efficiently on APIs like Anthropic's.
- The commenter says it is solvable in principle but needs transformer and serving changes.
A separate supervisor agent worked better, per one commenter's experience
- A concern was raised that self-management consumes limited attention.
- A separate hypervisor agent cost the main agent zero tokens and made cache misses easier to control.
A pushback that you can do this today
- A commenter noted you can send the file as the next context but pay for cache misses.
- CLM avoids that recomputation.
Invalid cache suffixes not hurting performance may be the key finding, one commenter says
- They noted the paper ignored regular caching rules and kept invalid suffixes without a performance loss.
- Another commenter asked whether this is just Recursive Language Models with a file instead of a variable.

How to Apply

Split agent context into a notes file
- Today, give one long-running agent a notes file it rewrites each turn instead of a raw chat log.
- Feed only the system prompt, the notes file, and recent messages next turn.
Measure cache-miss cost first
- Log prefix cache hit rate and input token cost on every notes edit.
- Keep the file's head fixed and edit only the tail to limit misses.
Iterate the management instruction against an eval set
- Write instructions on what to keep or drop and compare accuracy and tokens on held-out questions.
- Keep only the instructions that score better.
Compare self-managed vs supervisor agent
- Run the same task both ways and compare accuracy and total tokens.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the paper's method: keep context as a file the model edits itself)

You are an agent doing a long task. The [CONTEXT_FILE] below is your working memory.
Each turn, when you get new information, follow these rules.
1. Keep only facts, decisions, and todos that matter for the task; delete the rest or compress to one line.
2. Start your reply with the full updated [CONTEXT_FILE].
3. Below it, write this turn's work result.

[CONTEXT_FILE]
{{current context file contents}}

[NEW_INFO]
{{new information this turn}}

[TASK]
{{overall task}}

Next: feed the updated file back as [CONTEXT_FILE] for 10 turns, then compare accuracy and token count against passing the full conversation log.

Terminology

Context managementDeciding what to keep or shrink from conversation and tool outputs fed to the model. It is like clearing and summarizing papers on a small desk.

FLOPsA unit of compute used by the model. Treat it as a proxy for cost and speed.

KV cacheStored computation for an already-read prefix so it can be reused. If the early part changes, the cache after it is invalidated.

Suffix Cache ReuseA cache reuse method the paper co-designed for CLM serving. It cut server-side compute 35% versus standard SGLang.

zero-shotApplying a method to an existing model without extra training.