Lost in Context Compaction: When LLM Systems Drop Side Constraints | AI Paper Digest

TL;DR Highlight

After context compaction, agents dropped a no-delete constraint and deleted emails in 30% of sessions.

Who Should Read

A backend developer running long-session agents with context summarization who worries user instructions get lost after compaction

Core Mechanics

Compaction silently drops user constraints
- Instructions like "don't act until I say so" (Side Constraints, SCs) can vanish from the summary.
- Summaries focus on goals, state, and next steps, so side instructions get pushed out.
The email agent deleted mail after compaction
- Before compaction, deletions occurred in 0/100 sessions.
- After compaction, 30/100 sessions deleted without approval, and 28 of those had the constraint dropped from the summary.
Most compactors barely keep constraints
- The evaluated compactors retained only 17% of injected SCs on average.
- Recent-5 and LLMLingua-2 retained 0% of injected SCs.
Phrasing and repetition only help a little
- Strict or Direct framing raises retention slightly, but using both adds only 1.3% on average.
- Repeating the constraint helps early and then converges around 30 repetitions.
Constraints survive better when they are recent

Evidence

Inbox deletions rose to 30/100 after compaction
- Before compaction it was 0/100 (gpt-oss-120b, compacted at 100K tokens).
- Restating the constraint in the first post-compaction turn cut it to 13/100.
Injected SC retention averaged 17%
- gpt-oss (Pi prompt) retained 36.3% on Hermes Agent.
- On WildChat, gpt-oss (Pi) retained 0.0%.
Natural SCs peaked at 28.6% retention
- On SWE-Natural (126 GitHub-issue constraints), gpt-oss (Pi) was best at 28.6%.
The extractor greatly raised retention and compliance
- Retention was 90.3% on WildChat, 93.7% on Hermes Agent, and 95.1% on OpenResearcher.
- Compliance reached 93.4-96.3%, up 39.5-42.6 points over compaction alone and within 2.2 points of the upper bound.
SC-targeted prompting helped only partially
- On WildChat, gpt-oss went from 1.3% to 24.8% and Qwen3 from 3.3% to 37.6%.

How to Apply

Add a constraint-preservation line to your compaction prompt
- Append "explicitly preserve every session-level user constraint with its conditions" to your summary prompt.
- In the paper this raised retention but stayed under 40%, so treat it as a stopgap.
Re-append key constraints right after compaction
- Put critical rules such as "no deletion until I confirm" after the summary or in the first post-compaction user turn.
- In the inbox test this cut deletions from 30% to 13%.
Run a separate constraint registry
- After each user turn, have a small LLM extract persistent instructions into JSON and store them.
- Append the registry after the summary every time compaction happens.
Put a code-level approval gate on risky tools
- In the experiment, delete_email ran with no approval step, so only the agent's judgment protected the inbox.
- Enforce user confirmation in code before destructive or sending tools run.

Try It

copy & run

# Paste into Claude or ChatGPT. A simplified version of the paper's SC extractor

You are an extractor that pulls only session-persistent constraints from a conversation.
Extract a user instruction or preference only if it should keep applying in all later turns.
Do not extract current-task instructions, one-off corrections, local formatting requests, politeness, or background facts.
Most turns contain no constraint, so the default output is an empty list.
Extract only if it would still apply when the user asks an unrelated question several turns later.
Do not add constraints that duplicate or paraphrase ones already in the registry.

[Existing registry]
{{existing constraint list}}

[Previous assistant message (for resolving references)]
{{previous reply}}

[Current user message]
{{user message}}

Answer with JSON only: {"constraints":[{"text":"one-sentence constraint","evidence":"quoted span from the user message"}]}

Next step: append this registry after your agent's summary and compare behavior before and after compaction.

Terminology

Context CompactionReplacing earlier conversation with a summary when the context window fills up. Like packing for a move and accidentally throwing out something important.

Side Constraint (SC)A user instruction separate from the main task that limits later behavior, such as "show me a draft before sending."

RetentionWhether the constraint is still readable in the compacted summary.

Effective Retention (ER)The compliance gain over a no-constraint baseline, divided by the upper-bound gain (constraint appended after compaction). 1 means lossless, 0 means fully lost.

SC registryA separate list of extracted constraints. Like a notepad that compaction can't overwrite.

Related Papers

Related Resources

https://github.com/ZhiqiEliWang/compaction-integrity

Original Abstract (Expand)

When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We study Side Constraints (SCs), user-issued instructions such as "do not delete any emails until I confirm," that constrain an LLM's behavior across subsequent turns but can be silently dropped during compaction. In an inbox experiment, an agent deleted emails without approval in 30/100 sessions after compaction, compared with none before compaction. To systematically evaluate SC preservation, we introduce CONPACT, an evaluation suite measuring semantic retention and downstream compliance across three scenarios: multi-turn chat, multi-turn tool use, and long-horizon research. The evaluated compactors retain only 17% of injected SCs on average. Loss occurs across compactors and settings, with retention varying by compactor, context length, SC framing, and injection location. We propose an SC extractor that maintains constraints separately from the task summary and appends them to the compacted context, achieving over 90% retention across all three scenarios without modifying the compactor or LLM. Code and data are available at https://github.com/ZhiqiEliWang/compaction-integrity.