Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents
TL;DR Highlight
Linking lessons to their source runs gave 90.2 avg score and cut Regression to 4.5%
Who Should Read
Backend developer whose agent started failing tasks it used to solve after adding experience memory
Core Mechanics
- Memory can make agents worse
- Tasks solved in all three no-memory runs but failed with memory are called memory-induced degradation.
- Of 355 such failures, 52% traced to a followed memory item, most often a wrong one (20%).
- Lessons come from a one-bit label
- Runs carry only a success/failure label, so an LLM guesses which actions mattered when writing lessons.
- Existing methods hand over lessons without source runs, so the agent cannot check them.
- Three layers: insight, summary, raw run
- WikiTrace links a short insight to a compilation explaining why the run succeeded or failed, and to the raw trajectory, all mounted read-only.
- The agent descends via bash when an insight lacks detail, conflicts with another, or comes from a failed run.
- Descending often changes the plan
- In 43% of descending runs the opened run changed the plan (revise insight 24%, add detail 14%, new fact 4%).
- Plan-changing runs led Insights Only by 8.3 points versus 4.4 when the plan stayed or the run went unused.
- Harder tasks trigger more descent
- Runs opened source runs in 22% of Easy tasks and 44% of Hard tasks.
- Descents after a problem led Insights Only by 10.4 points versus 3.7 with nothing gone wrong.
- Failed runs help when paired with evidence
- Failed-run-only memory beat success-only memory for both methods, and both together was best.
- Insights Only had its highest Regression with failed runs alone (PlanCraft 9.3%, ScienceWorld 9.4%).
Evidence
- Average score over eight suites
- WikiTrace 90.2, No Memory 79.5, Insights Only 87.0.
- AWM 81.6, ReasoningBank 81.7, ExpeL 80.3 gained only 1-2 points over No Memory.
- Regression comparison
- WikiTrace 4.5%, Insights Only 6.5%, no-memory rerun baseline 9.7%.
- Gain exceeds the rerun baseline by 35 points for WikiTrace, 27 for Insights Only, and 7-14 for existing memory methods.
- Holds on another model (GLM-5.3-flash)
- On PlanCraft and ScienceWorld, WikiTrace averaged 92.7 vs 90.9 for Insights Only and 71.2 for No Memory.
- Each existing method fell below No Memory on at least one suite.
- Cost
- Test-time cost per task is 1.17x Insights Only.
- Raising baseline build budgets (AWM 9x, ExpeL and ReasoningBank 3x) gave 83.8, 83.3, 82.8, still below WikiTrace's 90.2.
How to Apply
- Add source run IDs to memory items
- For your agent's lessons, add fields for the run each came from and whether it succeeded or failed.
- Store the log file path with each item.
- Give the agent log access and a descent rule
- Mount the log folder read-only and enable bash/read tools.
- Tell it in the prompt to check the source run when a lesson lacks detail, conflicts with another, or came from a failed run.
- Evaluate memory with Gain and Regression
- Take the majority outcome of several no-memory runs as the baseline and measure fail-to-success and success-to-fail rates separately.
- Check whether Regression falls below the no-memory rerun variation, not just average accuracy.
- From failed runs, take only what to avoid
- Add a prompt rule that lessons from failed runs give actions to avoid, not plans to follow.
- Make the agent state what differs from the current state before reusing anything.
Try It
# Paste into Claude Code or your agent's system prompt (simplified version of the paper's method)
You are an agent that uses lessons (insights) drawn from past runs.
Lessons are not answers. Past run records are read-only at /work/corpus, and each lesson names its source run ID.
Before acting, use bash/read to check the source run's raw record first in these cases:
- The lesson says what to do but not exactly how
- The lesson came from a failed run
- Two lessons conflict
Check again whenever an action fails.
Rules:
- Never replay a past run. It started from a different state.
- Take only facts you can check against this task, such as command formats, and name what differs first.
- From a failed run, take only the action to avoid.
Lessons: {{lessons with source run IDs}}
Task: {{task}}
Next, try: run the same task without this prompt and compare the results.Terminology
Related Papers
LLM assisted reverse engineering
Pointing an agent at binaries with skills and MD notes made years-long hacks feasible
Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
Injecting per-turn complexity reports into coding agents cut code erosion without changing correctness
Why is Gemini 4 Carbon taking so long?
Carbon took nearly an hour on bug fixes that Astra and Opus finished in minutes, mostly from extra unrequested work.
How the Claude Code team thinks about verification
E2E, property and differential tests plus verification rules let Claude run in parallel.
Running coding agents without handing them my credentials
Credentials stay on the laptop; agents run commands there via Touch ID approval
Windows 11 Is Collecting Data on Which Coding Agents and Developer Tools You Use
Reverse engineering shows Windows 11 logs which coding agents you use and reports it as optional diagnostics
The Silent Cost of Conversation History in LLM Reasoning
Keeping full conversation history dropped accuracy from 0.6 to 0.1 within 20 turns; a verified state summary prevented it.
Anthropic upgrades Claude Code Projects to run parallel AI workers
Claude Code Projects now has a coordinator that splits work into parallel threads and opens PRs.
Related Resources
Original Abstract (Expand)
External memory enables LLM agents to improve from experience by distilling trajectories of past tasks into concise lessons for new tasks. However, memory can also lead to memory-induced degradation, where an agent fails on tasks it would solve without memory, often because it follows a lesson that is incorrect, or misapplies or misreads a valid one. This is because lessons are inferred from trajectories labeled only with task success or failure, yet are typically provided without the trajectories they came from, leaving the agent no way to check them. To address this, we introduce WikiTrace, an evidence-linked memory system designed to let agents check each lesson against its original trajectory. Specifically, motivated by the finding that a trajectory often still holds the evidence needed to catch an erroneous lesson, WikiTrace stores each lesson as a concise insight linked to a summary of why its trajectory succeeded or failed and to the raw trajectory itself. Then, on a new task, the agent starts from all insights and, whenever one lacks details needed to act, conflicts with another, or comes from a failed task, descends through its links with terminal tools to fill in missing details or correct the insight before acting. We validate WikiTrace on five agentic benchmarks, showing that it consistently outperforms existing memory approaches and a baseline given the same insights alone, while also reducing memory-induced degradation. Our code is available at https://github.com/yukyeongleee/wikitrace