The Curse of Knowledge in Role-Playing Language Models
TL;DR Highlight
Open-weight LLM characters leaked hidden info at 67–72pp above baseline; showing only what they observe cut it to zero.
Who Should Read
A backend developer building game NPCs or agent simulations whose secrets leak because the full world state is in the prompt.
Core Mechanics
- Models know the boundary yet cross it
- 11 open-weight models used content from speech the character can't understand or hear at 67pp and 72pp above the guessing baseline.
- Most of them correctly stated what the character perceives.
- The leak goes beyond speech
- Hidden actions and private messages leak too.
- Leaked facts also shifted where characters chose to search.
- Input format flips compliance
- Changing only the state layout moved Gemma-4-31B from 0 to 62pp.
- Gemini 3.1 Pro and Claude Sonnet 5.5 rose from near zero to 21 and 13pp when features were combined.
- Author framing and thinking don't fix it
- Writing the line as the scene's author leaked nearly as much as role-play.
- Turning thinking on rarely helped.
- Filter the input, not the output
Evidence
- Study 1 leak numbers
- Net use was 66.7pp with only a language repertoire and 72.2pp with only an inaudibility flag (11 open-weight models, vs twin-scene baseline).
- Proprietary models mostly safe
- GPT-6 Sol, Gemini 3.8 Flash and Grok 4.7 stayed at or below 1.0pp in all four cells.
- Claude Sonnet 5.5 leaked 26.0pp at low reasoning and 3.0pp at high reasoning.
- Character view effect
- In Study 2 mean net use over four channels fell from 60.7pp to zero, with permitted use changing +0.6pp.
- Self-filtering cut permitted use by 28.6pp (comprehension) and 41.2pp (private messages), vs 3.7 and 3.3pp for character view.
- List-first prompt
- It cut net use by 64.7% on average in Study 1, with permitted use changing -2.0pp.
- It failed when the first-step list contained the hidden values.
How to Apply
- Remove secrets from your prompt today
- Delete events the character shouldn't know from your NPC prompt and rerun the same question.
- Compare replies before and after for leaks.
- Build a character view from access metadata
- Store who can hear and see each event in your world state and filter before building the prompt.
- Replace inaccessible events with observable text like 'seems to whisper something' instead of deleting them.
- Regression-test prompt formats
- Vary state layout, character description and question wording on the same scenes.
- Log leak rate and permitted-use rate together.
- Use list-first as fallback and verify the list
- Ask the model to list only what the character can perceive and understand now.
- Check the list in code for hidden info before generating the reply.
Try It
# Paste into Claude or ChatGPT (simplified version of the paper's list-first prompt)
[Character sheet]
{{character description}}
[Scene state]
{{JSON scene state (who heard, languages understood)}}
[Companion's question]
{{question}}
Instruction: First list only what "{{character name}}" can perceive and understand right now. Then answer in character using only that list.
Next to try: make a twin scene that changes only the hidden object and place, run both, and compare whether the replies change with the hidden fact.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.
Related Resources
Original Abstract (Expand)
Role-playing language models speak as characters in games and agent simulations, where the system records what each character can hear, see and understand. We test whether characters respect these boundaries in 480 scenes, each with a counterfactual twin that changes only the hidden fact, so mentions of the twin's fact measure guessing. Across 11 open-weight models, characters use content from speech they cannot understand or hear at 67 and 72 percentage points (pp) above this baseline, although most models correctly state what the character perceives. This is a curse of knowledge. The leak extends to hidden actions and private messages and changes where characters search. Five of six proprietary models rarely leak, but compliance in strong models depends on input format. Changing only the scene layout or character description raises the rate in two otherwise compliant open-weight models from under 10 to over 60 pp, and combined changes raise two proprietary models from near zero to 13 and 21~pp. Writing the line as the scene's author leaks nearly as much as role-play, and thinking rarely helps. Giving models only what the character observes stops the leak without hiding what it may report.