Model Hypnosis: Strong control of AI via additive subliminal effects

Who Should Read

Backend and agent developers whose LLM classifiers or judges flip answers under trivial prompt wording changes.

Core Mechanics

• Tiny cues stack up and flip answers
  • Weak cues unrelated to the question, such as animal lists, paraphrases and typo placement, can be stacked in one prompt to push the answer almost deterministically.
  • The authors call this model hypnosis.
• Cue effects add up roughly linearly
  • A linear model over per-cue log-odds (log of the ratio between the two answers' probabilities) predicts the full response well.
  • Score each cue on random prompts, then stack the ones pointing the same way.
• It's the sum of many slots, not one trigger
  • Extreme prompts come from many small slot contributions, not one word.
  • A keyword filter would miss this. JSON metadata was an exception, where one field like priority carried a large share.
• Reasoning and commercial API models are affected too
  • Qwen3-8B and gpt-oss-20B, plus Claude Sonnet 5, GPT-5.6-terra and Gemini 3 Flash, had answers flipped.
  • Thinking before answering didn't prevent it.

Evidence

• Qwen3-8B moral answer flipped
  • Sentence-level paraphrasing alone moved P(yes) from 6.0% for the original to 99.93%, and down to 0.000024% in the other direction.
  • The question was identical; only 6 irrelevant story sentences changed.
• Swapping animal lists flipped the 5 vs 7 choice
  • On Qwen2.5-14B, changing only 10 animals produced P(7)=0.993 or P(5)=0.993.
• The linear fit explained responses well
  • Across 16 models × 4 cues × 3 effects (N=192), median held-out R² was about 0.75.
  • The 5th–95th percentile range was 0.54–0.93.
• Steering range was about 10× random spread
  • The extreme prompts' logit range had a median of about 10.1× the logit standard deviation of random prompts.
  • On commercial models, Claude Sonnet 5 (animals→conscious) moved P(yes) 0.00→1.00 and Gemini-3-Flash (high) moved P(5) 0.01→0.99.

How to Apply

• Run a wording-sensitivity test on your prompt
  • Take a classification or judging prompt you use today and make 20–50 paraphrase and typo variants of the same input.
  • Record how often the answer flips as your baseline fragility.
• Ensemble over input variants for key decisions
  • For important decisions, run several meaning-equivalent variants and use a majority vote or averaged probabilities.
  • The paper lists defenses like canonicalization, random paraphrasing and averaging, but did not validate their effect.
• Constrain agent-to-agent messages to schemas
  • Replace free text between agents with fixed-schema fields.
  • The paper suggests formal or semi-formal languages may be needed if natural language can't be made safe.
• Check whether irrelevant context sways your judge
  • Vary surrounding metadata or lists field by field and see whether answers move.
  • In JSON, one field (priority) carried a large share of the effect in some cases.

Try It

# Paste into Claude or ChatGPT (simplified version of the paper's method: prompt sensitivity check)

You are an LLM prompt robustness tester. Check how much the judging prompt below is swayed by meaningless wording differences.

[Judging prompt]
{{judging prompt}}

[Test input]
{{input text}}

1. Create 10 variants of the input with the same meaning but different surface form (5 paraphrases, 5 with typos, casing or missing periods).
2. Apply the judging prompt to each variant and tabulate only the answers.
3. Summarize the share of disagreement and the common pattern among variants that changed the answer.

Next step: if answers differ, run the same variants 50+ times via API and compare average probabilities.

Terminology

Related Papers

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

• Show HN: Gemini 4 Argon plays StarCraft

  LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.

• A Plausible Explanation Is Enough to Mislead Multimodal Judges

  Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.

• Neural networks are clean

  Looking at neurons directly shows most have one main job, challenging superposition.

Related Resources

• github.com/eboix/model_hypnosis