Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content

Who Should Read

Backend or product developers building LLM persona agents for simulated users or synthetic respondents.

Core Mechanics

• Demographics alone fall short
  • Demographics-only prompts scored a mean MCC of 0.300.
  • Adding the attitude questionnaire raised it to 0.406.
• Intuitive instruction scored best
  • Telling the model to respond immediately, not weigh attributes, and accept departures from the profile gave mean MCC 0.473, the highest of five conditions.
  • Only the instruction changed; the profile (V1) stayed the same.
• Asking to infer did nothing
  • The instruction to infer from attribute combinations gave MCC 0.434, identical to the interview condition.
  • Profile-consistency rose from 0.754 to 0.826.
• Agents became less uniform
  • Analytical conditions compressed individual differences 7.1 to 10.2 times versus humans.
  • The intuitive condition cut that to 3.14 times.
• Too faithful to the profile looks fake

Evidence

• Profile content mattered a lot
  • MCC went from 0.300 (demographics) to 0.406 (questionnaire), averaged over four models.
  • 367 cells were right only under the questionnaire versus 252 only under demographics (p<.001).
• Intuitive beat every alternative
  • Versus the interview condition: 186 to 144 (p=.024); versus inference: 143 to 101 (p=.009).
  • Versus questionnaire: 238 to 166; versus demographics: 404 to 217 (both p<.001).
• Beat supervised models and the crowd
  • Random forest scored MCC 0.302 and logistic regression 0.183.
  • The crowd baseline scored 0.450 overall but −0.008 on the 33 contested posts.
• Beat the crowd on unseen topics
  • The intuitive condition scored 0.399 versus the crowd baseline's 0.265, winning 241 cells to 167 (p<.001).
  • Compression was 3.48x versus 7.1 to 8.0x for the analytical conditions.

How to Apply

• Append an intuitive instruction to persona prompts
  • Add 'answer immediately, do not weigh attributes, it is fine to deviate from the profile' to your current simulation prompt.
  • Compare agreement with real reactions on the same 20-30 posts against your old prompt.
• Drop 'infer from combinations' instructions
  • If your prompt has one, remove it and run an A/B test.
  • Judge by match with real human reactions, not by consistency.
• Use attitude questions and narrative text, not demographics
  • Put trust and values questions into synthetic respondent profiles.
  • If agents must handle new topics, also add a self-introduction or interview text.
• Add a uniformity metric to your evals
  • Compute the standard deviation of predicted like-rates across agents and compare it with real people.
  • If every agent gets the same value, rework the prompt.

Try It

# Paste into Claude/ChatGPT — simplified version of the paper's method (V5 intuitive condition)

[Profile]
{{questionnaire answers + full interview/self-introduction}}

[Instruction]
You are the real person described above. For each post below, answer instantly and intuitively whether this person would like or dislike it.
- Do not analyze or weigh attributes against each other.
- Do not explain your reasons.
- It is fine if some reactions contradict the profile.

[Posts]
{{numbered list of posts}}

Output format: number:like or number:dislike, comma-separated

Next, try: run the same profile with an 'infer from combinations of attributes' instruction and compare match with real reactions.

Terminology

Related Papers

• AI is a tool and using it well takes skill

  A rebuttal to Hickey: LLMs are tools, and skill in directing and checking them decides the outcome

• Reflective Prompt Optimization of a Deep-Research Agent: A Topic-Disjoint, Multi-Seed Protocol for Measuring Transferable Gains and Prompt-Level Overfitting

  Reflective prompt optimization raised held-out citation reward from 0.702 to 0.781 on a deep-research agent.

• GPT‑6 and Intelligent UI for everyone

  GPT-6 reportedly adds generated UI to chat; with no article text, only comment reactions are available.

• Base Models Can Reason By Taking a Cue From Training Data

  Prefilling just two opening tokens lifted Olmo-3-7B's MATH-500 accuracy from 42% to 78%.

• Labels Override Definitions in Jev-Style Typed Decision Models

  Renaming options to A/B raised routing accuracy by +0.1511 over meaningful labels.

• How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs

  Delta debugging trimmed few-shot prompts by 65.3% on average with unchanged propositional output

Related Resources

• https://osf.io/gdq9j/overview?view_only=4a9096e0201a487c8c9231d148293dda