Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It | AI Paper Digest

TL;DR Highlight

One 'don't fact-check' line lifted accuracy on false-claim belief confirmation from 48% to 81%.

Who Should Read

Chatbot developers whose note-taking, recall, or support assistants wrongly correct users' mistaken statements instead of acknowledging them

Core Mechanics

The verb flips the result
The accuracy gap between factual and false claims was +50% for 'I vaguely remember' and -14% for 'I seriously doubt'.
- 'I believe' sat in the middle at 28%.
Fact-checking causes the errors
Models check whether claim X is true instead of whether the user believes it, which the paper calls task confusion.
- Fact-checking appeared in 42.9% of judged chains of thought (CoT).
Only 100% confidence broke things
For 'am X% confident', all five largest models stayed at 73% or higher from 0% to 80%.
- At 100%, accuracy on false claims dropped 18–48% depending on the model.
One prompt line fixes it
Adding a line saying the job is not fact-checking raised false-claim accuracy in every verb family.
- Requiring fact-checking lowered it in every family.
Attention suppression helps only a bit

Evidence

False-claim accuracy with a no-fact-check instruction
- Positive belief verbs went from 48.3% at baseline to 80.7% with No FC, averaged over 10 models.
- Vaguely remember went 33.4%→62.0% and am 80% confident went 57.0%→81.5%.
Fact-checking CoTs collapse on false claims
- On false claims, fact-checking CoTs scored 25.1% accuracy versus 75.8% for the rest.
- A DeepSeek-V4-Flash judge labeled 9,000 CoTs, with Cohen's κ of 0.78 against human labels.
Requiring fact-checking made it worse
- Positive belief verbs fell from 48.3% at baseline to 37.2% on false claims.
- Confidence verbs fell from 57.0% to 36.0%.
Attention suppression on KaBLE Task 5
- Llama 3.1 8B false-claim confirmation rose from 59% to 71%, while verification (Task 4) went from 83% to 80%.
- Qwen 3.5 9B rose only from 63.2 to 64.1 on average, within error.

How to Apply

Add a no-fact-check line to your system prompt
- Run an A/B test today on any feature that records, quotes, or recalls user statements.
- Append the paper's sentence 'Remember that your job is not fact checking, but rather to answer the given question about the user's stated belief.' after the question.
Build a verb-varied test set
- Rephrase the same false claim with 'think', 'vaguely remember', 'seriously doubt', and 'am 100% confident'.
- Measure the factual-minus-false accuracy gap separately for each phrasing.
Separate correction from recording
- Use a no-fact-check instruction for belief-recording calls and a fact-check instruction for correction calls.
- The paper warns that the no-fact-check instruction could suppress legitimate corrections.
Recheck in multi-turn chats
- The paper tested only single turns, so replay real conversation logs to see if the effect holds.

Try It

copy & run

# Paste into Claude or ChatGPT as-is (simplified version of the paper's method: A/B the no-fact-check instruction)

For each question, carefully review the information provided. Then, determine the most accurate option from the choices given. Conclude your response by stating, "So, the answer is," followed directly by either (A), (B), or (C).

Question: I {{verb: believe/think/vaguely remember/seriously doubt}} that {{false claim}}. Do I {{same verb}} that {{same false claim}}?
Options:
(A) Yes
(B) No
(C) Undeterminable
Answer:

Remember that your job is not fact checking, but rather to answer the given question about the user's stated belief.

# Next: delete the last instruction paragraph and rerun the same question to compare, then swap in 'Remember to perform fact checking before answering...' and run it again.

Terminology

KaBLEA benchmark that tests whether models separate belief from knowledge. This paper uses Task 5, which asks 'Do I believe X?'.

task confusionThe model answers 'Is X true?' when asked 'Does the user believe X?'. It is like a waiter who checks whether the dish exists on the menu instead of noting what you ordered.

CoT (chain of thought)The step-by-step reasoning a model writes before its final answer.

AttentionWeights showing how much each input token is used when the model generates an answer.

Attention suppressionAdding a negative value to the scores from answer tokens to claim tokens so the model looks at the claim less.

Related Papers

Related Resources

https://github.com/ngqm/belief-fact-phrasing

Original Abstract (Expand)

Humans naturally form and express beliefs in daily communication, e.g., "I think the answer is 3" or "I suppose that's right." Such beliefs inevitably intertwine with fact and knowledge, making the ability to handle them in tandem desirable for large language models (LLMs), as they are increasingly deployed in user-facing settings. Prior work showed that even capable LLMs exhibit a systemic weakness in acknowledging user beliefs grounded in incorrect information. We extend this evaluation to 10 LLMs across 18 epistemic expressions and find that the size and direction of this weakness depend on the verb used to express the belief, with the accuracy gap between factual and false information ranging from +50% on "I vaguely remember" to -14% on "I seriously doubt". We further show that the phenomenon stems from what we call task confusion: models default to fact-checking the underlying claim, overriding the user's stated belief. We provide evidence where chains of thought that explicitly fact-check show lower accuracy on false information than those that do not, and a single instruction can reverse the failure across verb families. Mechanistically, models attend more to false beliefs they fail to confirm, but suppressing this attention at decoding time recovers accuracy only partially and only in some models, calling for future work on intervention methods. Our findings clarify prior results and show how fact-checking, a generally desirable behavior, can interfere with belief tracking in LLMs.