Calling the AI bluff: Adding "Do not guess" cut made-up claims from 71% to 20% | AI Paper Digest

TL;DR Highlight

Adding "Do not guess" to extraction prompts cut made-up fields from 70.7% to 20.2%

Who Should Read

A backend developer extracting structured data from web pages with an LLM who sees it fill in values that aren't there

Core Mechanics

Twin pages measured invention
- Two pages differ by one row: one shows the answer, the other doesn't.
- Both carry a decoy such as "Was $493.00"; the test used 42 pairs across 7 page types.
One sentence helped all 16 models
- The sentence was "Use null for any field whose value is not on the page. Do not guess."
- Without it, models made up 405 of 573 missing fields (70.7%).
- With it, they made up 116 of 574 (20.2%).
Decoy price fooled everyone, then one
- On the "Was $493.00" page, all 16 models called 493 the price without the sentence.
- With the sentence, only 1 did.
A cheap model beat a paid API
- Firecrawl made up 24 of 36 missing fields, and all 24 copied the decoy.
- Plain fetch plus GPT-6 Luna made up 5 of 36 for $0.0049 across the full run.
- By nonoverlapping 95% ranges, Firecrawl made up more than 13 of the 16 models.

Community Reactions

Skeptics say prompting can't fix it
- One view: LLMs don't understand words, so prompts can't force truth.
- Another said the examples are contrived.
Magic-incantation treadmill
- One commenter expects harnesses to adopt this, the effect to fade, and everyone to move on to the next phrase.
Similar instructions worked for some
- One user got good results with "you're not trained on this data, return exclusively grounded results".
- Another put "Always ground your responses." in CLAUDE.md and Claude ignored it.
Validate in code instead
- One commenter gave up on prompt restrictions since measurable output can be validated programmatically.
- Another said the last 20% needs a system that knows what it doesn't know, not a command.

How to Apply

Add the null rule to your extraction prompt
- Append "Use null for any field whose value is not on the page. Do not guess." to your prompt today.
- Run 10 pages with missing answers and count made-up fields before and after.
Build twin-page regression tests
- Make a page with the answer and a twin with that row removed.
- Put decoys such as an old price or wrong author on both, and assert null in CI.
Add a cheap verification call per value
- Ask a small model whether the page supports each extracted value.
- Null out rejected values or send them to human review.
Check near-meaning fields separately
- The checker missed cooking/resting time given as prep time.
- Add rule-based checks or manual sampling for such fields.

Try It

copy & run

# Paste into Claude or ChatGPT as-is (simplified version of the twin-page test)

Extract the fields below from the page text as JSON.
Use null for any field whose value is not on the page. Do not guess.

Fields: {{field list, e.g. price, author, venue}}

Page:
{{page text}}

Then rerun on a version of the same page with the answer row removed, and compare whether the removed field comes back as null.

Next: run it again without the sentence and compare the number of made-up fields.

Terminology

Wilson intervalA statistical way to get an error range for a proportion with small samples. Overlapping ranges mean the ranking isn't clearly separated.

DecoyA value that looks like the answer but is wrong, such as an old price shown as "Was $493.00".

Twin pagesTwo pages differing by one row, one with the answer and one without. They show whether a model admits not knowing.

Null ruleAn instruction to return null instead of inventing a value. Like leaving an exam answer blank when you don't know.

Related Papers

Related Resources

https://earnanhonestdollar.com/bench