Don't ask an LLM for a confidence score

Who Should Read

A backend developer who adds a 0-100 confidence field to LLM JSON output and shows it to users as a trust signal.

Core Mechanics

• A confidence field is a comfort trick
  • The author argues LLM self-reported confidence scores are nearly useless.
  • They make output feel more trustworthy without making it more trustworthy.
• Evidence for self-knowledge is weak
  • Anthropic's introspection research itself notes the ability is highly unreliable and context-dependent.
  • A DeepMind paper found performance often degrades when models self-correct without external feedback.
• Nobody defines what confidence means
  • Correctness, coherence, and attempting the goal are three different questions.
  • One 0-100 number flattens all three.
• The 0-100 scale is especially bad
  • The Rescaling Confidence paper found over 78% of responses landed on just three values across six models.
  • One model said exactly 100 on 68% of instances, and coarser scales beat the 0-100 baseline on metacognitive sensitivity.

Community Reactions

• Line-number citations beat confidence scores
  • One commenter extracting structured data from transcripts had the model cite transcript line numbers instead of confidence, which cut hallucination dramatically.
  • Opus itself reportedly admitted a confidence number wouldn't be meaningful.
• Numbers didn't help clinicians
  • A commenter in healthcare said nurses facing 200+ question forms can't interpret the gap between 90% and 70%.
  • '60% sure the patient is allergic to nuts' meant nothing to them.
• The claim is called too strong
  • A commenter says it contradicts the Tian et al. and Anthropic results cited later in the post.
  • They report stated confidence works with a rubric and takes less work and data than training a classifier.
• Log-probs and calibration can work
  • Commenters suggest conformal inference on raw log-probs to control false-positive or recall rates.
  • Another suggests bucketing 1-5 scores on a known-answer corpus to map them to real probabilities.

How to Apply

• Audit where confidence is used
  • Find the branches and UI that read the confidence value in your code today.
  • Remove the field if no decision uses it.
• Require citations instead of numbers
  • In extraction or RAG prompts, require a line number or doc ID citation for each value.
  • Drop or re-retrieve any value without a citation.
• If you must, calibrate categorical labels
  • Define three levels such as high/medium/low with explicit criteria.
  • Run known-answer samples and measure actual accuracy per label.
• Let the model fix what it knows is wrong
  • Add a step in the tool loop that checks ungrounded claims and finds a source or deletes them.

Try It

# Paste into Claude or ChatGPT: simplified version of the post's idea (require citations instead of a confidence number)

Answer {{question}} using the {{document}} below.
Rules:
1. Attach the source line numbers as citations to every value.
2. Do not write claims you cannot cite; mark them as "no evidence".
3. Never output a confidence number.
Output JSON: {"answer": ..., "citations": [line numbers...], "ungrounded": [uncited items...]}

Document:
{{document}}

Question:
{{question}}

Next, try: run 20 samples with known answers and count the error rate of items with empty citations.

Terminology

Related Papers

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

• Show HN: Gemini 4 Argon plays StarCraft

  LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.

• A Plausible Explanation Is Enough to Mislead Multimodal Judges

  Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.

• Neural networks are clean

  Looking at neurons directly shows most have one main job, challenging superposition.

Related Resources

• https://justinflick.com/2026/07/27/llm-confidence-scores.html
• https://news.ycombinator.com/item?id=49010782
• https://crimede-coder.com/blogposts/2026/ConfClassification
• https://arxiv.org/abs/2603.09250
• https://arxiv.org/html/2603.17839v1