Reasoning Concentrates Errors, and Self-Consistency Never Notices

Who Should Read

Backend developers who use self-consistency or confidence-weighted voting over multiple LLM samples to pick a final answer

Core Mechanics

• Reasoning makes wrong answers pile up
  • Same Qwen3 weights, only the thinking flag changed.
  • The chance two wrong answers match rose in all 10 comparisons.
  • On open-ended problems, distinct wrong answers fell to 0.43-0.65x of the non-reasoning count.
• Accuracy gains cancel out for voting
  • Reasoning concentrates errors, which hurts voting.
  • It also shrinks the set of problems where voting matters by 2.7x.
  • The two effects roughly cancel, so accuracy shows neither.
• Confidence weighting never beats majority vote
  • 280 combinations: 7 methods, 8 models, 5 benchmarks.
  • None beat plain majority voting after correction.
  • Weighted voting matched majority on 98.5% of problem-method pairs.
• Signal direction flips with model settings
  • On the same 8B weights, answer log-probability predicted correctness with reasoning off.

Evidence

• Collision rose in all 10 comparisons
  • Reasoning on beat off in all 10 same-weights comparisons, sign test p=0.00098.
  • MATH-500 8B went from 0.379 to 0.509; 32B from 0.297 to 0.487.
  • Accuracy-matched subsets: 9 of 9 favored reasoning (p=0.00195).
• Distinct wrong answers roughly halved
  • On MATH-500 8B, reasoning off gave 522 distinct wrong answers vs 339 with reasoning on.
  • On multiple choice, with identical options, collision still rose by +0.121 to +0.181.
• Danger zone shrank 2.72x
  • The share of problems with 3 or fewer correct of 8 fell from a mean 0.369 to 0.136.
  • It fell in 10 of 10 cells (p=0.00098).
  • Headroom-normalized gain was 0.250 for reasoning and 0.249 for non-reasoning.
• No weighted-voting winner in 280 combinations
  • Only 5 reached uncorrected p<0.05, against 14.0 expected by chance.
  • When weighted and majority disagreed, weighted was right 56.3% of the time.

How to Apply

• A/B test confidence weighting against plain voting
  • Today, run k=8 majority vote and weighted vote on the same samples from your eval set.
  • Count problems where they disagree and how often the weighted answer is right.
  • If the difference is tiny, drop the weighting and use majority vote.
• Add a wrong-answer collision metric for reasoning models
  • Per problem, compute the share of wrong-sample pairs with the same answer.
  • If it is high, expect more samples not to help the vote filter errors.
• Log answer rate and truncation next to accuracy
  • Record parse failures and token-budget hits per dataset.
  • Check answer-rate gaps before comparing models.
• Validate log-prob signals per model and mode
  • Check the direction of the signal-correctness relationship on the exact model and setting you deploy.
  • Do not carry thresholds over from a setting where the direction differs.

Try It

# Run in Python: compute wrong-answer collision probability from per-problem samples (simplified version of the paper's metric)

from collections import Counter

def collision(answers, truth):
    wrong = [a for a in answers if a != truth]
    n = len(wrong)
    if n < 2:
        return None
    c = Counter(wrong)
    return sum(k * (k - 1) for k in c.values()) / (n * (n - 1))

# Replace with {{8 sampled answers per problem, ground truth}}
print(collision(["3", "3", "5", "3", "7", "5", "2", "2"], "2"))

# Next: compute this for reasoning on and off on the same problem set and compare, averaging only over problems with 2+ wrong samples.

Terminology

Related Papers

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

• Show HN: Gemini 4 Argon plays StarCraft

  LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.

• A Plausible Explanation Is Enough to Mislead Multimodal Judges

  Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.

• Neural networks are clean

  Looking at neurons directly shows most have one main job, challenging superposition.