When Should LLMs Abstain? Chain-of-Self-Questioning for Selective Risk Control | AI Paper Digest

TL;DR Highlight

Making LLMs self-assess needed info before answering cut wrong answers from 13.1% to 8.9%.

Who Should Read

Backend developers building QA or chatbot services whose LLM gives fluent but wrong answers and who want it to say "I don't know".

Core Mechanics

Make the model self-check before answering
- CoSQ (Chain-of-Self-Questioning) is prompt-only, with no fine-tuning or logit access.
- It has three stages: list needed information, rate confidence 0-100 for each, then answer only if the gate passes.
Low confidence means abstain
- The model answers only if mean confidence across information units is at least τ, otherwise it outputs "I don't know".
- Grounded-CoSQ passes only the accepted information to the final answer prompt.
Wrong answers dropped 32% at τ=0.90
- On TruthfulQA multiple choice, the share of all questions answered wrongly fell from 13.1% to 8.9% versus CoT.
- In exchange, 12.3% of questions are not answered.
Same direction across all 11 models
- Wrong-commitment rate fell and answered accuracy rose.
- This held at every threshold too.
Abstention is not always precise

Evidence

TruthfulQA multiple choice, 817 questions, mean over 11 models
- Grounded-CoSQ at τ=0.90 cut the wrong-commitment rate from 13.1% to 8.9% versus CoT (32.1% relative).
- Answered accuracy rose from 86.9% to 89.7% at 87.6% coverage.
Variants at τ=0.90
- Critical-CoSQ had 88.6% coverage and a 9.2% wrong-commitment rate.
- Adaptive-CoSQ had 86.5% coverage, 89.6% accuracy and an 8.9% wrong-commitment rate.
Statistics and per-model results
- Grounded's wrong-commitment reduction was 0.0422, with a 95% bootstrap CI of [0.0272, 0.0595].
- All 11 models moved in the same direction, with Wilcoxon p=0.00049.
NQ-Short, 300 questions, mean over 5 models
- Grounded-CoSQ cut the wrong-commitment rate from 0.412 (CoT) to 0.274.
- Answered accuracy rose from 0.588 to 0.670 at 0.830 coverage.

How to Apply

Add a confidence-check step to your current prompt
- In 30 minutes, have the model list needed information for one question and give 0-100 confidence for each.
- Treat a mean below 90 as "don't know" and compare with your existing answer.
Set τ based on your cost of errors
- Use τ=0.90 when wrong answers are costly; lower it or use Critical when broad answering matters.
- Measure coverage and wrong-answer rate per τ on your own validation set.
Route abstained questions to retrieval or humans
- Send gate-rejected questions to a RAG re-search or an escalation queue.
- Estimate reviewer load first, since about 12% of questions were abstained.
Apply it only to high-risk paths because of cost
- Calls grow to 3-4x, so enable it only where wrong answers are costly, such as medical, legal or finance.
- Keep plain CoT for general queries.

Try It

copy & run

# Paste into Claude/ChatGPT (simplified version of the paper's method: all 3 stages in one go)

Question: {{question}}
Options (if any): {{options}}

Perform these 3 steps in order.
1) List the pieces of information needed to answer this correctly as a numbered list. Do not answer yet.
2) For each, write the factual claim you would rely on (FACT) and a confidence from 0 to 100 (CONFIDENCE).
3) If the mean confidence is 90 or above, give a one-line final answer using only the high-confidence facts. If below 90, output only "ABSTAIN: I don't know".

On the last line, write the mean confidence as a number.

Next to try: run 10 of the same questions with thresholds 70/80/90 and compare abstention rate and wrong answers.

Terminology

AbstainDeclining to answer and saying "I don't know". Like a doctor referring a hard case to a specialist.

CoverageThe share of all questions the model actually answered. It drops as abstention rises.

Answered AccuracyAccuracy computed only over questions the model answered. Abstained questions are excluded from the denominator.

Wrong-commitment rateThe share of all questions where the model gave a wrong answer. Abstentions are not counted as wrong.

τ (threshold)The confidence cutoff for deciding whether to answer. It is a tuning dial for your service, not a probability.

TruthfulQAA benchmark testing whether models repeat common misconceptions.

Related Papers

Related Resources

Original Abstract (Expand)

Large language models can produce fluent answers when their factual support is weak. This paper introduces Chain-of-Self-Questioning (CoSQ), a prompt-only framework that makes answer commitment conditional on an explicit assessment of the information required to answer a question. We evaluate three CoSQ variants under seventeen conditions on the 817-item TruthfulQA multiple-choice validation set using eleven open-weight and hosted model families. In the final balanced-option protocol, Grounded-CoSQ at τ=0.90 reduces the mean unconditional wrong-commitment rate from 13.1% under chain-of-thought prompting to 8.9%, a 32.1% relative reduction, while increasing answered accuracy from 86.9% to 89.7% and answering 87.6% of questions. Both improvements hold for all eleven models and at every evaluated threshold. Critical-CoSQ and Adaptive-CoSQ provide neighboring operating points with 88.6% and 86.5% coverage, respectively, while remaining more reliable than the baseline. A secondary Natural Questions Short-Answer evaluation provides convergent open-form evidence. These findings show that self-assessment can support explicit, tunable answer-or-abstain decisions when an unsupported commitment is more costly than referral or review.