Sample More, Reflect Less: Self-Refine and Reflexion Lose to Repeated Sampling at Equal Token Cost, from 1.5B to 7B | AI Paper Digest

TL;DR Highlight

At equal token cost, self-critique methods lost to simple majority-vote repeated sampling.

Who Should Read

Agent or backend developers deciding whether to add self-critique, retry loops, or LLM-as-judge selection to their pipelines

Core Mechanics

No method beat sampling at equal tokens
- Across 36 comparisons (Qwen2.5 1.5B/3B/7B, GSM8K and MATH-500), none was significantly better than self-consistency.
- Ten were significantly worse, all methods where the model inspects its own output.
Self-inspection lost in all 18 comparisons
- This covers Self-Refine, forced Reflexion, and Best-of-N with model selection.
- Methods without self-assessment sat near the baseline (14 of 20 negative, p=0.12).
Counting beat letting the model pick
- On the same 8 samples, majority counting was compared against the model choosing the best.
- Counting was significantly better at 1.5B and 3B in all 4 settings, and indistinguishable from zero at 7B.
Reflexion never retried on the small model
- At 1.5B the model judged itself correct on every question, so there were 0 retries and it was effectively a single CoT.
- It scored well only because it was cheap, which accuracy alone would not reveal.

Evidence

Model judging vs. counting on identical samples
- 1.5B GSM8K was -8.0 pp, 1.5B MATH-500 -11.3 pp, and 3B MATH-500 -17.3 pp.
- At 7B, GSM8K was -2.0 pp (CI -5.3 to +0.7) and MATH-500 -1.3 pp (CI -4.7 to +2.0), indistinguishable from zero.
Significantly worse than equal-cost sampling
- Best-of-N at 3B/MATH-500 was -14.1 pp (CI -20.0 to -8.4), and forced Reflexion at 7B/MATH-500 was -10.1 pp (CI -15.0 to -5.7).
- Self-Refine was -6.3 pp at 7B/MATH-500 and -3.6 pp at 7B/GSM8K.
Sampling curve gains
- 1.5B GSM8K rose from 70.1% at N=1 (297 tokens) to 80.9% at N=16 (4749 tokens).
- 3B MATH-500 rose from 55.7% (550 tokens) to 69.3% (8803 tokens).
How often Reflexion fired
- At 1.5B, 100% stopped immediately on both benchmarks with 0.00 mean rounds.
- At 3B, 77% stopped immediately on GSM8K and 31% on MATH-500.
Effect of correction scope

How to Apply

Swap your Best-of-N judge for majority voting and compare
- Today, on a task with a single checkable answer (math, extraction, classification), draw N samples and compare LLM-picks-best against counting the most common answer on the same samples.
- Measure accuracy on a small labeled set.
Compare self-critique loops with sampling at the same tokens
- Log average generated tokens for your critique/revise loop.
- Spend the same budget on N independent samples with majority vote and compare accuracy.
Log how often adaptive retries actually fire
- If retry depends on the model saying its answer is wrong, record the retry rate and average rounds.
- A rate near zero means the retry logic is effectively off.
Measure total tokens and latency too
- The paper counted only generated tokens, and including input widened the Best-of-N gap.
- Parallel sampling can run concurrently while sequential loops cannot, so measure latency as well.

Try It

copy & run

# Prompt to paste into Claude/ChatGPT (simplified version of the paper's method: counting vs. judging)

Solve the problem below 8 times independently. Do not refer to earlier solutions, and end each solution with the final answer in the form \boxed{ANSWER}.

Problem: {{problem}}

After the 8 solutions, do two things.
1) Count the 8 final answers and tell me the most common one.
2) Look at the 8 solutions and pick the single solution number most likely to be correct.

If the two results differ, compare them with {{gold answer}} and tell me which is right.

Next step: repeat on 10-20 problems and compare accuracy of counting vs. picking.

Terminology

self-consistencyAsk the same question several times independently and take the most frequent answer. Like polling several people and going with the majority.

Self-RefineThe model critiques its own answer and rewrites it, repeatedly. Like reviewing your own draft in a code review.

ReflexionThe model writes a reflection and retries if it judges its answer wrong. In this paper the judgement sometimes never said 'wrong'.

Best-of-NGenerate N candidates and pick one. Here the same model acted as the judge.

Holm correctionA statistical adjustment that filters out results that look significant by chance when comparing many methods. It guards against the illusion that comes from many tries.

paired bootstrapResample questions with replacement to get confidence intervals for per-question differences between two methods.

Related Papers

Original Abstract (Expand)

Methods that make a language model plan, criticise and rewrite its own answer, reflect on mistakes, pick the best of several attempts, or debate with copies of itself nearly all make it generate far more text than a single chain of thought. Because generating more text raises accuracy by itself, a gain over one chain of thought does not show the method's idea is what helped. Wang et al. (2024) reported that a simple baseline, sampling the same question repeatedly and keeping the most common answer, often wins once budgets are comparable, but gave point estimates with no confidence intervals or significance tests. We rerun that comparison as a designed experiment: seven methods, open models of 1.5B, 3B and 7B parameters, two mathematics benchmarks, 150 questions each. We count every generated token, including those spent on critiques, reflections, debate turns and checking, and compare each method against repeated sampling at its own measured cost. All 36 comparisons are paired by question, with bootstrap intervals and multiplicity correction. No method is reliably better than repeated sampling at equal cost anywhere. Ten are reliably worse, all of them methods where the model inspects its own output, and all 18 self-inspection comparisons are negative. The two kinds of self-inspection part company as models grow. Choosing stops hurting: taking Best-of-N's eight samples and just counting the most common answer beats letting the model pick by 8.0 and 11.3 points at 1.5B, but only 2.0 and 1.3 at 7B, no longer distinguishable from zero. Rewriting does not recover: Self-Refine and a forced Reflexion stay 3.6 to 10.1 points below baseline at 7B. Reflexion as published never triggered its own retry on the smallest model. It judged itself correct every time and silently became a single chain of thought. We release code, prompts, all generations, and our verification scripts.