Base Models Can Reason By Taking a Cue From Training Data | AI Paper Digest

TL;DR Highlight

Prefilling just two opening tokens lifted Olmo-3-7B's MATH-500 accuracy from 42% to 78%.

Who Should Read

Developers using open base models for math or code reasoning who want gains without RL training by prefilling the response opening.

Core Mechanics

Two fixed tokens reach RL level
- Prefilling '.\n\n Okay' raised Olmo-3-7B's MATH-500 pass@1 from 42% to 78%.
- Its RL counterpart scored 75%.
- No reasoning instruction or demonstration was given.
Qwen3-14B's cue is 'Alright ,'
- MATH-500 pass@1 went from 72% to 87%, matching its RL version.
- The best cue differs by model.
RL made the cue more likely
- '.\n\n Okay' probability rose from 0.14 to 0.65 in Olmo-3-7B.
- 'Alright ,' rose from 0.04 to 0.58 in Qwen3-14B.
- Base-vs-RL KL divergence peaked at the first two positions.
Cue effects come from training-data associations
- Replacing 'okay' with 'chicken' in mid-training made '.\n\n Chicken' a reasoning cue.
- Replacing 'step by step' with 'duck duck goose' made 'Think duck duck goose' about as effective.

Evidence

Accuracy nearly doubled on Olmo-3-7B
- MATH-500 pass@1 rose from 42.2% (no cue) to 77.9% with '.\n\n Okay'.
- GSM8K went from 45.1% to 85.9%, AMC 23 from 30.9% to 59.5%.
Label-free search came close to the best
- The cue picked by answer agreement was within 1.1 points of the best of 500 openings.
- Random-token openings scored only 33.3–47.1%.
Beat GEPA-optimized prefills
- Olmo MATH-500: GEPA 67.0% vs cue 77.9%.
- Qwen3-14B: GEPA 79.3% vs cue 86.6%.
Fewer tokens for the same accuracy
- No-cue majority voting needed 3.6–12x more tokens to match one cued response.
- On Olmo-3-7B that was 9 samples and 6.7x tokens.
Data-edit retraining results
- With 10B mid-training tokens, '.\n\n Chicken' on MATH-500 rose from 2.4% to 37.2%.

How to Apply

Try prefill on a local base model today
- Run 30 math problems with no cue and with '.\n\n Okay' prefilled under identical settings.
- If you have labels, measure pass@1 directly.
Pick a cue per model without answer labels
- Collect 20 likely openings from the model.
- Sample 16 responses per problem and choose the opening whose final answers agree most.
Validate Llama-family and post-trained models separately
- The paper found no working cue for Llama-3.1-8B.
- Always compare against no cue on your own model before shipping.
Add opening-token variants to safety evals
- Test variants such as 'Okay ,' and '.\n\n Okay' for refusal and harmful-response rates.
- Treat this as mandatory if your service fixes the opening.

Try It

copy & run

# Simplified version of the paper's method: compare prefilled openings on a base model with transformers
# pip install transformers torch

from transformers import AutoModelForCausalLM, AutoTokenizer
name = "{{base model name}}"
tok = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name, device_map="auto")
problem = "{{math problem}}"
prompt = ("Solve the following math problem step by step.\n"
 "The last line of your response should be the answer in form Answer: $Answer (without quotes).\n"
 f"{problem}\nRemember to put your answer on its own line after \"Answer:\"")
for cue in ["", ".\n\nOkay"]:
    ids = tok(prompt + cue, return_tensors="pt").to(model.device)
    out = model.generate(**ids, max_new_tokens=2000, do_sample=True, temperature=0.6, top_p=0.95)
    print(repr(cue), tok.decode(out[0][ids.input_ids.shape[1]:]))

# Next: run 30 problems 32 times each, compare pass@1, and try other openings like 'Alright ,'.

Terminology

prefillFilling in the first tokens of a response before the model writes. Like giving an actor the first line and letting them continue.

base modelA pretrained model without instruction tuning or RL post-training. It has only learned next-token prediction.

pass@1The probability that a single sampled answer is correct.

GRPOAn RL method that compares several answers to the same problem and reinforces the better ones, like grading on a curve within a group.

KL divergenceA measure of how different two models' next-token distributions are. Larger means their choices differ more.

Related Papers

Related Resources

Original Abstract (Expand)

In this paper, we study how training data creates associations between the tokens at the start of a base model's response and the reasoning behavior that follows. First, we demonstrate that fixing particular starting token cues makes a base model's performance competitive with that of its reinforcement learning (RL)-trained counterparts on math and coding. For instance, the cue ".\n\nOkay" raises Olmo-3-7B's MATH-500 pass@1 accuracy from 42% to 78%, while "Alright," raises Qwen3-14B's from 72% to 87%. Second, RL makes these cues more likely, while fixing them recovers much of its performance gain over the base model. Third, we trace the reasoning effects of token cues to the training data. We perform causal data interventions to turn an arbitrary word, such as "chicken", into an effective reasoning cue, or remove an existing cue's effect. A similar edit makes the prompt instruction "Think duck duck goose" as effective as "Think step by step" at eliciting reasoning. We also find that the hidden state representations induced by different cues correlate with different document types from the training set. Finally, we extend our study of token cues with a case study in language model safety, finding that different cues elicit distinct refusal and compliance behaviors that correspond to different types of training data.