Qwen 3.8 follows GPT-5.5 Pro reasoning prefills

Who Should Read

A developer who selects or evaluates models and wants to know whether an open model was distilled from a commercial one.

Core Mechanics

• Reasoning prefill probes for distillation
  • For each model, generate a normal answer and one whose reasoning channel starts with the first 1% of the teacher's reasoning.
  • Measure how much of the teacher's visible answer shows up in the target's first 100 tokens.
  • The score is the mean of unigram, bigram and trigram recall.
• The eval has 45 problems
  • 15 STEM, 15 non-STEM and 15 private synthetic puzzles.
  • Deltas are absolute percentage-point changes.
• Only Qwen3.8 A95B jumped
  • It went from 16.79% unprefilled to 34.97% prefilled, +18.18 pp.
  • DeepSeek V4 Flash was −1.17 pp, Inkling +0.46 pp and Kimi K3 +4.54 pp.
• Qwen rose in every category
  • STEM went from 19.26% to 46.24%, +26.99 pp.
  • Non-STEM rose +12.80 pp and puzzles +14.75 pp.

Community Reactions

• Commenter explaining the method and background
  • The authors are described as writers of the Stolen Thoughts paper that recovered readable CoT from OpenAI and Anthropic models.
  • In the earlier paper, prefilling Claude 4.8 reasoning moved Kimi-K3 answers closer to Claude 4.8.
• Opinions on training data contamination
  • One commenter notes Qwen 3.8 0902 was trained after the August 10 paper, so it may have seen those traces.
  • Another asks whether both models were trained on the same benchmark solutions.
• Opinions against over-interpreting
  • A commenter says even a small amount of post-training could raise the correlation.
  • The result can't separate stylistic influence from real capability transfer.
• Commenter's own comparison experience
  • They saw GPT-5.6 Sol's CoT leak into a tool call, then compared Qwen3.8 27B on the same geometry problem.
  • Both started with 'We need solve...', which they say proves nothing.

How to Apply

• Compare candidate model outputs directly
  • Today, send 10 identical problems to the open model and the commercial model and collect the first 100 tokens of each answer.
  • Compute n-gram recall with the code below to check overlap between models.
• Measure prefill against a control
  • When choosing models, generate an unprefilled and a prefilled answer for each problem.
  • Compare the delta in pp across models to spot outliers.
• Mix private problems into your eval set
  • Add your own synthetic puzzles to avoid public benchmark contamination.
  • Record scores on public and private problems separately.

Try It

# Run in a Python 3 terminal (simplified version of the paper's method: measures only n-gram recall between two answers, no prefill)

import re

def toks(s, n=100):
    return re.findall(r"\w+", s.lower())[:n]

def recall(src, tgt, k):
    a = {tuple(src[i:i+k]) for i in range(len(src)-k+1)}
    b = {tuple(tgt[i:i+k]) for i in range(len(tgt)-k+1)}
    return len(a & b) / len(a) if a else 0

def score(teacher, target):
    t, g = toks(teacher, 10**6), toks(target, 100)
    return sum(recall(t, g, k) for k in (1, 2, 3)) / 3

teacher = "{{teacher model final answer}}"
target = "{{target model answer}}"
print(score(teacher, target))

# Next: average over 10 problems per model and compare

Terminology

Related Papers

• The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

  Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.

• 90% of US working hours are spent on tasks not covered well by AI benchmarks

  Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage

• Postgres 19 delayed after AI reviewers find more bugs than humans can fix

  LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted

• Show HN: Gemini 4 Argon plays StarCraft

  LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.

• A Plausible Explanation Is Enough to Mislead Multimodal Judges

  Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.

• Neural networks are clean

  Looking at neurons directly shows most have one main job, challenging superposition.

Related Resources

• https://gist.github.com/wsxiaoys/e0286dc6bb624ff5fdf49e7f4c528ba3
• https://stolen-thoughts.com/paper.pdf