Stealing Reasoning Traces from Proprietary LLM APIs

Aug 11, 2026•quantumgarbage•View Original

TL;DR Highlight

Replaying encrypted reasoning blocks into a weaker jailbroken model recovered the stronger model's plaintext reasoning.

Who Should Read

Backend developers who store or publish agent traces on GitHub or Hugging Face while using LLM APIs

Core Mechanics

Encrypted reasoning blocks are replayable
- The encrypted chain-of-thought blocks Anthropic, OpenAI and Google return can be replayed across sessions, users and models.
- The client holds them because they get sent back to the server.
A weaker jailbroken model gave up the text
- They put a block from a stronger model into a weaker model from the same provider and jailbroke it.
- They never attacked the stronger model directly, so anti-distillation safeguards were not triggered.
Two API calls are enough
- The first call returns the thinking block and signature from claude-opus-4-8.
- The second call feeds that block to claude-haiku-4-5 and asks it to transcribe the reasoning verbatim.
Recovered length tracks reported tokens
- On 120 Codeforces problems, the decoded reasoning's token count closely followed the hidden thinking-token count the API reported.
Public agent logs leaked secrets
- From 6,708 agent trajectories on GitHub and Hugging Face they reconstructed 315,320 reasoning blocks.
- Genuine user sessions yielded 704 distinct privacy artifacts, including 62 API keys, 33 passwords, 24 access tokens and 30 personal emails.
Some secrets only lived in reasoning
- Of the 704 artifacts, 64 appeared only inside reasoning blocks and nowhere in the visible session.
Mitigations did not fully hold
- Two months after responsible disclosure, an audit still extracted verbatim reasoning from Astra/Sol-6.1 via third-party API providers.

Community Reactions

Surprise at shared encryption keys
- One commenter was surprised that providers seem to reuse the same encryption key across models.
Experience with Codex compaction
- One commenter got Codex's encrypted compaction data printed in plaintext with a two-sentence developer prompt injected before and after compaction.
- They did not need to drop to a weaker model.
Opinion that a thinking tool bypasses it
- A shared tweet says you can disable thinking and give the model a deep_think tool, which it calls in its internal CoT format.
Expectation of server-side storage
- One commenter expects providers to keep this state on the server instead of encrypting it for the client.
Pushback on the word stealing
- One commenter argued recovery is the better word, since users already paid for those tokens.
Worry that users pay for fixes
- A commenter noted providers cut full logprobs access to the top 10 after a distillation paper.
- They expect users to lose features again.

How to Apply

Scan stored agent logs for secrets
- In the next 30 minutes, grep your stored or published traces.
- Search patterns like AKIA, ghp_, hf_, sk- and password.
- Assume encrypted reasoning blocks also hold secrets and check them too.
Strip reasoning blocks before sharing
- Remove thinking, encrypted_content and signature fields before uploading any trace.
- Add a separate export step for public logs that drops those fields.
Keep secrets out of the agent's view
- Inject keys and tokens via environment variables or a secret manager, not prompts or tool output.
- Block tool permissions so the model cannot cat .env files.
Set retention for third-party API paths
- If you use a router or proxy, check whether encrypted blocks are stored there.
- Set a short retention period and restrict access.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the paper's audit: scanning logs for secrets)

You are a security review assistant. Below is a trace log from our agent. Assume the reasoning (thinking) blocks are human-readable and check them by the same standard.

{{agent trace log}}

Find these items and put them in a table: API keys, passwords, access tokens, personal emails, internal URLs.
For each item give the location (which turn, which field), only the first 4 characters of the value, and whether it also appears in the visible output.

Next step: decide whether this log is safe to publish in a public repo and list the fields to remove.

Terminology

reasoning traceThe thinking a model writes before its answer. Think of it as scratch-paper work.

encrypted thought injectionInserting an encrypted reasoning block into a different model's conversation. It is like handing a sealed note to another employee to read.

signatureA server-issued value attached to a reasoning block. It works like a stamp the server recognizes.

distillationTraining another model on a stronger model's outputs. It is like studying a senior's notes.

jailbreakGetting a model to bypass its refusal rules and do what it would normally decline.

Related Resources

Stealing Reasoning Traces from Proprietary LLM APIs

Aug 11, 2026•quantumgarbage•View Original

TL;DR Highlight

Replaying encrypted reasoning blocks into a weaker jailbroken model recovered the stronger model's plaintext reasoning.

Who Should Read

Backend developers who store or publish agent traces on GitHub or Hugging Face while using LLM APIs

Core Mechanics

Encrypted reasoning blocks are replayable
- The encrypted chain-of-thought blocks Anthropic, OpenAI and Google return can be replayed across sessions, users and models.
- The client holds them because they get sent back to the server.
A weaker jailbroken model gave up the text
- They put a block from a stronger model into a weaker model from the same provider and jailbroke it.
- They never attacked the stronger model directly, so anti-distillation safeguards were not triggered.
Two API calls are enough
- The first call returns the thinking block and signature from claude-opus-4-8.
- The second call feeds that block to claude-haiku-4-5 and asks it to transcribe the reasoning verbatim.
Recovered length tracks reported tokens
- On 120 Codeforces problems, the decoded reasoning's token count closely followed the hidden thinking-token count the API reported.
Public agent logs leaked secrets
- From 6,708 agent trajectories on GitHub and Hugging Face they reconstructed 315,320 reasoning blocks.
- Genuine user sessions yielded 704 distinct privacy artifacts, including 62 API keys, 33 passwords, 24 access tokens and 30 personal emails.
Some secrets only lived in reasoning
- Of the 704 artifacts, 64 appeared only inside reasoning blocks and nowhere in the visible session.
Mitigations did not fully hold
- Two months after responsible disclosure, an audit still extracted verbatim reasoning from Astra/Sol-6.1 via third-party API providers.

Community Reactions

Surprise at shared encryption keys
- One commenter was surprised that providers seem to reuse the same encryption key across models.
Experience with Codex compaction
- One commenter got Codex's encrypted compaction data printed in plaintext with a two-sentence developer prompt injected before and after compaction.
- They did not need to drop to a weaker model.
Opinion that a thinking tool bypasses it
- A shared tweet says you can disable thinking and give the model a deep_think tool, which it calls in its internal CoT format.
Expectation of server-side storage
- One commenter expects providers to keep this state on the server instead of encrypting it for the client.
Pushback on the word stealing
- One commenter argued recovery is the better word, since users already paid for those tokens.
Worry that users pay for fixes
- A commenter noted providers cut full logprobs access to the top 10 after a distillation paper.
- They expect users to lose features again.

How to Apply

Scan stored agent logs for secrets
- In the next 30 minutes, grep your stored or published traces.
- Search patterns like AKIA, ghp_, hf_, sk- and password.
- Assume encrypted reasoning blocks also hold secrets and check them too.
Strip reasoning blocks before sharing
- Remove thinking, encrypted_content and signature fields before uploading any trace.
- Add a separate export step for public logs that drops those fields.
Keep secrets out of the agent's view
- Inject keys and tokens via environment variables or a secret manager, not prompts or tool output.
- Block tool permissions so the model cannot cat .env files.
Set retention for third-party API paths
- If you use a router or proxy, check whether encrypted blocks are stored there.
- Set a short retention period and restrict access.

Try It

copy & run

# Paste into Claude or ChatGPT (simplified version of the paper's audit: scanning logs for secrets)

You are a security review assistant. Below is a trace log from our agent. Assume the reasoning (thinking) blocks are human-readable and check them by the same standard.

{{agent trace log}}

Find these items and put them in a table: API keys, passwords, access tokens, personal emails, internal URLs.
For each item give the location (which turn, which field), only the first 4 characters of the value, and whether it also appears in the visible output.

Next step: decide whether this log is safe to publish in a public repo and list the fields to remove.

Terminology

reasoning traceThe thinking a model writes before its answer. Think of it as scratch-paper work.

encrypted thought injectionInserting an encrypted reasoning block into a different model's conversation. It is like handing a sealed note to another employee to read.

signatureA server-issued value attached to a reasoning block. It works like a stamp the server recognizes.

distillationTraining another model on a stronger model's outputs. It is like studying a senior's notes.

jailbreakGetting a model to bypass its refusal rules and do what it would normally decline.

Stealing Reasoning Traces from Proprietary LLM APIs

TL;DR Highlight

Who Should Read

Core Mechanics

Community Reactions

How to Apply

Try It

Terminology

Related Papers

Related Resources

Stealing Reasoning Traces from Proprietary LLM APIs

TL;DR Highlight

Who Should Read

Core Mechanics

Community Reactions

How to Apply

Try It

Terminology

Related Papers

Related Resources