GPT-6 Astra, looped transformers, and hidden reasoning

TL;DR Highlight

Looped transformers reuse blocks to add depth; the article says this alone doesn't hide chain of thought.

Who Should Read

An agent developer who heard GPT-6 Astra and looped transformer rumors and is rethinking AGENTS.md/SKILL.md files and model choice.

Core Mechanics

Looped transformers reuse blocks
- Intermediate representations pass through the same transformer blocks multiple times.
- Unlike stacking more blocks, the weights are shared.
Nanbeige4.2-3B loops 22 blocks twice
- Block applications go from 22 to 44 while storing only 22 blocks of weights.
- Block parameters are roughly half those of a 44-distinct-block model.
Memory is saved but compute and KV cache are not
- The forward pass still costs 44 block applications.
- The KV cache needs separate entries per pass, same as a 44-block model.
- Nanbeige tried sharing the KV cache across passes, which halved its size but hurt performance.
Loop counts can vary per token
- Universal Transformer (2018) stops looping at a position once cumulative halting probability passes a threshold.
- ByteDance Ouro-Thinking 2.6B runs 48 blocks four times, giving 192 block applications.
- The released Hugging Face code computes all passes before choosing an output, so it is effectively fixed at 4.

Community Reactions

Opinion that looped does not mean hidden reasoning
- One comment explained it just reuses layers and still generates one token at a time.
- It added that OpenAI can monitor CoT, while users haven't seen unsummarized traces since the o1 days.
Counterargument that looped is inherently hidden
- One commenter said feeding output back in without emitting it is hidden reasoning by definition.
- Another said the article is outdated and benchmarks show Astra is much less monitorable.
Experience of being disappointed using Astra for coding
- One user said a feature that would be a ~50-line diff became ~400 mostly useless lines over 10 minutes.
- Another reported overzealous behavior in Codex, such as accessing files it shouldn't.
- Another said Astra was great until Monday and then felt like Sol.
Experience running Nanbeige4.2-3B (Q8) locally
- A user found it compute intensive due to the loops.
- It made the same tool-call mistake as Ling 3 Tiny MoE on the same prompt.

How to Apply

A/B test with half-length instruction files
- Today, make a copy of AGENTS.md and SKILL.md with the parts the model handles on its own removed.
- Run 3 identical tasks with the original and the copy and compare results and token usage.
Compare models with the harness matched
- When choosing a model by benchmark, check which harness was used.
- Run your own tasks in both the primary and your existing harness.
Review Astra coding output by diff size
- If a feature diff is much larger than expected, add instructions limiting scope.
- Restrict file access to the needed directories.

Try It

copy & run

# Paste into Claude Code or ChatGPT: a simple check based on the article's advice to trim AGENTS.md

Below is our project's AGENTS.md.
{{AGENTS.md content}}

Newer LLMs understand prompts well, so extra guidance can constrain solutions.
Classify each item as keep / removal candidate / rewrite, with a one-line reason.
Also produce a minimal version without the removal candidates.

Next: run 3 identical tasks with the original and minimal versions and compare results and token usage.

Terminology

looped transformerAn architecture that passes data through the same block stack several times. Like climbing one staircase twice, you keep one set of layers but gain depth.

KV cacheA memo of previous tokens' attention keys and values reused when generating the next token. Each loop pass has different values, so each needs its own.

adaptive haltingThe model decides how many loops each token gets. Easy tokens stop early, like skipping extra study on easy questions.

RLVRReinforcement learning with rewards that can be automatically verified. It is like practicing problems that have an auto-grader.

harnessThe runtime wrapping a model that handles tool calls and screenshots. The same model can perform differently under different harnesses.