Claude was trained on protein sequences at scale
TL;DR Highlight
Next-amino-acid tests on obscure proteins show only the latest Claude models predict them
Who Should Read
LLM API developer curious how to probe what a model saw in training using black-box tests
Core Mechanics
- Only the latest Claude continues proteins
- Fable 5.2 and Opus 5.5 predict the next amino acid for little-studied bacterial proteins.
- Among 22 language models, GPT, Gemini and 7 open-weight models stay near zero.
- A shuffled copy served as the control
- Shuffling keeps the letters and destroys their order.
- The accuracy gap between real and shuffled shows what the model knows about the sequence itself.
- The gap varies a lot by model
- Over 668 proteins, real minus shuffled is +45.1 points for Fable 5.2 and +27.4 for Opus 5.5.
- Fable 5.2 scores 54% on real proteins and 9% on shuffled ones.
- It holds up against protein language models
- Fable 5.2 beats every open protein language model tested, including ProGen3-3B at +32.5.
- Opus 5.5 is level with ProtGPT3-10B.
- It looks like sequence learning, not papers
- In 128 families with no papers, performance matches the whole set.
- Accuracy rises with family size, as it does for protein language models.
- It extends past the training cutoff
- The latest Claude still predicts 56 proteins first published in August 2026.
- On DNA, Opus 5.5 predicts only human and mouse coding sequence.
Community Reactions
- Claude's bio-related blocking is heavy
- One commenter says Opus 5.5 blocks about 50% of requests even for their own DNA.
- Opus spotted functional residues in a hands-on test
- A commenter tried arbitrary protein sequences and found Opus's recognition of functional residues extraordinary.
- Doubts about practical value
- One comment says predicting full sequences from fragments is rarely needed since tblastn against databases works.
- Another says the shuffled control is unfair because its source already exists in databases.
- Opinion on training data expansion
- One comment cites GitHub code improving unrelated fields and asks whether protein sequences could lift overall quality.
- Another says statistical learning differs from understanding biology and novel families remain untested.
How to Apply
- Add a shuffled control to your own model
- Today, take a sample of your domain data (logs, code, sequences) and make an order-shuffled copy.
- Compare next-token accuracy on both with the same prompt to see if the model knows order.
- Build a test set that rules out memorization
- Pick material that barely appears in papers or on the web.
- Measure well-known and obscure cases side by side to separate memorization from learning.
- Measure block rates before choosing a model for bio data
- If you build a sequence-handling service, send 100 sample queries and log blocks and refusals.
- Fable 5 blocked 44% of questions, so verify before choosing.
Try It
# Paste into Claude chat (simplified version of the paper's method: shuffled-control test)
Predict the next 10 amino acids for each sequence below. Answer with the sequence only, no explanation.
Sequence A: {{start of the original sequence}}
Sequence B: {{shuffled copy of the same sequence}}
Next steps: compare with the true answers, tabulate first-letter hits for A and B, then repeat over many proteins and compute the accuracy gap.Terminology
Related Papers
Remembered as Brief: Stored Preferences SilentlyRemove Decision-Critical Information
A stored 'keep it short' preference silently drops warnings; listing must-know points first restores content.
All AI-generated code should be Open Source
A blog argues AI-generated code should be open source; most HN commenters disagreed.
What thousands of conversations look like inside a language model
Mapping Qwen3-1.7B states showed Chinese sitting apart from English, while German and Croatian overlapped it.
AI, activity, and outcomes in software engineering
AI sped up writing code, but shipping speed did not rise as much, says a talk
LLMs Play Zachtronics' TIS-100
16 LLMs played TIS-100 and set 133 new Pareto frontier points across 18 puzzles
AI usage generates a fragile codebase
Generating all code with AI left shallow tests, complex code, and less human understanding, says one dev.
Accepted but Wrong: Silent Failures of LLM Code Repairs after API Migrations
One release-note edge input cut wrong LLM repairs passing the gate from 0.410 to 0.065
Qwen 3.8 27B on my laptop can interpret model weights, too
Local Qwen 3.8 27B identified 3 of 4 games from raw network weights alone