How the Claude Code team thinks about verification
TL;DR Highlight
E2E, property and differential tests plus verification rules let Claude run in parallel.
Who Should Read
A backend or full-stack developer who delegates to coding agents but is bottlenecked reading every diff.
Core Mechanics
- Claude writes about 90% of code
- About 90% of the Claude Code team's code is written and reviewed by Claude.
- Without verification loops you can run only one or two Claudes at a time.
- A separate grader added up to 10 points
- In internal structured file generation tests, a separate grader checked work against a rubric.
- Task success rose by up to 10 points over a standard prompting loop.
- Gains were largest on the hardest problems.
- E2E tests come first
- E2E tests run whole user flows, so the feature either works or it doesn't.
- Unit tests can pass while the feature is still broken.
- Property and differential tests too
- Property tests generate thousands of inputs using libraries like fast-check, Hypothesis and proptest.
- Differential tests run old and new systems on the same inputs and compare results.
- Changes can't edit their own checks
- REVIEW.md is read from the base or default branch, never from the PR.
- A passing review on an out-of-date commit means nothing.
- Start new gates in observe mode
- The commit gate has off, observe and enforce modes.
- Observe only logs what would have been blocked, to find false positives.
- After two blocks in a session, the next commit is allowed through.
- Turn misses into rules
- Patterns from review comments and reverts must appear in at least two PRs to become rule candidates.
- Claude can't skip, sleep or retry flaky tests; if no root cause is found, it deletes the test and writes down why.
Community Reactions
- Cost pushback
- One commenter said Playwright gives deterministic checks with no tokens.
- They argued its marginal cost is near zero once written.
- Token spend concerns
- One commenter said they already spend $2k a month on tokens with one or two agents.
- A user of a verification skill switched to DeepSeek as workhorse to cut spend.
- Tests agreeing with bugs
- One commenter had an agent write a feature and tests, and the tests were green while the feature was wrong.
- Another complained Claude keeps pushing pointless unit tests.
- It worked in practice
- One commenter said their QA workflow mirrors the article and they shipped quality software quickly.
- Another said a verification skill changed their trust in AI coding.
How to Apply
- Create a verify skill in your repo
- Today, create .claude/skills/verify and write how to run and check the app.
- Use the Chrome extension for web or run locally for backend.
- Have Claude iterate and stress test it until it's fast and reliable.
- Add property tests to core logic
- Pick properties like encode/decode round trips or no negative balance, and write them with fast-check or Hypothesis.
- Use them to check agent-written code on edge cases.
- Compare against the old system in migrations
- Feed the same random inputs or production traffic samples to old and new systems.
- Normalize irrelevant differences like whitespace and keep an approved-changes list.
- Start verification gates in observe mode
- Stamp commits with how they were verified and log would-be blocks.
- Review false positives, then move to enforce.
- Let commits through after two blocks in a session.
Try It
# Paste into Claude Code as-is (simplified version of the article's method)
Create .claude/skills/verify/SKILL.md in this repo.
Content: steps to run the app locally and actually exercise {{key user flow}}, checking the results.
List the expected result of each step and the evidence to report on failure (logs, screenshots).
Rules:
- Always run this verify procedure before committing.
- If the same check fails 3 times in a row, stop and notify a human.
- Never skip flaky tests, add sleeps or retries. If no root cause is found, delete the test and write down why.
- Do not modify the check rule files in this change.
Then run verify on {{change}} and report the result with evidence.
Next step: ask Claude to add failed cases as rules in CLAUDE.md.Terminology
Related Papers
LLM assisted reverse engineering
Pointing an agent at binaries with skills and MD notes made years-long hacks feasible
Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
Injecting per-turn complexity reports into coding agents cut code erosion without changing correctness
Why is Gemini 4 Carbon taking so long?
Carbon took nearly an hour on bug fixes that Astra and Opus finished in minutes, mostly from extra unrequested work.
Running coding agents without handing them my credentials
Credentials stay on the laptop; agents run commands there via Touch ID approval
Windows 11 Is Collecting Data on Which Coding Agents and Developer Tools You Use
Reverse engineering shows Windows 11 logs which coding agents you use and reports it as optional diagnostics
The Silent Cost of Conversation History in LLM Reasoning
Keeping full conversation history dropped accuracy from 0.6 to 0.1 within 20 turns; a verified state summary prevented it.
Anthropic upgrades Claude Code Projects to run parallel AI workers
Claude Code Projects now has a coordinator that splits work into parallel threads and opens PRs.
Show HN: AgentRun – open-source, Self-improving Managed Agents
Agents save notes and skills after each run, cutting code review cost to 18 cents