Building an AI-native engineering org
TL;DR Highlight
Automating each shipping step raised merges per engineer about 5x with reverts under 0.5%
Who Should Read
A team lead or backend developer who adopted coding agents and now sees bottlenecks in review, CI and bug intake
Core Mechanics
- Merges per engineer up about 5x
- Since January, code changes merged per engineer per day grew about 5x.
- The revert rate stayed under half a percent.
- Handing out coding agents alone didn't get them there.
- Speeding one step jams the next
- Whenever one step got faster, work piled up at the next.
- So they went through code, tests, review, CI, bug intake and merging one step at a time.
- Concrete numbers per step
- Changes with tests rose from 15% to 91%.
- Changes reviewed by AI before a human rose from 60% to 99%.
- CI runs 20x the tests in the same 17 minutes via parallelization and running only affected tests.
- Tribal knowledge became Python checks
- Rules only engineers knew, exposed by bugs or review comments, became small Python checks.
- The checks run on every commit and PR, fail the build and print the fix.
- Agents hit them and fix their own code.
- Rules went from zero to 21.
- Low-risk changes merge without people
- Low-risk changes that pass tests, automated checks and AI review merge on their own.
- Up to a third of changes now merge with no person involved.
- It wasn't free
- Much of the summer's engineering effort went into tests and tooling.
- Customer-facing changes per engineer stayed between 17 and 30 a month.
- September jumped to 45 and October is on pace for 48.
- The new bottleneck is deciding what to build
- In a ten-day feature (3 plan, 6 build, 1 review), making build 3.5x faster speeds the whole feature only 1.75x.
- Planning and review rise from 40% to 70% of the time.
- So they bet on giving engineers more ownership of planning, execution and review.
Community Reactions
- Merge count is the wrong metric, some say
- One commenter said tracking output per engineer incentivizes over-engineered, bloated code.
- Several commenters pointed out there were no business metrics like revenue, NPS or customer satisfaction.
- Hands-on experience says review is still needed
- One commenter said Claude's error messages had irrelevant technical details and no fix instructions.
- Another said generated code went beyond scope and changed a public API, so they nearly redid it.
- Legacy orgs pay a big adoption cost
- A commenter said enabling agentic coding over a large legacy codebase with near zero defect tolerance takes major effort.
- They said the gap versus greenfield vibe coding is enormous.
- What worked at an AI-native company, per a commenter
- Giving engineers ownership and product decisions improved engagement.
- Review still matters as a way to share codebase knowledge across the team.
- A simple change touching 27 files should be treated as a tech debt signal.
How to Apply
- Pull repeated comments from your last 10 merged PRs
- Within 30 minutes today, skim recent PR review comments and pick 3 rules only humans know.
- Note whether each can be checked mechanically.
- Turn tribal rules into Python checks
- Write each chosen rule as a small script that runs on every commit and PR.
- Make it fail the build and print how to fix it.
- Let the agent self-correct from that output.
- Cut CI to change-affected tests
- Parallelize tests and run only those relevant to the change.
- Check that CI time doesn't grow as agents open more PRs.
- Time each stage to find the bottleneck
- Write down plan, build, review and CI time for one feature.
- Pick the longest stage after build speeds up as the next target.
Try It
# Paste into Claude or ChatGPT (the article's 'rule → automated check' method as a prompt)
Below are review comments that repeated across our team's PRs.
{{list of review comments}}
Pick the rules that can be checked mechanically from the code alone.
For each rule, produce:
1. A short Python check script that can run on every commit and PR
2. Behavior that returns a failing exit code so the build breaks
3. A message that prints how to fix the violation
Next step: wire the checks into CI, then tell your coding agent to fix its code itself using the printed instructions when a check fails.Terminology
Related Papers
AI isn't replacing SREs, it's deskilling them
Letting AI handle 95% of incidents may dull the skills SREs need for the critical 5%.
Backdoor in Claude Code for VS Code extension on the Cursor Marketplace
A tampered Claude Code extension on Cursor Marketplace planted an SSH backdoor
We ran AI agent swarms in Minecraft and Mindustry
We put 50 agents in Minecraft and they invented ration ledgers and bureaucracy.
The Agent SDK
Cursor released Agent SDK, letting you define agents in code and deploy them with one command.
Here be monsters: What do you still need to know when agents do the work?
Owning agent-written code depends on fundamentals, breadth, and diving deep just in time.
AI promised less work. I've never worked more
Running agents in parallel brought my GitHub activity back to full-time engineer levels, and I'm exhausted.
Show HN: Corgi, a helpful {agent, terminal} multiplexer
A tmux-like terminal tool that shows the status of all your AI agents at a glance
Ask HN: If the models are so good, why doesn't OpenAI just start 100 companies?
HN answers: code is easy, knowing what to build isn't, so they sell tokens instead