AI usage generates a fragile codebase
TL;DR Highlight
Generating all code with AI left shallow tests, complex code, and less human understanding, says one dev.
Who Should Read
A backend or SaaS developer using coding agents at scale who worries about rising incidents and maintenance costs
Core Mechanics
- AI writes shallow tests
- It writes many tests for trivial get/set methods.
- Its e2e tests cover only cases already in the codebase, like one type of user or surface interactions.
- Those gaps are invisible to line-level coverage metrics.
- AI writes overly complex code
- It uses two or three helper methods where one would do.
- It follows guidelines like DRY to a fault, which makes code harder for humans to debug.
- Reliance on AI reduces human understanding
- The longer AI writes the code, the fewer developers understand the codebase.
- They struggle to notice PRs with abstractions that do not fit existing structures.
- AI outpaces review
- Each PR arrives faster and gets less review time.
- Skimming misses testing gaps and poor naming, which pile up into spaghetti code.
- Four signs of a fragile codebase
- Incidents rise and recovery takes longer.
- Errors only users see appear, and user experience degrades.
- The author blames humans not checking that AI adds analytics and error handling.
- The evidence is personal experience
- At Meta the author generated 90%+ of code with AI, but output did not even double.
- A peer tried to merge JS events for mouseover effects instead of CSS :hover.
- The post gives no incident statistics or comparison data.
Community Reactions
- Catch AI patterns with linters
- Commenters suggested static analysis or lint rules to fail patterns like JS mouseover events so the agent rewrites them.
- One reported that statically typed languages and functional style leave tests only for what types cannot assert.
- Rebuttal: the problem is missing review
- One commenter said they are the only colleague still reviewing every line, and they find flaws like over-checking edge cases and adding libraries for <1% cases.
- Others noted hand-written large codebases also decay into spaghetti, so the same quality gates should apply.
- It worked fine for some
- One commenter reported a 500k LOC mobile app built over 7 months with no consequential fragility.
- Another with a 220k-line Python/Rust/C++ codebase said Python worked best while C++ came out bloated.
- Verification loops matter
- Commenters said without good tests or specs the feedback loop turns haphazard and quality declines.
- Some reported using instructions like 'do not overengineer' plus adversarial reviews and deterministic gates.
How to Apply
- Audit AI-written tests today
- Pick one recent AI-written test file and look for trivial get/set tests and e2e cases that only repeat existing ones.
- Use the example prompt to list missing user types and failure paths.
- Add an anti-overengineering rule to agent config
- Turn recurring bad patterns into lint rules
- When review flags the same issue twice, encode it as a lint or static analysis rule.
- Wire CI failures back to the agent so it fixes them itself.
- Set review rules for AI-generated PRs
- Set a PR size cap and required review items such as test gaps and new abstractions.
- Track monthly incident counts and recovery time to compare before and after AI adoption.
Try It
# Paste into Claude or ChatGPT (a simple prompt version of the article's 'shallow tests' check)
Review the code and tests below.
[Code]
{{code}}
[Tests]
{{tests}}
Answer the following.
1. Which tests are trivial get/set tests or duplicates?
2. Which user types, failure paths, and edge cases are not covered?
3. List the top 5 gaps that coverage numbers would not reveal, in priority order.
4. Point out needlessly complex parts (extra helpers, excessive DRY).
Next step: have it write tests for the top 2 gaps found in items 1-3.Terminology
Related Papers
Claude was trained on protein sequences at scale
Next-amino-acid tests on obscure proteins show only the latest Claude models predict them
Remembered as Brief: Stored Preferences SilentlyRemove Decision-Critical Information
A stored 'keep it short' preference silently drops warnings; listing must-know points first restores content.
All AI-generated code should be Open Source
A blog argues AI-generated code should be open source; most HN commenters disagreed.
What thousands of conversations look like inside a language model
Mapping Qwen3-1.7B states showed Chinese sitting apart from English, while German and Croatian overlapped it.
AI, activity, and outcomes in software engineering
AI sped up writing code, but shipping speed did not rise as much, says a talk
LLMs Play Zachtronics' TIS-100
16 LLMs played TIS-100 and set 133 new Pareto frontier points across 18 puzzles
Accepted but Wrong: Silent Failures of LLM Code Repairs after API Migrations
One release-note edge input cut wrong LLM repairs passing the gate from 0.410 to 0.065
Qwen 3.8 27B on my laptop can interpret model weights, too
Local Qwen 3.8 27B identified 3 of 4 games from raw network weights alone