What's the largest software project AI can complete on its own?
TL;DR Highlight
HN debates how large a project AI can build alone, per the MirrorCode benchmark
Who Should Read
Developers considering handing large projects to AI coding agents who aren't sure how far to trust them
Core Mechanics
- The source article was not provided
- Only the title and URL (epoch.ai/MirrorCode) were given.
- So no benchmark numbers or results are covered here.
Community Reactions
- Big projects still need a human in the loop
- One commenter has built a Bash clone in Rust with Claude over 81 days and 2,600 commits.
- It sometimes spent hours on a few-line fix and pushed back on big changes despite 3,000+ tests.
- Unsupervised, models do half the work
- A commenter running very large projects with almost no hand-written code said unsupervised models leave stubs and TODOs.
- They found a role-based workflow engine wasted fewer tokens than agent swarms.
- Reproducing software isn't building new software
- Some commenters said reproducing existing software doesn't generalize to developing new software.
- Others noted models may have seen the original code in training, muddying the results.
- The real question is long-term maintenance
- One commenter said the interesting question is how large a system AI can maintain coherently over time.
- Another pointed out MirrorCode only covers plain-text output programs.
How to Apply
- Write completion tests before handing off a task
- Pick one task today and list unit and integration tests first.
- Treat any leftover stub or TODO as a failure.
- Split planning and implementation sessions
- Plan first and add one adversarial critique pass before coding.
- Specify requirements as concretely as possible upfront.
- Limit loops to small bugs
- Build an issue list and let the agent loop on small bugs first.
- Define when you step in if it repeats the same fix.
Try It
# Paste into Claude Code or ChatGPT (simplified version built from commenter experience, since no source text was available)
You are a senior developer. Before implementing, make a plan for the task below.
Task: {{task description}}
1. Clarify the requirements and ask me about anything ambiguous.
2. Write the unit and integration test list first.
3. Critique the weaknesses of your own plan.
4. Implement only after I approve, and leave no stubs or TODOs.
Next to try: when done, ask it to list remaining TODOs and unimplemented items.Terminology
Related Papers
The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge
Warning an LLM judge about bias or letting it self-critique improved 20 of 21 model–bias combinations.
90% of US working hours are spent on tasks not covered well by AI benchmarks
Mapping 29 benchmarks to job tasks shows 90% of US working hours lack good coverage
Postgres 19 delayed after AI reviewers find more bugs than humans can fix
LLM code review surfaced so many bugs that Postgres 19 slipped to November and 8-10 features were reverted
Show HN: Gemini 4 Argon plays StarCraft
LLMs wrote StarCraft bots in one hour; GPT-6 Astra scored highest.
A Plausible Explanation Is Enough to Mislead Multimodal Judges
Adding plausible explanations to fixed answers cut MLLM judges' wrong-answer rejection by 12.7–13.8 points.
Neural networks are clean
Looking at neurons directly shows most have one main job, challenging superposition.