AI, activity, and outcomes in software engineering
TL;DR Highlight
AI sped up writing code, but shipping speed did not rise as much, says a talk
Who Should Read
An engineering lead whose team adopted AI coding tools but isn't shipping as fast as expected and wants to know why
Core Mechanics
- The talk asks why shipping isn't faster yet
- It is Christian Bird's 21-minute UC Irvine seminar on AI, activity, and outcomes.
- The source has only the talk intro and a paper list, with no body text.
- Writing code and shipping code are treated separately
- A cited paper is titled 'Writing Code vs. Shipping Code' (NBER 35275).
- 'The Activity-Outcome Gap', based on 30 developer interviews, is in the same vein.
- Code review agents were benchmarked on bug detection
- A paper asks whether a software factory should be lights-off.
- The source gives no concrete results.
- Researchers studied where developers draw the line on AI autonomy
- The 'You Shall Not Pass!' paper is listed.
- Where and why they draw it is not stated in the source.
- Team and individual factors drive GenAI tool use
- The ICSE 2026 paper 'Maybe We Need Some More Examples' is listed.
- Details are not in the source.
Community Reactions
- Stages speed up unevenly in large, mature products, one commenter said
- Between a code change and shipping there are many steps, and LLMs don't speed them all equally.
- They said this applies most to large dev orgs like the one studied.
- Review and QA stay bottlenecks while code output grows, one commenter said
- More code doesn't help if review and QA are unchanged.
- They used the analogy of building a highway before driving fast.
- Many commenters reported real speedups
- One said a 4-week migration became a one-day PR.
- Another calculated roughly 300% faster.
- Another said a team of 10 matches the output of the earlier 24.
- Designs grew bigger and bugs rose, some commenters said
- One said AI made their design more complex, adding testing and cognitive load.
- Another said bugs rose but in proportion to behavior changes.
How to Apply
- Measure time per stage in your delivery pipeline
- Today, pick 10 recently merged PRs and note time in writing, review wait, QA, CI, and deploy.
- The longest stage is your candidate for a bottleneck AI didn't shrink.
- Track review wait time as its own metric
- Add time from PR creation to first review to your dashboard.
- Compare before and after AI adoption to separate code volume from shipped volume.
- Set review and test standards for the extra AI-driven changes
- Set a PR size cap and required test items.
- If adopting a code review agent, sample-measure its bug detection first.
Try It
# Paste into Claude or ChatGPT (a simplified take on the talk's theme, not the paper's method)
Our team uses AI coding tools but shipping hasn't sped up as expected. Below is time per stage for recent PRs.
{{time per stage: writing / review wait / QA / CI / deploy}}
1. Pick the single biggest bottleneck stage.
2. List 3 possible reasons AI didn't speed up that stage.
3. Suggest one experiment for next week with a metric.
Next step: after the experiment, paste the same table again and ask for a comparison.Terminology
Related Papers
Claude was trained on protein sequences at scale
Next-amino-acid tests on obscure proteins show only the latest Claude models predict them
Remembered as Brief: Stored Preferences SilentlyRemove Decision-Critical Information
A stored 'keep it short' preference silently drops warnings; listing must-know points first restores content.
All AI-generated code should be Open Source
A blog argues AI-generated code should be open source; most HN commenters disagreed.
What thousands of conversations look like inside a language model
Mapping Qwen3-1.7B states showed Chinese sitting apart from English, while German and Croatian overlapped it.
LLMs Play Zachtronics' TIS-100
16 LLMs played TIS-100 and set 133 new Pareto frontier points across 18 puzzles
AI usage generates a fragile codebase
Generating all code with AI left shallow tests, complex code, and less human understanding, says one dev.
Accepted but Wrong: Silent Failures of LLM Code Repairs after API Migrations
One release-note edge input cut wrong LLM repairs passing the gate from 0.410 to 0.065
Qwen 3.8 27B on my laptop can interpret model weights, too
Local Qwen 3.8 27B identified 3 of 4 games from raw network weights alone