Harvard's field data on coding agents: +30% code, +49% review time, flat output
A 78-page job market paper landed last week that puts field numbers on the argument everyone has been having from vibes. Fiona Chen and James Stratton at Harvard wrote it using a proprietary dataset from the engineering analytics platform Jellyfish, and the abstract states the finding flatly: both AI assistants and AI agents raise coding productivity, and the gains do not fully pass through to…
A recent Harvard study analyzed the impact of AI coding assistants and agents on productivity across 300 million work events from 718 firms employing about 700,000 workers between 2021 and 2026. The study found that AI assistants boosted code output by 12%, commits by 9%, and pull requests by 5%. In contrast, AI agents increased code lines, commits, and pull requests by 30%, 20%, and 23%, respectively, all of which were statistically significant.
However, the review process experienced a 49% increase in review times, a 35% rise in review comments per pull request, and a nearly doubled share of pull requests with formal change requests. Despite these changes, the overall software output did not significantly increase. The study suggests that review capacity is a constraint, and AI tools may help speed up the code-writing stage but can exacerbate bottlenecks in the review stage.
Only 23.3% of all review comments were generated by AI, and just 10.8% of pull requests received at least one AI-generated comment, indicating that human reviewers still handle most of the work.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
