What 10 years of writing code actually changes (it's not the code)
Ask sixteen experienced open-source developers how much faster AI tools would make them, and they'll say about 24%. Ask them again right after they finish the task, and they'll still swear they were roughly 20% faster. The stopwatch says otherwise: they were actually 19% slower. That gap is the headline result from a 2025 randomized controlled trial run by METR , a group that studies how AI…
A 2025 study by METR examined how artificial intelligence tools impact the productivity of experienced open-source developers. The developers were given real issues from their own open-source repositories and asked to either use AI tools or not for each task. Despite believing AI would make them 24% faster, tasks completed with AI assistance were found to be 19% slower.
This discrepancy was due to the additional time required for prompting, reviewing suggestions, and correcting output, which added overhead to work that developers could already do quickly from memory. The study highlights that experienced developers' internal models of systems can mislead them into believing AI tools are faster than they actually are.
To determine if a tool is beneficial, developers should time their tasks and treat the question of its effectiveness as a measurable one.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
