Urgent.News

What's breaking now, across hundreds of outlets.

AI

Survey Finds AI-Generated Code Increases Debugging and Failure Rates and Creates a Comprehension Gap

A survey conducted by independent research firm Coleman Parkes on behalf of Undo, a company focused on scaling AI-powered root-cause analysis, found that while AI coding agents have accelerated code generation, they have shifted the primary bottleneck to debugging, code comprehension, and maintenance. By Sergio De Simone

A recent survey conducted by Coleman Parkes on behalf of Undo reveals that the increasing use of AI-generated code has negatively impacted debugging and failure rates, as well as comprehension among engineers. The survey of 300 senior engineer leaders working with mission-critical software found that while AI coding agents have accelerated code generation, they have shifted the primary bottleneck to debugging and maintaining the code.

The survey's focus on mission-critical codebases highlighted understanding and debugging AI-generated code as the most demanding tasks for engineers. Teams spend an average of 9.8 hours per week generating code, but 16.9 hours per week debugging issues, accounting for 42% of their working week. This increased emphasis on debugging is coupled with a comprehension gap, as engineers no longer have the same level of inherent understanding of the code they produce.

Approximately 35% of generated code reaches production before the team fully understands it, and 80% of respondents reported that coding agents struggle to solve complex problems in complex codebases. As a result, about one-third of teams utilize AI agents for comprehension and debugging only in straightforward codebases, relying on additional techniques to ensure confidence when working with more complex systems.

Other significant challenges reported by surveyed teams include production incidents or service outages affecting internal users or customers (experienced by 81% at least once in the past six months), incorrect root-cause or issue diagnosis due to hallucination (93% at least once), and test escapes, which involve serious defects or poorly optimized code entering production (experienced by 91% at least once).

Despite AI agents being able to generate code significantly faster, 79% of engineering leaders note that the overall release cycle has not become faster due to the shift in effort toward debugging and unraveling AI-generated code. Undo's founder and CEO, Greg Law, emphasized that engineers lose days trying to unravel what went wrong with AI-generated code that is almost, but not quite right.

He also noted that while AI agents are excellent at quickly generating code, they are less capable at debugging it.

Rewritten by urgent.news; may contain errors — check the original.

Read the original at infoq.com →

Report a problem with this story

More in AI

More from Wednesday 7 October →