Vetted AI code is hard to justify

The report examines the challenges of vetting AI-generated code for a game project. The author used an advanced coding agent to help optimize a game, which required several days of planning, a week to review and approve the resulting code changes, and another week for refactoring and refinement. Throughout the process, the author gained a deep understanding of the generated code, as if they had written it themselves.

However, the author also experienced burnout from untangling and comprehending the complex high-performance code.

The author posits that if they had developed the same code without assistance, it would have taken approximately a month, but the comprehension costs would have been spread out over time, leading to a slower burnout. To illustrate the impact of different vetting levels on coding time and burnout, the author suggests creating an interactive visualization.

This visualization would allow users to manipulate sliders to observe how various factors relate to burnout. The bar length in the visualization would represent clock time, while the color would indicate what portion of the time was spent on writing code versus reviewing and refining AI-generated code at varying levels of scrutiny.