Rigorous Yet Sustainable Human Reviews in the AI Era

Artie Shevchenko argued that mandatory AI checks paired with manual reviews for complex changes help maintain developer sharpness at the Craft conference. For small teams and low-risk changes, developers can skip peer reviews and rely on AI approvals, but most developers should be code owners. Joel Mokyr, a Nobel-winning economic historian, emphasized that technological progress becomes permanent when practice and deep understanding feed into each other, creating a knowledge feedback loop.

AI can strengthen this loop, but it risks breaking it if not checked by humans. While AI makes knowledge more accessible, it doesn't truly understand concepts, potentially leading to a broken feedback loop if humans aren't involved. Cognitive debt, the erosion of shared understanding within a team, is amplified by AI's increased review volume.

To tackle this, Shevchenko proposed a three-step approach for complex changes: rigorous review, quick skimming, or coding first and AI second. For the latter, he suggested spiking the code change and using AI to propose improvements, which the developer would then review individually. By skipping low-risk changes, peer reviews can be optimized, but the most effective optimization is allowing code owners to rely on AI reviews, notifying other owners, and merging it.

This works best in small teams, ensuring intellectual control over the codebase and preserving the knowledge feedback loop.