You Have a Review Ceiling. Measure It Before It Measures You.
I sat in on Margaret-Anne Storey's DORA community session last week, and she put a name on the thing I'd been circling since April. It isn't technical debt. Her ACM Queue piece splits software health into three debts. Technical debt is the familiar one: implementation choices that make tomorrow's change harder. Intent debt is the missing rationale, the goals and constraints that say what a system…
Margaret-Anne Storey identified a new software health issue called cognitive debt during a DORA community session. Cognitive debt refers to the erosion of shared understanding among team members, making it difficult to predict the impact of changes and maintain the system effectively. The author, who reviewed pull requests for a living, faced a retrospective where they couldn't answer how many AI-generated pull requests could be handled before the review process broke down.
Cognitive debt doesn't show up as a red number on dashboards; instead, it appears as green metrics improving as review collapses. To measure cognitive debt, the author suggests monitoring detection rates, similar to mutation testing for test suites. By seeding a small number of known-defective changes into the review queue, teams can detect when cognitive debt starts to accumulate.
The author recommends starting with one canary per twenty pull requests and increasing the number as needed. As AI-generated PR volumes increase, detection rates will initially remain stable but will eventually drop off a cliff, indicating the team's review ceiling. Measuring this ceiling can help teams understand when their review process is no longer effective and adjust accordingly.
The author invites teams to measure and publish their detection curves to contribute to the industry's understanding of engineering in the AI era.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
