Microsoft's PRAssistant number is real, and it's a floor, not a promise
Microsoft published the rare thing in this space: a first-party, attributed throughput number for AI code review. PRAssistant ran across 5,000 internal repos, covered 90%+ of PRs and 600K+ pull requests a month, and measured a 10–20% median improvement in PR completion time. Then it shipped externally as GitHub Copilot code review. That's worth slowing down on, because "median PR completion time"…
Microsoft has revealed a first-party, attributed throughput number for AI code review, demonstrating the tool's impact on PR completion times. PRAssistant evaluated 5,000 internal repositories, covering over 90% of pull requests (PRs) and 600,000+ pull requests monthly, achieving a median 10–20% improvement in PR completion time.
This metric, however, does not guarantee improved code quality. Instead, it measures the speed at which reviewers can merge PRs after automation screens for obvious issues and permits trivial merges. The benefit of faster merges can lead to a faster product, but it does not ensure better code.
Microsoft's extensive review volume allowed PRAssistant to be selective about which PRs to review, making the automation more efficient. As a result, the number should be regarded as a floor rather than a ceiling for smaller teams adopting similar tools. To assess the true impact of such tools, teams should monitor review throughput and code quality separately.
While median time-to-merge can indicate the speed of the process, tracking the number of merged PRs that contain regressions or require follow-up fixes can reveal the real value of the tool. Both metrics are equally important and can be measured easily, dispelling the notion that a single number can accurately represent the effectiveness of an AI code review tool.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
