AI Code Review at Scale: LinkedIn's Multi-Agent Approach
At LinkedIn's scale, relying solely on human reviewers or simply putting an off-the-shelf AI reviewer in front of GitHub is not an effective way to manage PRs. To address this, LinkedIn engineers built a multi-agent AI code review platform that understands the organization’s coding context, treats code review as production infrastructure, and minimizes hallucinations and low-signal feedback. By…
LinkedIn has developed a multi-agent AI code review platform to handle the vast number of pull requests (PRs) it receives on a scale that would overwhelm human reviewers or off-the-shelf AI reviewers. Recognizing that generic AI models often miss codebase-specific standards and conventions, LinkedIn aimed to create a system that generates high-quality, relevant reviews that developers would actually act upon.
The primary challenges faced by LinkedIn were ensuring factual accuracy in the AI-generated review comments, customizing the reviews to align with the organization's policies and repository conventions, and maintaining operational control over the review process. These challenges led to the design of LinkedIn's multi-agent platform, which employs distinct AI models and reasoning approaches for each review, ensuring comprehensive coverage and cross-validation of findings.
By utilizing a Kubernetes-based architecture with an event-driven pipeline, LinkedIn can monitor various metrics such as latency, acceptance rates, and completion rates, while also handling potential provider failures. The platform's multi-agent design enables cross-validation of identified issues, treating convergence as strong evidence when multiple agents independently flag the same problem. However, unique findings aren't automatically discarded; they undergo separate verification to ensure quality.
To assess the effectiveness of the AI-generated suggestions, LinkedIn built an automated pipeline that evaluates how often developers implement the recommended changes. In a comprehensive evaluation covering 5,230 sampled review comments across 1,727 PRs, LinkedIn found that 90.1% of the suggestions could be evaluated with high confidence based on the merged codebase.
Of the evaluated suggestions, 63.9% were ultimately accepted, with the highest acceptance rates for logic errors (80%), bug fixes (58.1%), concurrency bugs (43.5%), security-related fixes (40.6%), and all concurrency bugs (100%). These results demonstrate the platform's ability to deliver actionable, high-quality code reviews at scale, effectively addressing the limitations associated with relying on single models or off-the-shelf AI reviewers.
Written by urgent.news from InfoQ's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
