Claude Opus 5 Code Quality: What Sonar’s Benchmark Reveals

Claude Opus 5, Anthropic's newest flagship model, was released on July 24 and showed significant improvements in code quality in Sonar's benchmark test. The model demonstrated remarkable ability in verifying its own work and catching logical flaws while planning. To verify these claims, the author ran Opus 5 through Sonar's LLM evaluation framework against the same Java benchmark used for every model.

The results revealed a substantial increase in code correctness, with bug density, vulnerability density, and cognitive complexity per line of code all falling, while code smell density, overall issue density, and cyclomatic complexity rose. Additionally, the volume of generated code doubled, with Opus 5 producing 916,813 lines of code, compared to 391,456 lines for Opus 4.8. The increase in functions produced, from 48,734 to 122,650, suggests a shift towards more decomposition rather than simple sprawl.

The model's functional pass rate improved from 82.9% to 88.6%, a gain of 5.7 percentage points over the same tasks. This represents a substantial improvement in the model's ability to generate correct code, moving it past the previous Opus releases, which had scores ranging from 82.72% to 83.66%. Missing completions also improved, falling from 1.22% to 0.83%.

The complexity metrics showed a 10% reduction in cognitive complexity and a 4% increase in cyclomatic complexity per kilo line of code. Both measures remained relatively stable once normalized, indicating either consistent or slightly improved code structure. The reduction in bug density was particularly notable, falling from 672 to 576 per million lines of code, a 14% decrease. This reduction occurred while output volume more than doubled, emphasizing the model's ability to generate cleaner code.

The improvements extended to different severity levels, with all four categories (BLOCKER, HIGH, MEDIUM, and LOW) experiencing reductions in bug count. Notably, BLOCKER reliability impact issues dropped from 38% to 33%, the highest severity tier, indicating a significant improvement in the overall reliability of the code generated by Claude Opus 5.