What Independent Benchmarks Say About Opus 5.5
Two independent benchmark results for Opus 5.5 came out this week, and they don't really agree. One has it in first place. The other has it fast and cheap, but third on secure code once you take out the answers it memorized. A few days ago I read through Anthropic's Opus 5.5 docs to see what they say got better. That's Anthropic talking about their own model, though. I wanted to see what people…
Two independent benchmark results for Opus 5.5 were released this week, but the findings are inconsistent. One analysis places Opus in first place, while another considers it third in secure code performance. These results come from two separate evaluations: Artificial Analysis and Endor Labs.
Artificial Analysis' Intelligence Index ranks Opus 5.5 first with 58 points, followed by OpenAI's GPT-6 Astra and Anthropic's Fable 5.1, both at 53 points. SciCode (scientific programming) saw Opus 5.5 leading with an 11-point margin over Astra and 4 points ahead of Fable. However, the indexing was conducted at the highest settings, which may not reflect typical usage.
When considering cost, Opus 5.5 is cheaper than GPT-6 Astra in terms of token usage, but Astra completes tasks with fewer tokens, making it more cost-effective per task. However, Fable 5.1 incurs higher costs per token and per task. Average costs per task are $5.98 for Opus 5.5, $3.26 for Astra, and $7.63 for Fable 5.1.
Endor Labs focused on secure code, testing a coding agent within open-source projects to ensure it adheres to security best practices. Opus 5.5 finished third with 68.7% FuncPass and 33.5% SecPass, compared to Fable 5.1's 87.2% FuncPass and 37.4% SecPass. The memorization of known fixes was a significant factor, with Opus 5.5 scoring 51 out of 51 confirmed cases, versus 17 for Opus 5 and 38 for Claude Code.
In summary, while Opus 5.5 excels in speed and cost, its performance in secure code remains less impressive compared to Fable 5.1. Analysts suggest testing models on personal code to gauge their suitability better.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
