Artificial Analysis unveils the Intelligence Index v5 with new coding and science benchmarks; 60% of the Index is based on private test sets or held-out answers (Artificial Analysis)
The Artificial Analysis Intelligence Index is our composite measure of model capabilities across tasks reflecting real-world usage today. v5 …
Artificial Analysis has released the Intelligence Index v5, a composite measure designed to assess language model capabilities across tasks reflecting real-world usage. The index is calculated as a weighted average across four equal categories: Knowledge Work, Coding, Science, and General, each representing 25% of the total. According to Techmeme, 60% of the index is based on held-out answers or private test sets.
The v5.0 index incorporates nine specific evaluations, including AA-Briefcase v1.1, AutomationBench-AA, GDP.pdf, FrontierCode 2.0, Terminal-Bench 4.0, AA-Omniscience v1.1, HLE-Diamond, CritPt v2 (Hard Subset), and Terminal-Bench-Science 0.1.0. Hacker News reports that the suite is primarily a text-based, English-language evaluation used to assess reasoning, knowledge, math, and programming.
While the index compares language models, Hacker News notes it has limitations and may not apply to every use case. Benchmarks for speech, image inputs, and multilingual performance are conducted separately from the Intelligence Index suite.
Brief written by urgent.news from Techmeme, Hacker News — 2 reports on this story. Machine-written — may contain errors; check the original before relying on it.
