I checked 30 frontier model cards. Here are the benchmarks labs report
The graph presents benchmark scores from 30 frontier model cards. Each orange diamond signifies an organization's initial report of a benchmark, which increments the cumulative count. Subsequent cards from the same organization are depicted with gray ticks, maintaining the staircase's flat appearance. Cards lacking a publication date are absent from both bands, yet they are still included in the overall totals.
A flat line in the score track indicates saturation, not a claim of benchmark score saturation. The score track below is generated from distinct readings, connected only when the instrument and protocol remain constant. Each model card is counted once per benchmark, regardless of the number of configurations reported, ensuring that a long appendix cannot surpass a card from a different vendor.
Organizational ties are resolved by the count from multiple vendors, with a shared standard emerging from the collective effort of six vendors, while a single vendor maintains a distinct style. The curated source list serves as the foundation for this ranking. To verify the data, users can expand any card to view all benchmarks it reports, organized in the same manner as the source document.
Each bar represents an independent count, separate from a stacked total. The source mix calculates ranked evidence following the scoring process. Raw records returned by sources are counted before scoring, allowing for a possible empty source despite having an "ok" rating. Refreshing the page or inspecting the latest daily issue while the dashboard rebuilds are recommended solutions if the graph appears incomplete.
Written by urgent.news from Hacker News's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.