olmOCR-bench on Hugging Face: Three Rows Use a Different Average, and One Moves From 13th to 7th or 19th

On November 8, 2026, the Hugging Face olmOCR-bench leaderboard presented a peculiar ranking of 24 repositories based on their OCR performance. Out of these, 18 rows were derived from the main branches of the repositories, while the remaining six came from open pull requests submitted by a single user account.

The overall ranking was determined by averaging eight columns, encompassing seven test categories and a baseline column, for a total of fifteen repositories. However, three rows - ranked 13, 20, and 23 - utilized a distinct average that excluded the 'headers_footers' category. Their respective notes explicitly indicated this deviation.

Additionally, six more rows remained unverifiable due to various reasons: one was gated, another only provided a mean value, and four contained fewer than eight columns. In the case of rank 13, repository 'LightOnOCR-2-1B', the overall score of 83.2 was derived from eight columns averaging 75.24. Removing the 'headers_footers' category shifted its position to the 7th place among 22 rows, whereas including it placed the row at the 19th position among 21 rows. Consequently, rank 13 presented a blend of both scenarios.

The leaderboard demonstrated a stark contrast in rankings across the board, with rank 1 remaining consistent across multiple recomputations, and rank 2 surviving all but a test-count weighting. Rank 1 excelled in three out of the eight columns, occupying the 16th position among 17 when considering the baseline. Moreover, six different repositories led the eight columns individually.

It is important to note that the dataset card accompanying the olmOCR-bench also provided its own table of results, encompassing a range of ± 0.9 to 1.2 for the Overall scores. Therefore, when assessing the top positions on the leaderboard, it is prudent to consider this range.