Split your OCR error rate in two before you touch preprocessing

In September, a test was conducted to compare the performance of 14 different OCR preprocessing methods on a single crop containing vertical Chinese text. The text achieved a character error rate (CER) between 89.7% and 94.9% across all versions, while the original 1x crop scored 92.3% on all three OCR tiers. This inconsistency indicated that pixel quality may not be the primary factor affecting OCR accuracy.

The OCR samples were taken from a custom web page featuring community service hour information, with additional crops from a holiday notice template obtained online. The recognition was performed using ImgIng, Fast OCR, and Ultimate OCR on an Apple M4 device with a Chromium 149 build, employing the WebGPU backend. Tesseract.js 5 (default parameters) served as the control for comparison.

Upon analyzing the results, it became apparent that the order of characters played a significant role in determining the OCR accuracy. A new metric was introduced to account for this, treating both the reference and hypothesis texts as bags of characters, and counting the shared characters while calculating the remaining errors. High CER with low order-free rates indicated that characters were correct but in the wrong sequence, while high values for both suggested that text was lost or misread.

After identifying the importance of character order, the test proceeded by evaluating the impact of rotation and tilt on recognition accuracy. Vertical text was rotated left by 90°, resulting in zero errors for all three OCR tiers on the 2x crop, while only Fast and Professional OCR made mistakes on the 1x crop (2 punctuation marks error for Fast and no errors for Ultimate). Tesseract.js continued to perform poorly on both rules, regardless of the applied preprocessing.