The Dedicated OCR Engine Lost to the General-Purpose Model — 300 Slower

A test image was created with a fixed answer - Korean title and date, a table and prose. The dedicated OCR engine took 82.8 seconds to process it, while the general-purpose model completed the task in just 0.27 seconds. Despite the speed advantage, the dedicated OCR engine made 2 errors, compared to 8 errors by the general-purpose model.

However, Vision's output structure was flawed - table cells were dropped and the reading order was destroyed. This made it difficult to identify which speed belonged to which model. The general-purpose model, although slower, excelled in the task where the second factor was important - maintaining document structure. This highlights the importance of choosing the right tool for the task at hand, rather than relying on speed alone.

Additionally, the experiment revealed that any escalation ladder, such as using a cheap model first and escalating to a more expensive one, is only valid when the failure of the cheap model is easily detectable. If the failure mode is silent, like dropping information, the "cheap first" approach can lead to undetected data loss.