Stop OCRing Every PDF: Route It First with pdf-inspector

OCR is frequently the most costly and time-consuming stage in document ingestion. Many PDFs already contain readable text, yet naive pipelines still perform OCR on every document. pdf_inspector offers a superior method: classify first, extract native text when possible, and only OCR pages that require it. The routing process begins with the PDF arriving, then classifying the document and its pages.

If native text is already present, it is extracted locally and converted to Markdown. If text is missing or broken, those pages are routed to OCR. This simple decision can significantly reduce unnecessary OCR work in RAG ingestion, invoice processing, research paper parsing, and document search. pdf_inspector classifies PDFs as TextBased, Scanned ImageBased, or Mixed, providing a confidence score and identifying specific pages needing OCR.

Using Python, install the package via pip install pdf_inspector, then process a PDF with result = pdf_inspector.process_pdf(document.pdf). The result includes the PDF type, pages needing OCR, and Markdown output. For selective OCR, use the process_pdf_with_ocr function to get the pages routed to OCR. Node.js and browser support are also available through the @firecrawl/pdf-inspector and @firecrawl/pdf-inspector-wasm packages.

Classification preserves structure such as headings, bold and italic text, numbered and bulleted lists, code blocks, tables, links, page breaks, captions, and common font encodings. The output is Markdown, ideal for search indexing and LLM/RAG pipelines. Classification works by inspecting PDF content streams for text operators (Tj and TJ) and image operators (Do).

It can scan all pages, sample large documents, or inspect a caller-provided page set. While the library provides a routing signal, it does not guarantee perfect parsing for every PDF. The library reports encoding issues to allow fallback to specialized parsers if needed. Benchmark numbers are available for a 200-document corpus, showing strong performance but not a universal latency guarantee.

The key takeaway is that OCR should be a per-page fallback, not a default for every file. A conservative router might check the PDF type and confidence score, storing Markdown for text-based PDFs with high confidence or routing pages needing OCR accordingly. This approach balances extraction costs and false positive risks, making it suitable for various use cases like casual knowledge bases or legal and financial workflows.