Late Edition: Giving OCR Text a Second Look 📰

This project explores a cautious approach to using AI for correcting OCR text, which is the process of converting scanned text into digital, editable format. The challenge with OCR is that it can misread characters and words, turning a clear page into a confusing mess of text. The developer created "Late Edition," a Python prototype that leverages language models for corrections while maintaining control over potentially risky changes.

Late Edition takes OCR text, breaks it into smaller chunks, and sends those chunks to a specific Gemma model for proposed repairs. Before accepting any suggested changes, the system runs several strict checks. These checks ensure the model doesn't make drastic changes, like altering numbers or capitalizing new words, and that the proposed corrections are at least 80% similar to the original text.

The system also produces an HTML report showing the original text, approved edits, and rejected suggestions. This report is designed to help users inspect the correction process, including changes the system opted not to accept. The project is built with Python and uses Google's Gemini API to access the Gemma model. The main steps include splitting the OCR text into chunks, requesting corrections from the model, and validating the output through the aforementioned checks.

The validation process is intentionally cautious, as a model might produce a coherent sentence even if it inadvertently changes important details. This project doesn't aim to produce a verified historical transcription; instead, it's about exploring a more careful way of using AI for text correction. The code for the project is available on GitHub (https://github.com/ali-scribe/late-edition), where the correction checks are implemented in guard.py and the report generation in report.py.