DIY archivists push budget Nikons to 902,000 clicks to save 1,800 rare books — team trains neural net on Photoshop edits to process 526,000 scans
An epic book preservation effort.
In 2015, a group of three Pakistani friends initiated the Ibteda Digital Library, a project to digitize out-of-print Urdu books, many of which are lithographs. They embarked on this endeavor using their own resources, with no formal budget, driven by a passion for the language. Over a decade, they processed 526,000 scans, reaching a point where manual processing became unfeasible.
One of the researchers then turned to machine learning to automate the post-processing, specifically using a neural network trained on Photoshop edits. The challenge lay in the unique nature of each book, given their diverse formats and the intricacies of the Urdu script, including the use of diacritics and small symbols. The team recognized that training the neural network on manually processed images could establish a source/target correspondence, allowing for more accurate homography fits and automating the cropping process.
Despite the complexity, the final method still required ten calibration crops per book, a manageable amount of manual labor for the overall effort. The processed books are now available in a ZFS pool with BLAKE3 manifests, ensuring their preservation and accessibility.
Written by urgent.news from Tom's Hardware's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
