Half of my corrupted text was pure ASCII, and my character check passed it
I generate long documents as files and publish them with a script: reports, proposals, article bodies. The publisher never reads the sentences. It checks that the media paths exist and that the body is not empty, then puts the page live. That gap is where two different text corruptions survived for weeks. One of them is invisible to the check most people write first. The defect that looks like…
Two distinct forms of text corruption went undetected for weeks in a publishing system. The first corruption involved a file with words fused together, such as "marinhoprofundo" where two words should have been separate, and "branco,.grid" where a table header was mixed in with the text. The second corruption was more subtle, as the characters involved were within the seven-bit ASCII range, making them appear clean to most checks.
Examples of this form of corruption included a list item reading "agendamento, CRM,|" and a sentence containing the English verb "priced" within a Portuguese sentence. Despite these issues, the publishing system passed the documents as valid, allowing them to be published with broken text. The root cause was found to be the tool that wrote the files, which occasionally caused chunks of text to have pieces glued together or replaced.
A Python script used as a gate before publishing was designed to catch these defects. It checked for non-ASCII characters, common English words that were improperly inserted, and markdown syntax errors. The script was able to detect the visible corruption but not the invisible corruption, which only revealed itself when a reader noticed the broken text.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
