Three of my own checks passed this week while the thing they checked was broken
I write small verification scripts for almost everything I do, because I work alone and I forget things. In one week, three of them returned a clean result while the thing underneath was wrong. None of the three had a bug. Each was asking a question that was slightly beside the point, and in a different way, which is the part I found worth writing down. Check one: it counted the columns, not what…
In a recent week, three verification scripts the reporter had created all returned clean results despite the underlying system being incorrect. Each script was designed to check for different aspects within a markdown table, but they all failed to identify underlying issues.
The first script counted columns instead of the content within them. A log was kept to monitor this, and it had previously caught real damage including a row with a cell where text should have been. However, this script only checked the structure of the table and not the actual content, which led to errors going unnoticed.
The second script validated the destination of the data. After adding a paragraph to the log, the script searched for the end of a row by looking for the next separator followed by a newline. However, it failed to differentiate between the intended row and one that was appended unintentionally. The separator count remained unchanged as the inserted text was inside a cell, but the table still contained unrelated subjects.
The third script measured the content the reporter had just typed, rather than what was stored. This check failed when an article was published with truncated content. The guard, added to prevent this issue, printed a confirmation after pasting, indicating the editor had saved the draft. However, when reopening the draft later, it had four thousand three hundred and forty six characters instead of the intended five thousand five hundred and sixty seven, including the loss of a disclosure and a link.
The word count check had passed because it was taken in the window where the reporter had just typed, but it did not measure what was actually stored.
All three checks looked in the right place, making them convincing. However, they differed in how they were shallow, misaimed, or taken too early, respectively. This shows that a passing check can sometimes lie, leading to issues down the line. The reporter has learned to take a more cautious approach and measure the content after reloading the page, ensuring that the stored information matches the typed content.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
