Two Bugs in Our Email Checker Were Hiding Each Other

Two programming bugs had been hiding each other in Thicket AI's email checker, according to a recent article. The first bug caused the checker to incorrectly match vendor names as bare substrings, failing to account for word boundaries. This led to incorrect reporting of vendor names in email reports. The second bug was a result of the first, as the checker relied on the first bug's incorrect matching to incorrectly credit names in email reports.

The pre-send checker in the email system was supposed to confirm the validity of each vendor name quoted in the email and count how many times each name appeared in the model answers. However, due to these bugs, the checker was not accurately reflecting the correct vendor names in the reports.

To fix both bugs, the engineering team added word boundaries to the matcher, improving its ability to correctly identify and report vendor names. The changes were made in the same commit, and the output of the checker was diffed against previous versions using real data. The diff revealed two added lines: one confirming the correct presence of the vendor name "iA Writer" and another confirming the correct interpretation of an email stating that no vendors were named.

After implementing the changes, all batches passed the checks, and no false positives or false rejections were recorded. The fixes were verified by running the old and new code over the same ten batch files and comparing the outputs line by line. However, one limitation remains: the checker still matches names on their first word, which may cause issues with hyphenated vendor names.