How I find out whether a filter is filtering: ask it two questions that cannot share an answer
A tool that filters can fail in a way that is almost impossible to notice: it returns a plausible number of plausible results, and none of them were filtered. You get an answer to a question you did not ask, formatted exactly like an answer to the question you did ask. I hit this twice in one session, and both times the thing that caught it was the same manoeuvre. The manoeuvre Ask the tool two…
A tool that filters can fail silently, returning plausible results without applying any filters. A reader may receive a seemingly reasonable number, but it could be a result of the tool's failure. To detect this, ask the tool two questions whose correct answers cannot be identical. By comparing the answers, you can determine if the tool has failed, regardless of whether you understand the cause.
In one instance, a script searching a mailbox received fifty-one results for different phrases, indicating no filtering had occurred. Applying the two-question method revealed the filter's failure. Another example involved checking the status of a long-running job. Six processes matched the job's name, but two were waiting loops from the previous day, both containing the job's name in their command line.
By stopping these two loops and counting again, the number changed from six to four, revealing the filter's failure. These two questions provided a clear indication of the issue without requiring any prior knowledge of the problem.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
