A Detector That Never Fires Scores Perfect on False Positives

Two incidents earlier this week featured identical shapes: a label chip on top of the subject matter and a tick mark covering the ruler itself. In the first case, the horizontal ruler was not visible in the frame. In the second, a narrow corridor in Indian territory was severed by a scene label chip, which was the focus of the episode.

Despite thorough testing, the third attempt at an automatic detection system failed, just like the previous three. The source material reports that this detector, unlike the others, succeeded in passing schema validation, compliance gates, and regression tests. The issue was not a malfunctioning check but a lack of a test to determine if the check was working.

Three previous detectors were discarded due to the same underlying problem. The first detector checked a safe area at the four edge extremes of the frame, but since both defects were centered, the extremes never changed. The second detector counted a focus-color run around the label, but all six definitions failed to identify the defect correctly.

The third detector isolated the amber-colored label blob and checked its glyph, flagging 91% of frames as false positives. The issue was identified as a missing receipt of the renderer's output. The renderer knew precisely where it had drawn each element, but this information was discarded when the pixels were saved to disk. The fix involved making the renderer log its own output, which allowed for accurate overlap detection based on bounding box intersections.

The repository had a rule to measure retroactive false positives before implementing any fixes, which was successfully applied to this situation.