Why face index 2 is not a stable ID across detectors

In the world of image editing, a single face detector may not accurately identify the same person across different detectors. A group-photo editor can utilize one detector in the browser and another on a worker, resulting in varying numbers of face boxes. When a user selects the second thumbnail, the worker must determine which face to edit.

Simply sending faceIndex: 1 is insufficient, as it only refers to a specific entry in the detector's output, not the actual person in the photograph. Even sorting the lists in a consistent order does not guarantee that a missed detection or a false positive will be resolved.

The issue at hand is a coordinate correspondence problem. It does not involve face recognition, identity verification, or proof of consent for editing. To address this, the selection should be represented in image coordinates rather than relying on the thumbnail's presentation data, such as padding or square crop. Normalization handles proportional resizing, but it cannot compensate for rotated images, different crops, mirrored previews, or discrepancies in EXIF orientation interpretation.

A selected detection must be associated with an opaque source revision, which should be discarded when the source changes. Generating a source revision on the browser side can prevent accidental stale selections. When the source changes, the server still needs to verify ownership and bind the job to the actual uploaded asset.

One approach to compare the selections is through intersection over union (IoU) measurements. This metric assesses the overlap between two boxes and serves as a baseline when both detectors generate reasonably comparable face boxes. Rank the worker's detections against the selected browser box, then reject both a weak match and an ambiguous match.

The IoU function calculates the overlap, while the matchSelection function determines whether the selection and detections meet certain criteria, such as a minimum IoU and gap threshold.