A Weighted Sum Is Not an AND

A single arithmetic line nearly enabled any tree on Earth to open a sealed memory, and a change I implemented after it did is the focus of this account. When I was a child, my sister and I buried a biscuit tin in the garden, along with drawings, a cricket card, and a note for future discoverers. We drew a map on notebook paper, but lost it.

We spent more time digging up half the garden than we did on any photo in my camera roll. Each memory exists in a scrolling grid, sorted by an unknown timestamp, viewable from anywhere, and holds little value. So, I built the app version of the tin, called Vantage Point, an Android app written in Kotlin, Jetpack Compose, offline-first, and end-to-end encrypted, allowing users to bury memories at specific locations.

Photos, voice notes, and letters are stored in the capsules which remain sealed until someone physically stands in front of them and points a camera. Memories can also be time-locked for years, or shared with others. I planned the app as nine milestones and completed eight, leaving only the Launch phase. The core of the product hinges on the question: is this the right object in the right place?

This story details how I initially got this question wrong and how a single change fixed it. Many products combine multiple noisy signals into one confidence score, such as fraud scoring, liveness checks, sensor fusion, or LLM-as-judge rubrics. The same bug I encountered is easily written and hard to detect once implemented. The obvious version fails immediately.

The naive design asks an on-device labeler what's in the frame, compares its labels to the buried items, and opens if they match. Using ML Kit's default image labeler, showing a photo of an oak in my garden returns "Tree," which is true of every tree on Earth. A label match indicates the general class of the object but provides no information about which specific object.

As a rough gate, it is sufficient but inadequate for identity checks, which are worthless and any product built on it would open on the wrong object with absolute confidence. Therefore, I redesigned recognition as a composite of three legs: label agreement, visual similarity, and proximity. Each leg contributes to a weighted average score acting as a confidence rating.

While the design document stated the object and the place were required, the code only needed two of the three to pass. The weighted average is not an AND; it is a vote, where strong legs can compensate for weak ones. This bug went unnoticed for a long time because the weighted average score appeared within the acceptable range, and tests confirmed it.

The fix involves using a different kind of average: multiplying the weighted legs instead of adding them. When one leg approaches zero, the whole result collapses towards zero, regardless of the other legs' scores. This AND property ensures containment without additional design. The most significant consequence was a hard boundary enforced by the multiplication, which made it impossible to bypass the security check by combining strong label matches and visual similarity alone.