Building TargetV1: A Confidence-Gated Recognition Pipeline (and the Blackwell GPU Bug That Almost Stopped Me)

I built a recognition app called TargetV1 that admits when it's not sure. This project took me a while to develop because of several hardware and software challenges. I started with an RTX 5060 GPU, but the Blackwell architecture was still new, and my GPU wasn't fully supported by the tooling. I had to install the torch, torchvision, and torchaudio packages from the cu128 build to get my GPU working.

I made a few mistakes early on. First, I cloned the model's repository and downloaded the weights separately, which caused issues with Windows paths and corrupted model weights. Second, the code expected layer names that didn't match the weights I had downloaded. To solve these problems, I switched to using Hugging Face's transformers library, which allowed me to use models directly from the library.

I built TargetV1 as a pipeline with five separate models, each focusing on a specific task. DINOv3 creates a numeric fingerprint for image similarity search, SAM finds objects in images without knowing their names, Moondream2 generates plain English descriptions of the images, CLIP scores the confidence of the guesses, and BLIP writes short captions for context. This approach worked better than relying on any single model, as no single model excels at everything.

When the system wasn't confident about its guess, it checked Wikipedia for confirmation. It would compare the Wikipedia image against the original using the same similarity math from the DINOv3 step. If the similarity was high enough, it would confirm the guess; otherwise, it would state that the match was too low.

One unexpected finding was that DINOv3's similarity math gave lower scores than expected, even for correct guesses. This was because DINOv3 focuses on pose and visual style, not just category. To improve this, I plan to switch to CLIP's similarity space, which bridges different image styles better.

Lastly, I added a feature that checks image sharpness before trusting a confidence score. This ensures that the system doesn't trust low-quality images, even if they contain the correct object.