The Matte Learns Only Inside the Band

The Matte Learns Only Inside the Band

In the world of automotive imagery, a minuscule flaw can easily go unnoticed at first glance. A vehicle appears pristine on a new backdrop, and the paintwork seems flawless, yet upon closer inspection, a tiny piece is missing - an antenna, a tire lip, or a dark seam beneath a rocker panel. When customers report these issues, it's not because they're technically savvy; it's because the car looks off.

The goal is to refine the final touch to eliminate fuzzy edges without completely reworking the entire model. This seemingly minor distinction becomes crucial the moment a model excels at correcting one boundary while inadvertently harming another.

This article delves into the second part of the discussion, focusing on the runtime phase. Once a mask is established, the learned stage's role comes into play. Part 1, which explored the concept of negative space, examined how surrounding pixels inform a model and the importance of labeling a shadow as evidence against the foreground.

The CarSegNet pipeline splits the image processing route based on media type. Stills undergo a sequence of processing stages: SAM 3 text concept, NSJ alpha, and composite. Video processing involves SAM 3.1 multiplex propagation, per-frame NSJ with temporal handling, a depth-parallax plate, composite, and encoding. The order of operations is more important than the specific components used.

SAM provides a semantic prior, while NSJ receives a trimap band, and the compositor receives a matte after both the prior and refiner have completed their designated tasks.

The diagram illustrates the contract between components. The refiner edits the uncertain strip, while the semantic prior governs the rest of the frame. Models operate lazily, only loading a large segmentation checkpoint when necessary. Cached matte work should remain inexpensive and easily inspected, while loading every model for every run can obscure the orchestration problem behind hardware capacity limitations.

The band, which carries the trimap, is constructed using image disagreement. The NSJ alpha refiner operates independently, with a one-line file description detailing the trimap, band crop, NSJ, or deterministic fallback. A plain band can be created from the prior alone by growing it, shrinking it, and designating the unknown area as a boundary.

Both halves of a morphological band are functions of the prior, so a prior with no doubt produces no band at all. By filling in a wheel opening or removing a roof antenna, the corresponding pixels become confidently classified as foreground or background, even if they don't lie on an iso-contour. The refiner only processes the area near the subject, with a bounded search radius, a relative edge criterion, and a ceiling to prevent the band from expanding uncontrollably.

A learned correction should have a declared edit domain, which in this case is a trimap band carrying a disagreement term. This approach allows for examining confident prior mistakes near the vehicle. However, full-frame alpha drifts quietly, offering a simpler design where segmentation produces a rough mask and a neural refiner outputs a complete alpha matte.

While this approach may seem more efficient, it can lead to hidden authority and mistakes in critical areas, such as tire edges, rooflines, window halos, and mirror reflections. The key takeaway is that boundary scores and overall model performance should not be taken as evidence of the system's understanding of complex vehicle structures.

Instead, separate gates should be implemented to address structural errors like window holes, wheel openings, and glass ownership.