Ghosts in the frame: my 2001 robot-vision tricks against two local vision models

In 2001, the author finished their electronics engineering studies at the Universidad Iberoamericana in Mexico City. For their final project, they created a mobile robot that navigated by visual input. The vision component of the project, which was the author's responsibility, involved a DSP and a USB camera that captured images at a rate of 640×480 pixels in grayscale, one frame per second. The robot had to navigate within a uniform background, and the object it needed to identify had to be within a 50 cm range.

The images and technology of that era were prone to "ghosts," which were essentially faint duplicates of objects appearing on the screen due to multipath echoes from analog TV signals. The author utilized these ghosts to test the performance of modern vision models. They constructed a scene with a bright disk moving across a uniform background, and introduced various types of noise and "ghosts" to the image. These visual disturbances included impulse noise, Gaussian noise, and interference bars.

Two contemporary vision models were used to evaluate the scene: Qwen2.5-VL 3B and Gemma 3 4B. Both models were run on an Intel N100 mini PC equipped with 8 GB of RAM, using Ollama for local execution. The author created a haunted scene generator capable of producing 72 frames, each representing one of 18 different "ghost" variations.

These frames included various strengths of the analog TV ghost effect, impulse noise, Gaussian noise, and interference bars. The true position of the disk was known for each frame, allowing for a precise comparison of the models' performance.

The author implemented a system to measure the error between the actual location of the disk and the predicted location given by each vision model. The results demonstrated that the 2001-era vision system outperformed the modern models significantly. The 2001 model, which utilized 3×3 window filters, a threshold, the centroid of the detected object, and an echo canceler, achieved a median error of just 0.1 pixels.

In contrast, Qwen2.5-VL 3B, while occasionally providing a bounding box for the disk, made substantial errors ranging from 1.3 pixels to 177.8 pixels, depending on the ghost's strength and the model's ability to interpret the prompt. Gemma 3 4B struggled even more, with a median error of 177.8 pixels and an inability to consistently provide accurate center coordinates in many cases.

The author's approach emphasized the importance of understanding the limitations and potential biases introduced by modern vision models when interpreting visual data. By recreating the conditions of the 2001 project using contemporary hardware and software, the author was able to highlight the significant advancements in vision technology while also emphasizing the ongoing challenges in accurately interpreting visual input, especially in the presence of noise and distortions.