Clearing up a robot’s fuzzy memory

A new memory system developed by MIT researchers aims to improve robots' ability to correctly identify and locate everyday objects. Called UQ-DAAAM (Uncertainty Quantification for Describe Anything, Anywhere, at Any Moment), the system addresses errors that can occur when robots mislabel objects in their memory. For example, a robot might mistake a wall-mounted defibrillator for a fire extinguisher, causing it to fail to retrieve the life-saving device during an emergency.

The new method quantifies the robot's uncertainty about its object descriptions and incorporates additional images to correct mistakes. By constantly updating its memory in real-time, UQ-DAAAM boosts the robot's confidence and accuracy in answering questions about objects, their locations, and when it last observed them. This uncertainty quantification framework builds upon the researchers' prior work, DAAAM, which uses a vision-language model to create detailed captions describing objects in a robot's environment.

The new system selects additional viewpoints to reduce uncertainty and combines these captions into a more accurate description of the object, which is then saved to memory. Tests showed that UQ-DAAAM improved the robot's confidence and accuracy by 5 percent compared to existing methods. The researchers believe this uncertainty quantification technique could enhance the performance of various types of robots, from household and industrial models to mobile robotic manipulators, making complex tasks more feasible for these machines to handle safely and effectively.