AI agents went shopping for TVs and laptops—and messed up nearly 50% of the time, study shows

A new study by Visa shows that AI agents often fail when attempting to shop for items like TVs, laptops, and hair appliances. In an unprecedented experiment, Visa instructed 22 different AI agents to purchase three everyday products. While some agents performed better than others, the overall success rate was only around 57%, meaning nearly 50% of the time the agents messed up.

The agents using more advanced AI models, such as Google's Gemini Aragon 4 and Meta's Wonder version, achieved the highest success rates, hitting 82% for both. Claude came in third with a 76% success rate, while an OpenAI agent finished 6th with a 64% success rate. The study's findings are available on a new Visa website, which shows the outcomes of 15 shopping expeditions conducted by each of the 22 agents.

The agents' struggles stem from the fact that consumers often lack information and are unsure about their preferences, making it challenging for AI agents to understand and meet their needs. Visa's Global Head of Growth, Rubail Birwadker, stated that while agent-driven shopping is still in its early stages, it is advancing rapidly.

However, there are still trust and technology issues that need to be addressed. Additionally, legal questions about liability in case of rogue purchases or fraud remain unclear. Visa has also launched a website allowing consumers to interact with the agents used in the study, marking a step towards mainstream acceptance of AI-driven shopping.