AI chatbots are recommending fake retail websites, study finds

A study conducted by Podqi, a firm specializing in IP protection, has discovered that AI chatbots such as Gemini, Perplexity, and ChatGPT have been directing users to fraudulent retail websites in up to 15% of their conversations. This research, shared exclusively with The Verge, involved the chatbots being presented with various shopping-related questions concerning 151 different brands.

These questions encompassed inquiries about the top websites for purchasing specific brands, finding those brands online, and the international shipping options of these brands.

Upon conducting the study over two separate testing periods in September, it was observed that Perplexity performed the worst, displaying risky websites in 15% of its answers. Over the course of 6,795 responses, all three AI engines were found to have pointed shoppers towards risky sites, which were defined as fake sites that imitate legitimate retailers or sell counterfeit products, expired brand domains that have been re-registered, or inactive sites that could potentially be purchased and utilized by others.

Google's Gemini showed the least frequent occurrence of risky sites, linking to or discussing such sites in only 4% of its answers. Perplexity exhibited the highest rate of risky sites at 15%. ChatGPT, on the other hand, fell somewhere in between, with 8% of its answers containing risky sites. However, ChatGPT performed the best when it came to issuing warnings about the risky sites it presented, doing so in 24% of its responses.

The study highlighted that smaller brands are particularly vulnerable to this risk. When queries related to smaller companies were made, risky sites appeared in answers approximately half as frequently compared to queries concerning larger companies. As the holiday shopping season approaches, this issue could become even more critical, as consumers are increasingly likely to use AI for shopping tasks, including purchasing items from unfamiliar brands or websites.

To address this concern, Podqi CEO and co-founder Trevor West emphasized the importance of consumers verifying the source of information before entering personal or financial details. Google and OpenAI, the companies behind Gemini and ChatGPT, respectively, acknowledged the issue and expressed their commitment to improving their AI models and providing users with the tools to verify information.