We worried AI would make things up, we should also worry when it doesn’t
We’ve spent the past few years worrying about AI inventing facts, quotes or events. But what happens when AI isn’t making things up at all, but accurately repeating information somebody deliberately put there for it to find?
For years, marketers have been concerned with influencing what appears online, aiming to become the businesses Google recommends when people search for products or services. This same incentive is now moving into AI, where appearing in AI-generated answers can influence purchasing decisions before people visit websites. As businesses manipulate AI results, the danger lies not only in AI hallucinations—when models confidently invent facts but also in AI accurately repeating information deliberately planted by others.
This "manufactured consensus" can make a claim appear credible and widespread, leading people to accept it as truth without verifying its origins. The consequences are not limited to commercial decisions, as questions about finance, immigration, politics, and medical treatments could also be influenced by AI-generated answers. To address this issue, we need to develop a new kind of media literacy that focuses on verifying the information that produces AI answers, as traditional media literacy taught us to check the reliability of websites.
The real battle in the online world may not be for search rankings or website traffic but for the power to shape perceived truth.
Written by urgent.news from Daily Maverick's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
