AI-generated summaries can manipulate memories, study says

A new study has revealed that AI-generated summaries may manipulate memories, according to research conducted by Georgetown University and the University of Washington. The peer-reviewed paper, titled "AI-Enabled Human Memory Manipulation," highlights how AI overviews of articles and videos often exclude crucial information and misrepresent events.

Across all tested prompts and models, summaries omitted, on average, 51.6 percent of central details. The study specifically examined an animated video depicting a car-pedestrian accident, finding that the accuracy of participants' recall was significantly affected by the quality of the AI summary. While 83.6 percent of participants who read summaries containing accurate information answered correctly, only 44.8 percent of those who read misleading summaries were accurate.

The research utilized popular AI models such as OpenAI’s ChatGPT and Google’s Gemini. Yael Eiger, a PhD student at the University of Washington and co-author of the study, expressed concern about the implications of these findings. She noted that inaccuracy and omissions in AI summaries could have serious consequences, particularly in fields like law enforcement where video summarization technologies may be employed without proper testing or awareness of potential inaccuracies.

Mattea Sim, an assistant research professor at Georgetown, emphasized that the study's results should prompt a reevaluation of when and how AI-generated summaries should be used. Sim pointed out that AI-generated summaries pose a new form of misinformation capable of creating false memories in readers. She urged a critical and thoughtful examination of AI's role in summarizing information, especially in high-stakes situations.

The research will be presented at an AI, ethics and society conference in Sweden organized by the Association for the Advancement of AI in October. The findings are expected to generate interest as concerns surrounding AI continue to grow, encompassing issues like job displacement, misinformation, disinformation, and environmental impacts from the rapid expansion of data centers.

Yoshi Kohno, chairman of computer science, ethics, and society at Georgetown, expressed optimism that the study could contribute to positive change, noting that it is part of a broader effort at Georgetown to explore the intersection of AI and human psychology.