Human, Beware: The Secrets We Tell AI Chatbots Aren’t Private

Many people are unknowingly sharing sensitive information with AI chatbots, according to a new survey called The AI Privacy Problem by DuckDuckGo Research. Roughly one-third of respondents admitted to sharing secrets with chatbots that they wouldn't discuss with friends, family, or medical professionals. This number jumps to 56% among those who consider themselves AI enthusiasts. Parents, in particular, revealed personal details to AI that they wouldn't share with their children.

While some people seek assistance from AI for health, financial, parenting, and marital issues, the concern lies in the lack of control over how companies handle and use this data. Once shared, the information can be utilized for training AI systems, accessed through subpoenas by law enforcement, and even employed by employers if used in a work setting.

AI companies attempt to portray chatbots as trustworthy confidants, but this is misleading. The DuckDuckGo report highlights that these chats are not private, as many respondents were unaware of how their data is used or stored. Nearly 40% of respondents lacked trust in companies, and nearly 50% had no confidence in the US government to protect their data.

The survey uncovered that AI data-sharing extends beyond targeted advertising. If sensitive information is provided to a chatbot, it could potentially be exposed or disclosed. A researcher from Wake Forest University discovered that AI agents, utilizing large language model (LLM) data to execute autonomous tasks, may unintentionally leak data. Chatbot conversations are recorded and saved unless users opt-out, serving as additional training for AI models.

Researchers warn that what is shared in "private" chats could end up in court or during police investigations. Companies using AI models like Anthropic (Claude), Google (Gemini), and OpenAI (ChatGPT) are legally obligated to provide chat data if subpoenaed. This was evident in several recent cases, including a double murder investigation in South Carolina and a murder trial involving a Snapchat AI conversation.

AI chat conversations are considered business records, not confidential consultations, as stated by MemX, an AI memory agent. In fact, transcripts from ChatGPT were utilized during a double murder investigation in South Carolina and a Palisades arson case. A Snapchat AI chat was also used as evidence in a 2024 murder trial.

Most alarmingly, OpenAI founder Sam Altman revealed on a podcast that if someone discusses sensitive information with ChatGPT and later faces a lawsuit, the company could be required to provide the chat data. Despite this, nearly three in five survey respondents were "uncomfortable" with their chatbot conversations being used for AI training, and 30% expressed "very uncomfortable" feelings about it.