Anthropic says AI is being used to spy on dissidents

San Francisco-based AI lab Anthropic announced on Thursday that it had shut down numerous instances of state-sponsored surveillance operations that utilized its artificial intelligence (AI) models. The incidents, which took place between January and July, originated in China, Iran, and West Africa, according to a report by Anthropic about the misuse of its models.

These surveillance campaigns aimed at dissidents and minority communities, such as pro-democracy figures in Hong Kong, Tibetan and Falun Gong communities in Asia, Iranian minority communities, and Iranian regime opponents abroad.

One case involved Iranian actors who developed a method to identify individuals through their social media accounts. Another incident involved a contractor employed by Malian national security authorities who used Anthropic's Claude AI to design software enabling intelligence gathering. Anthropic also disrupted efforts by users to design weapons, conduct dubious biological research, and create dating scams.

The AI lab accused Chinese developers of deceptively using Claude to generate responses to user queries while simultaneously distilling that data to enhance their own models. Distillation is the process of using one AI model to train another, effectively stealing resources from American labs like Anthropic and OpenAI. Chinese developers, including Moonshot and Deepseek, have been accused of using this technique without permission.

In one instance, Moonshot relayed nearly 300,000 customer requests to Anthropic through a network of 5,380 fraudulent accounts, mostly located in Singapore and Japan. This data contained sensitive user information, potentially violating privacy agreements. It is unclear if Moonshot notified their customers about the rerouting of their requests to Anthropic.

The research detected and blocked by Anthropic related to biological weapons involved working scientists and their intent was less clear-cut. The actors were researching the mosquito-borne chikungunya virus, a highly-pathogenic strain of bird flu, a family of viruses that include smallpox and mpox, as well as other toxins.