Anthropic says it blocked misuse of its AI that could have supported biological weapons

Anthropic disclosed on Thursday that it had prohibited researchers from utilizing its Claude AI models in ways that could facilitate biological weapons development. The company detailed this in a comprehensive report, which also highlighted other detrimental activities such as surveillance, scams, conventional weapons development, and propaganda.

The report serves as a case study of the most notable and novel threat activities observed by Anthropic so far, encompassing criminals, suspected state-sponsored groups, spyware vendors, and state propaganda institutions.

Five cases involving potential biological weapons support were shared by Anthropic. The company emphasized that biological misuse constitutes one of the most severe risks associated with advanced AI models. The cases presented offer evidence of AI's capability but cannot definitively prove that such capability would be utilized to create biological weapons in reality. The individuals involved in these cases are working scientists, whose identities Anthropic decided not to reveal.

The company clarified that it cannot assert their intentions. Upon discovering and investigating these instances, Anthropic banned the relevant accounts and integrated the findings into its safeguards, enforcement, and threat intelligence mechanisms to enhance future prevention, detection, and disruption of such activities. Recent, more sophisticated models like Claude Fable 5 boast stronger safeguards, restricting access to a broad spectrum of dual-use biological research queries.

The dual-use nature of AI's information is highlighted, noting that the same data can be employed for both malicious and beneficial purposes, such as vaccine or disease cure development.

Anthropic also identified and eliminated accounts utilizing Claude for influence operations aimed at shaping public opinion, including three Iranian state-aligned accounts. Each operation was orchestrated by an entity associated with an Iranian state propaganda institution. Claude was leveraged to create content, make posts appear as if from independent news sources, and disseminate content across platforms like X, Instagram, and TikTok.

One Iranian threat actor utilized Claude to devise targeting strategies against U.S. naval forces in the region. Additionally, the same account designed software for a domestic mass-surveillance platform catering to Iranian state systems. Claude facilitated surveillance operations by actors in China and West Africa, often employed to identify targets.

Anthropic also revealed instances where AI substituted for an engineering workforce. In China, three accounts linked to the Chinese municipal security service employed Claude for surveillance and transnational repression, encompassing a municipal bureau profiling overseas activists and organizations.