Anthropic Says Claude Used in Possible Bioweapon Research

Anthropic, an AI company, revealed that its Claude AI model was utilized in biological research that could potentially support the development of bioweapons. The company disclosed five instances of biological research involving Claude, which could be used to develop biological weapons. These cases included studies on chikungunya, avian influenza, orthopoxviruses, venom peptides, and toxins.

Some researchers had spent weeks planning avian influenza experiments and used Claude to draft an orthopoxvirus immune-evasion grant application. Despite blocking some requests and banning certain accounts, Anthropic could not determine whether the scientists intended to cause harm. One case involved a state-sponsored grant for gain-of-function research on chikungunya, aiming to make the mosquito-borne virus more harmful.

Anthropic stated that while the research could contribute to vaccines or treatments, it was unclear if the research was intended to be weaponized. The company emphasized that the older Claude models were not capable of meaningfully assisting sophisticated users with dangerous biological research. However, newer models pose a higher risk, necessitating stronger safeguards around dual-use biology queries.

The report highlights the difficulty of judging biological research on a prompt-by-prompt basis, as the same knowledge can contribute to both vaccines and treatments while potentially making pathogens more dangerous. Organizations using multiple AI providers must consider the activity surrounding a prompt, not just the prompt itself, to prevent the undermining of safety controls.