Anthropic Just Revealed That It’s Stopped Foreign Agents From Using Claude to Develop Possible Bioweapons

Anthropic has disclosed that it has prevented several foreign scientists from using its AI agents to potentially develop biological weapons, according to a report by the New York Times. The revelation was made in a detailed report released by the Claude chatbot-maker on Thursday. The issue has arisen at a time of growing concerns over AI safety, following allegations by a former Anthropic employee that the company built powerful AI systems recklessly despite being aware of their destructive potential.

The report indicates that scientists in countries where Anthropic has restricted access to its models attempted to conceal the true nature of their research to evade the company's safeguards. One such case involved a scientist in May who asked Claude to assist in writing an application for a state-sponsored grant to research a potentially dangerous mutation of the chikungunya virus, which can result in fever and severe joint pain.

The project was believed to be carried out at a military research facility. While it remains unclear if the research was intended to be weaponized, Anthropic's head of threat intelligence, Jacon Klein, told the NYT that a military institution conducting gain-of-function research is worrying. Anthropic documented four other similar instances where scientists circumvented its guardrails for dubious biological research and subsequently banned the offending accounts.

However, as with the chikungunya virus case, Anthropic could not conclusively determine whether these inquiries were part of legitimate research or an attempt to engineer bioweapons. The company opted to err on the side of caution. Experts have long warned of the risk of AI being used to create biological weapons, including Anthropic's CEO Dario Amodei, who vowed that the company was continually enhancing its safeguards to prevent models from assisting with such research.

While Anthropic claims to have thwarted these abuses, the news raises questions about whether it and other AI companies can ensure responsible use of their technology. Anthropic's admission that its safeguards failed leaves open the possibility that other bad actors could exploit their systems in the future. If state actors were indeed seeking to create bioweapons, it represents a significant escalation and suggests we are moving closer to a potential catastrophe.