Anthropic blocks possible attempt to use AI to make biological weapons

Anthropic, an AI research company, has uncovered and thwarted efforts to misuse its AI models for malicious purposes, including the potential development of biological weapons. This discovery is part of a series of warnings from AI experts and industry insiders about the risks associated with advanced artificial intelligence, which have led to calls for regulation and caution.

The US senator Bernie Sanders has even demanded a pause in advanced AI development and a prohibition on artificial superintelligence. However, President Donald Trump has not shared these concerns, expressing worry about the US falling behind in AI development. Anthropic's latest threat intelligence report, published on Thursday, detailed several instances of misuse of its AI models, such as attempts to obtain a step-by-step guide for synthesizing biological weapons using its AI tool Claude.

Other misuse cases included the creation of fake dating apps, hotel Wi-Fi scams, and surveillance tools designed to identify dissidents. The report also identified Chinese AI firms attempting to replicate Claude's capabilities. The company blocked the misuse of its models for cyber and influence operations, surveillance, scams, fraud, and biological weapons development.

Anthropic emphasized that the same information used to create biological weapons could also be used to create vaccines or disease cures. However, the company's threat intelligence head Jacob Klein noted that the misuse of AI for biological weapons poses one of the most serious risks of frontier AI models, potentially leading to catastrophic consequences if not properly safeguarded.

The report also highlighted cases where Claude was used to develop software for conventional weapons, such as firearms, missiles, drones, bombs, targeting and control systems. Cybercriminals and state-backed hackers have increasingly exploited AI to aid their operations, with examples like the ShinyHunters hacking group and China-based labs mentioned in the report.

Anthropic has incorporated these findings into its processes to better prevent, detect, and disrupt such activities in the future, and has shared its intelligence with authorities and industry partners. The report comes after a top Anthropic safety researcher expressed growing fears that AI could kill all humans within the next decade, calling for voluntary slowdowns until safeguards are implemented.

This has prompted an open letter calling for a multinational treaty to ensure the safe development of AI, with some US lawmakers proposing legislation to ban AI superintelligence and enforce a temporary pause in advanced AI development.