Anthropic blocks AI weapon plots after ‘catastrophic’ risk warning
Anthropic has blocked attempts to use its AI models to help develop biological and conventional weapons, days after its own researchers sounded the alarm over the risks posed by increasingly powerful AI. The Claude-maker uncovered five cases in which researchers used its technology “in ways that could support biological weapons development”, according to a new [...]
Anthropic, the creator of the AI model Claude, has taken action to prevent its technology from being used in the development of biological and conventional weapons. This decision came shortly after Anthropic's own researchers raised concerns about the risks posed by increasingly powerful AI. In a new threat report, Anthropic revealed five instances where researchers utilized Claude to assist in the development of biological weapons and six cases involving conventional weapons, including the creation of missiles, armed drones, bombs, and targeting systems.
Anthropic emphasized that biological misuse posed one of the most serious risks associated with frontier AI models, stating that without proper safeguards, such capabilities could have catastrophic consequences. The company banned the accounts involved in these cases and shared relevant intelligence with authorities and industry partners where appropriate. The report, spanning 154 pages, offers insight into the practical applications of AI safety concerns.
Examples of misuse include a northern Yemen operation attempting to use different Claude models to write guidance and flight-control software for a guided rocket and long-range ballistic missile. The researchers split their work across multiple conversations and attempted to conceal the ultimate purpose of their actions to bypass Anthropic's controls. While the company did not confirm the production of a functioning weapon, there appears to have been an unsuccessful test launch.
The findings from Anthropic's report coincide with recent comments from a former researcher and Anthropic's current alignment science leader, who expressed concerns about the potential for advanced AI to pose existential risks to humanity within the next decade. Despite these apprehensions, Anthropic maintains that it has implemented some of the strongest safeguards in the industry and has consistently been transparent about the potential benefits and risks of AI technology.
Written by urgent.news from City AM's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
