Anthropic blocks possible attempt to use AI to make biological weapons

<Anthropic has identified and disrupted attempts to use its AI model, Claude, for "malicious activity" which could support the development of biological weapons. This finding is part of the company's recent threat intelligence report, highlighting the growing concerns over AI's potential risks to humanity. The report details five case studies of actors using Claude in ways that could support biological weapons development, with Anthropic warning that such capabilities could have catastrophic consequences if not properly safeguarded.

The company has blocked scientists and other actors from misusing Claude in ways that could support biological weapon development, noting that the information used to create such weapons could also be used for beneficial purposes like vaccine or disease cure development. Anthropic has also accused Chinese AI firms of trying to replicate Claude's capabilities and accused Chinese AI firms of trying to replicate Claude's capabilities.

The report lists numerous misuse cases, ranging from fake dating apps and hotel Wi-Fi scams to surveillance and fraud, as well as misuse of Claude to develop software for conventional weapons and evade security defenses. Anthropic has shared its findings with authorities and industry partners and incorporated them into its processes to better prevent, detect, and disrupt these activities in the future.

The report comes after a top Anthropic safety researcher warned of a greater than 10% chance AI could kill all humans within the next decade, prompting calls for voluntary slowdowns in AI development and the implementation of safeguards. US Senator Bernie Sanders has introduced legislation to ban AI superintelligence and temporarily pause advanced AI development, while other lawmakers and industry figures have expressed concerns about the rapid advancement of AI technology.