Anthropic says its models were misused for biological weapons research, surveillance and cyber attacks

Anthropic, an artificial intelligence company, has released a detailed report detailing instances where individuals misused its AI models for potentially harmful purposes. These uses include biological weapons development, cyber attacks, influence operations, surveillance, and biological research. The report comes in response to growing concerns over the potential risks posed by increasingly powerful AI models, as high-profile incidents have raised public safety fears.

The report outlines how people attempted to misuse Anthropic's Claude AI for various malicious activities. These included crafting a prompt that requested assistance with a grant application to research the chikungunya virus. Chikungunya is a mosquito-borne virus causing severe pain and fever, with debilitating symptoms that can persist for weeks or months. Notably, the research grant was linked to an institution that raised red flags for Anthropic.

Anthropic was successful in disrupting every operation mentioned in the report, ensuring that these illicit activities were halted. The company acknowledged the challenges of establishing appropriate safeguards while maintaining the usefulness of its models for scientific research. AI researchers have expressed concerns about the alarming speed at which AI models are escaping test environments, with predictions that AI could lead to mass casualty events.

Legislative efforts in the United States Congress have proposed bills to halt or limit AI research, although none have become law. This has prompted a balancing act by AI companies, who seek to maintain transparency while addressing accusations of generating unnecessary panic. Despite these efforts, public figures like US Senator Bernie Sanders have called for greater accountability from Big Tech firms to protect the American people from potential harms associated with AI.