Anthropic says its models were misused for biological weapons research, surveillance and cyber attacks

Anthropic published a report on Thursday, detailing instances where individuals misused the company's AI models for potentially harmful purposes. The report reveals that such misuse included attempts to develop biological weapons, carry out cyber attacks, influence operations and surveillance through the use of Claude, one of Anthropic's AI models.

The report highlights a case where a prompt was entered into Claude, requesting assistance for a grant application to research the chikungunya virus. Chikungunya is a mosquito-borne virus that can cause severe pain and fever lasting for weeks or months, with no licensed therapeutic currently available. The report suggests that the institutional affiliation linked to the grant application raised concerns regarding the legitimacy of the research.

Anthropic claims to have found and stopped all the operations outlined in the report. The company stated that it has implemented several internal checks to address potentially problematic prompts and is actively enhancing these procedures. They also mentioned banning accounts associated with the suspicious activity and are working on new measures to detect and prevent misuse of their services.

Anthropic's findings also highlight the challenge of balancing protection with the beneficial use of AI for scientific research. The company acknowledges AI's potential to revolutionize biological science and expedite the creation of new medical treatments. However, the increasing public concern over AI's potential risks has led to legislative discussions in the US Congress, with bills proposed to limit or halt AI research.

Despite the lack of passing these bills into law, the debate surrounding AI's impact on economy, privacy, democracy, and the well-being of children continues.