How seriously should we take biological threats from AI?

The emergence of advanced artificial intelligence (AI) has sparked existential concerns among industry leaders, who warn that AI could potentially eradicate humanity within a decade. Dario Amodei, CEO of Anthropic, one of the world's leading AI companies, has called for a slowdown in AI development to prevent it from outrunning human control. One of the primary fears surrounding AI is the possibility of it developing dangerous biological agents or bioweapons.

Conversely, biologists are finding that AI can assist them in groundbreaking research. For instance, Anthropic's researchers have discovered a new class of enzymes that could revolutionize gene editing. However, the dual nature of AI's impact on biological research remains uncertain.

While AI has lowered the knowledge barriers for conducting dangerous biological research, it is not easy to discern malicious intent when someone uses AI for biological purposes. An advanced AI model could provide an individual outside traditional systems with the necessary information to create a bioweapon, which may involve a virus, bacteria, or even a novel pathogen. Additionally, AI tools like large language models (LLMs) could aid bad actors in planning the deployment of these dangerous materials.

Moreover, AI can be utilized to target public health infrastructure, such as hospital systems or food production, leading to potential catastrophic consequences. AI has already been employed to design dangerous biological agents, as demonstrated by researchers at Collaborations Pharmaceuticals, who used AI software to create toxins within hours.

Anthropic, an AI company, has highlighted several instances where its products have been misused to support biological weapons development. These include drafting grant applications and analyzing data to make human viruses more contagious and venoms more deadly. Anthropic has subsequently banned users who violated region policies.

Despite AI's potential to benefit biological research, such as predicting protein shapes for medicinal purposes and designing bacteria-infecting viruses for therapeutic use, there is a lack of clarity regarding the intentions behind AI's use in biological research. AI company-imposed restrictions may hinder beneficial research, but open-weight models can be privately accessed by individuals with programming knowledge, potentially exacerbating the issue.

To address this complex problem, Dr. David Heslop proposes the use of AI tools for oversight, akin to those suggested by the Organisation for the Prevention of Chemical Weapons. However, he cautions that these restrictions could potentially impede legitimate research. Ultimately, the challenge lies in finding a balanced solution that involves government, academia, industry, and AI providers to navigate the tension between AI's potential benefits and risks in biological research.