The specter of AI-enabled bioweapons is a wake-up call for biotech

Recent warnings from leaders of major AI companies suggest that their groundbreaking technology poses significant risks. Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman have both cautioned that AI should be developed more cautiously. Jacob Coxon, an AI researcher, left his position at Anthropic after claiming that both companies were not acting responsibly, expressing concern that AI could potentially cause human extinction within a decade.

Evan Hubinger, another Anthropic employee, echoed these fears, estimating a 10% probability that AI could kill all humans within the next decade. A primary concern is that AI could aid in the creation of bioweapons, such as highly lethal viruses, genetically targeted diseases, or toxins that could be introduced into water supplies undetected.

In 2022, researchers at Collaborations Pharmaceuticals demonstrated that an AI molecule generator could quickly produce 40,000 molecules potentially usable as chemical warfare agents, some of which were more toxic than known nerve agents. AI tools can now provide instructions and video training on conducting experiments, and the accessibility of gene editing and synthetic biology tools has lowered barriers for individuals to set up labs at home.

While safeguards exist, such as DNA screening for suspicious requests and red-teaming and blue-teaming approaches to identify misuse, these measures are not foolproof. For instance, Anthropic acknowledged that users had attempted to use its models to enhance the transmissibility of the chikungunya virus, develop more dangerous avian flu strains, or create an atlas of venom peptide toxins.

Despite differing opinions among scientists about the extent of the risk, many agree that the situation warrants heightened vigilance. Experts emphasize the need for improved surveillance, screening tools, and the continued development of AI capabilities to identify and mitigate potential misuse.