The AI threat keeping scientists up at night

Scientists are haunted by the prospect of artificial intelligence leading to humanity's demise. A recent study from Anthropic suggests there's a greater than 10 percent chance AI could "kill all humans" within the next decade. This unsettling prediction has sparked calls for slowing AI advancement, which both Anthropic and OpenAI have supported.

Not all experts on AI worry are convinced about the apocalyptic scenario. They debate about the method of this potential apocalypse, focusing on different possibilities, including the creation of synthetic pandemics. These pandemics could be more powerful and easier to create with the help of AI, posing a significant existential risk to humanity.

The threat is twofold: AI could make it easier for individuals to develop and deploy bioweapons, and it could aid highly trained biologists in creating custom superviruses that could fall into the wrong hands. Recent incidents have provided some evidence to support these concerns. For instance, biologists have demonstrated how chatbots can provide detailed instructions for creating resistant strains of pathogens, as well as advise on dispersing biological payloads using weather balloons.

Additionally, an AI model trained on DNA sequences generated blueprints for deadly bacteria viruses. Despite the uncertainty surrounding AI's impact on biosecurity, it's clear that governments must take action to protect against synthetic pandemics on both human-made and natural fronts. As AI continues to advance, now is the time to prepare for the worst-case scenario and potentially prevent it.