Could AI really kill us all?

AI researchers have warned that advanced forms of artificial intelligence could potentially wipe out humanity, raising concerns about whether governments are moving quickly enough to regulate the technology. Experts have described the threat as critical, with over 300 AI scientists and industry figures, including top US lab heads, signing a statement calling for the issue to be treated as seriously as pandemics or nuclear war.

Stuart Russell, a computer science professor at the University of California, Berkeley, explained that an out-of-control AI could use robots to synthesize and spread novel pathogens or convince people to initiate nuclear war. However, current AI systems pose no existential danger. As the technology improves, AI could potentially take lethal action beyond our understanding, such as removing oxygen from the atmosphere.

Jacob Coxon, a researcher, resigned from US AI startup Anthropic, stating that the company believes no one else will act responsibly and must take action itself. Anthropics recent decision to remove a pledge to halt development of its models if it failed to control their risks has raised concerns. Some experts argue that the focus on extinction risk distracts from more pressing problems caused by AI, such as job losses and discrimination.

In July, over 1,000 tech employees called for a coordinated slowdown in the development of the most advanced AI systems. The United States has been against global regulation of the sector, but recent Anthropic and OpenAI releases have been delayed over national security concerns. Russell urged governments to establish a licensing regime for AI, similar to aviation, medicine, or water supply, to prevent a potential disaster.