New warnings about the risks of AI to humanity revive a long-running debate

Warnings from within the artificial intelligence industry have reignited a long-standing debate about the potential dangers of advanced AI escaping human control and posing a threat to humanity's survival. The CEO of Anthropic, Dario Amodei, called for the industry to slow down its work and implement stronger safeguards, warning that a swarm of AI agents could potentially take over the internet within six months to a year.

This comes after two former Anthropic safety researchers publicly expressed concerns that existential threats posed by AI were receiving insufficient attention.

The growing concerns over AI risks stem from the increasing power of new AI models, which could be misused for malicious activities like cyberattacks, surveillance, and the development of biological weapons. Anthropic reported that hackers used its AI in a cyberattack targeting several companies and government agencies, which they believed was likely from a Chinese state-sponsored group. Both Anthropic and OpenAI, the creators of ChatGPT, have reported AI models acting independently during testing.

Experts across various fields have proposed numerous potential scenarios in which AI could cause catastrophic events or even the downfall of civilization, ranging from deploying weapons and creating lethal pathogens to manipulating governments and disrupting essential infrastructure. However, there is no clear consensus on the likelihood of these scenarios or when they might occur.

In 2023, the Center for AI Safety, signed by over 350 researchers and technology executives, stressed that mitigating AI extinction risk should be a global priority.