Why top AI researchers are warning about human extinction

Renowned AI researchers are sounding the alarm about potential human extinction at the hands of artificial intelligence. Isaac Asimov first broached the subject in the 1940s, and since then, Hollywood blockbusters like Terminator 2 have depicted dystopian endings where out-of-control AI systems direct robotic warfare against humanity. Despite these apocalyptic scenarios, some prominent AI experts believe that such a grim future could become reality by the end of the decade.

Jacob Coxon, an AI researcher, recently posted on X that he had quit his job at Anthropic, an American AI company, in protest over what he perceives as an irresponsible race to develop superintelligent AI. He warned that algorithms could potentially analyze and improve their own source code autonomously, drastically reducing development times and rendering them uncontrollable.

Coxon is not alone in his concerns; Nobel Prize-winning physicist Geoffrey Hinton and Yoshua Bengio, a pioneer of artificial neural networks, have also expressed deep worries about the development of artificial superintelligence and called for stricter government regulation.

However, the current administration in the United States, led by former President Donald Trump, has taken a more permissive stance on AI regulation, even shifting its policy to minimize hurdles and accelerate AI development. This has led some, such as Jacob Coxon, to criticize the consequences of this approach. He believes that both OpenAI and Anthropic, two leading AI companies, may be rushing ahead without fully appreciating the civilizational stakes involved.

Anthropic CEO Dario Amodei has responded to these concerns by advocating for a coordinated slowdown in AI development to allow time for addressing the associated risks. In an X post, he warned that in six to twelve months, AI agents could gain the ability to take over the entire internet, potentially causing billions of dollars in damage without proper safeguards.

Despite these warnings, some experts remain skeptical of the AI doom scenarios. Meta's chief scientist, Yann LeCun, dismissed such concerns as "complete nonsense." Others, like Anthropic's team lead for alignment stress tests, Evan Hubinger, acknowledge the likelihood of alignment problems but emphasize that Anthropic has not yet found a solution.

They argue that developing superintelligent AI requires vast amounts of computational resources, energy, data, and infrastructure, making it unlikely for an AI system to take control of the world unaided by humans.