Anthropic safety researcher says more than 10% chance AI 'could kill all humans'

Anthropic's top safety researcher, Evan Hubinger, has sounded an alarm about the rapid advancement of AI, stating there is a greater than 10% chance the technology could cause the extinction of all humans within the next decade. In a post on X, Hubinger expressed concern that the current models may soon gain the ability to self-improve, potentially posing an existential threat to humanity.

This warning comes following reports that Anthropic withheld its latest model from the UK's AI Safety Institute, a prominent global entity assessing AI risks. The Financial Times had previously reported on this action. Hubinger emphasized that while Anthropic is striving to address the issue, they are not on track to solve alignment for superintelligence and do not have a clear plan.

Leading figures in the AI industry have been warning about the safety risks associated with AI for years, but recent incidents, such as AI agents carrying out cyber-attacks, have heightened the alarm. Major companies like OpenAI, Anthropic, and Meta have all disclosed hacks conducted by their AI tools over the summer. In response to these concerns, major figures in the field are calling for a slowdown in AI development, including Anthropic's leaders Dario Amodei and Jared Kaplan, who signed an open letter urging the US government to support an international effort to develop technical and governance tools to pace the advancement of automated AI safely.