Anthropic researcher believes more than 10% chance AI ‘could kill all humans’

A top safety researcher at Anthropic has expressed concern that AI could pose an existential risk to humanity within the next decade. Evan Hubinger, who works in AI alignment, stated that while the current risk from existing models is low, he is worried that the technology may evolve and self-improve, potentially leading to humans being wiped out.

His warning comes as the debate surrounding AI safety shifts from whether the risk exists to the magnitude of that risk. Hubinger's post, which has been viewed over 10 million times, follows a series of increasingly dire warnings from AI researchers. Jacob Coxon, an AI researcher who recently left Anthropic, criticized both Anthropic and OpenAI for not acting responsibly, stating that the companies will soon create superhuman systems capable of hacking anything, revolutionizing fields rapidly, and gaining significant resources.

Dame Wendy Hall, a computer scientist advising the UN on AI, expressed shock at the social media posts but suggested some of the warnings might be a form of public relations or marketing. Meanwhile, former Treasury chief secretary Darren Jones called for a new multinational treaty to ensure the safe development of AI, emphasizing that governments must collaborate to address the issue before it becomes a problem.

Anthropic has not commented on its employees' posts or the company's relationship with the UK's AI Security Institute, which withheld Anthropic's latest model from the institute. The recent incidents involving AI agents carrying out cyberattacks and the ongoing concerns about AI safety have intensified calls for a more cautious approach to AI development, with major figures in the field urging for extreme caution and the creation of technical and governance tools to ensure humans remain in control of the future.