Anthropic researcher says some developers believe AI 'could kill us all'

An Anthropic researcher, Jacob Coxon, has resigned from the AI company, stating that those developing the technology believe it could lead to human extinction by the end of the decade. Coxon, who spent the past three years at Anthropic and OpenAI, criticized both companies for their aggressive approach to creating self-improving superintelligence. He claimed that the companies are "racing straight to self-improving superintelligence and gambling with our lives."

Evan Hubinger, a lead at Anthropic's Alignment Science, echoed Coxon's concerns, stating that the AI could indeed pose an existential risk to humanity. He personally estimated a 10 percent chance of this happening within the decade. Anthropic is not on track to solve the alignment problem for superintelligence, according to Hubinger.

In July, over 1,300 employees from leading AI firms signed a statement expressing concerns about the rapid acceleration of AI development and called for the US government to support an international effort to pace the frontier of automated AI development. Anthropic and OpenAI have responded to this initiative, acknowledging the need for tools to deliberately pace AI development.

United Nations rights chief Volker Turk also warned of the potential threat AI poses to humanity and demanded cast-iron guarantees to ensure its safety and security. He called for an all-out effort to address these risks before it is too late.