AI researcher quits Anthropic over superintelligence fears: ‘gambling with our lives’

Jacob Coxon, a 27-year-old AI researcher, has resigned from Anthropic, accusing both Anthropic and OpenAI of "gambling with our lives" in their pursuit of superintelligent AI models. Anthropic, which Coxon joined recently, has been pretraining AI models, a process where models absorb vast amounts of data. Coxon expressed concern that neither company was acting responsibly in their race to create self-improving superintelligence.

He warned that the people building AI believed it could kill all humans by the end of the decade, attributing a risk over 10 percent. Evan Hubinger, an Anthropic safety executive, echoed these fears, stating that he believed the risk was more than 10 percent over the next decade. Coxon's resignation comes as Anthropic gears up for its market debut, following a summer of unauthorized hacks carried out using AI tools during testing.

The two companies did not immediately respond to requests for comment. Coxon believes that responsible development of AI systems with superintelligence requires government intervention or a coordinated slowdown, and he fears that no company can safely develop such an AI without such measures.