The Anonymous Math Geek Who Quit Anthropic—and Became the Face of AI Safety

Over the past week, the world has been grappling with the potential risks of artificial intelligence, as revelations about the dangers of AI systems that could end humanity started to gain traction. This began with the anonymous resignation of Jacob Coxon, a 27-year-old British researcher from Cambridge, who had previously been a part of the "rationalist" community in London and briefly dabbled in commodity trading.

Despite his lack of an online presence and minimal recognition, Coxon's decision to quit his job at Anthropic sent shockwaves through the AI industry.

Coxon warned that OpenAI and Anthropic researchers believed their technology could pose an existential threat by the end of the decade, describing the labs' efforts as "gambling with our lives." His resignation sparked an unexpected wave of concern, leading rival lab leaders to unite and call for a slowdown in AI development. President Trump downplayed the issue, calling for a strong and smart U.S. president, while China dismissed the concerns as "fearmongering."

Coxon's sudden prominence as a face of AI safety came as a surprise to him, as he had previously remained an anonymous figure in the tight-knit AI safety advocate community.

His decision to leave the industry on his own was influenced by friends and members of the AI safety community, many of whom are funded by effective altruism, a philosophy focused on preventing AI harm. Coxon's comments have drawn attention from both AI boosters and critics, with the latter accusing him of being a whistleblower and promoting stronger global regulations favorable to the biggest companies.

Coxon rejects this label, stating that he simply shared what AI researchers already discuss privately. He now finds himself at the center of a debate between accelerationists, who believe the benefits of AI outweigh the risks, and researchers who want to slow down its development before AI companies lose control.