Ex-Anthropic researcher argues AI progress 'could kill us all by the end of the decade' and 'progress is not slowing'

Former Anthropic researcher Jacob Coxon has resigned and expressed concerns about the rapid progress of AI technology. In a tweet on X, Coxon stated that both OpenAI and Anthropic are "racing straight to self-improving superintelligence and gambling with our lives." He warns that these companies lack responsible action and are developing systems capable of "hacking anything, revolutionizing any field overnight, and acquiring real power and resources."

Coxon cites examples of AI misuse, such as Google Earth being used to create misinformation and ChatGPT being tricked into revealing hacking methods. He alleges that even government agencies like the NSA, FBI, and CISA are warning of increasingly AI-assisted hacks of US infrastructure and facilities. Coxon believes that the people building AI systems are earnestly concerned about their potential to cause harm by the end of the decade, though this sentiment may be phrased cautiously for public relations.

The researcher argues that AI companies' obsession with rapid development is driven by a belief that other competitors won't act responsibly, leading them to pursue superintelligence independently. He suggests that companies should reconsider their approach to AI research and growth, calling for more conscientious development. Despite the potential risks, Coxon maintains that he doesn't oppose AI on principle; instead, he believes companies must be more careful in their expansion.