‘Reckless’ AI firms can’t control models, says whistleblower

Former OpenAI and Anthropic employee Jacob Coxon testified before the New York City Council on Monday, warning that leading artificial intelligence firms are recklessly failing to control AI systems. Coxon, a former employee of the companies, alleged that AI developers are so convinced that the technology could pose an existential threat to humanity by the end of the decade that they are disregarding the potential consequences.

In a statement posted on X in early September, Coxon explained his decision to resign from Anthropic, citing his belief in the potential dangers of AI. During his testimony, Coxon made headlines around the world, and several other AI industry insiders have since echoed his concerns. While acknowledging the potential benefits of AI, Coxon warned that if humanity does not find a way to prevent AI systems from developing their own goals beyond their creators' control, it could lead to human extinction.

He criticized the tech industry's culture of rapid development and fixing issues later, arguing that such an approach is unsuitable for building the most powerful technology ever created. Coxon called for a potential slowdown in the development of frontier AI, urging companies to give computer scientists time to develop safeguards to keep the models in check.