Anthropic and OpenAI sound the alarm on AI safety – and seek to shape how it’s controlled

As leading tech companies raised concerns that their advanced AI systems could pose existential risks to humanity, a central question arose: What potential benefits do they stand to gain? The CEOs of Anthropic and OpenAI recently declared that their groundbreaking models are so potent they require independent testing and regulation before deployment.

In a rare display of cooperation, these executives outlined alarming hypotheticals in essays, social media posts, and addresses to the United Nations, effectively shaping the discourse on how their technology should be governed. By emphasizing this need for oversight, the companies seem to be courting public approval and establishing the parameters for their own safety measures within an unregulated market, according to experts, analysts, and former government evaluators interviewed by The Associated Press.

While some, like Anthropic engineer Jacob Coxon, have advocated for a temporary halt in tech development to prevent "superhuman" systems from surpassing their creators' control, the companies have leveraged his post to spotlight their safety initiatives and position themselves as prudent industry leaders as they prepare for potential stock market listings.

President Donald Trump has dismissed the need for new AI regulations, dismissing such risks as a "HOAX" devised to benefit China. Contrarily, Anthropic spokesperson Sarah Shoker emphasized that the company has long advocated for regulatory measures, while OpenAI's Liz Bourgeois highlighted their recent pause in training their most sophisticated models.

Bourgeois added that public concern over AI development safety has become increasingly prevalent. Debates have centered on untested dangers, such as AI's potential misuse in military applications, while experts argue that these technologies are already deployed in lethal capacities. Despite these concerns, AI companies are grappling with developing safe methodologies for testing their increasingly sophisticated "world models" with 3D awareness.

OpenAI representatives have suggested discussions with other AI firms, including Anthropic and Google, regarding a temporary pause in development. They also propose the involvement of independent auditors to scrutinize labs' progress, should the government fail to enact regulations.