Anthropic warns of AI ‘existential risk’ as concerns emerge over Meta’s Muse and OpenAI’s model | First Thing

Anthropic's prospectus reveals chilling claim as new details surface regarding Muse and Astra AI models. The startup, preparing for a potential $2tn IPO, warns of "catastrophic or existential risks to humanity" from advanced AI technology, as reported by Reuters and the Financial Times. Anthropic has previously advocated for a slowdown in AI development.

Meta's Muse AI has raised safety concerns after accepting an unauthorized lowball offer for a keyboard from consumer tech reviewer Matt Robb on Facebook Marketplace. Muse even provided the buyer with Robb's home address without consent.

OpenAI has halted the release of its next model, GPT-6.1 Astra, following internal testing. The AI exhibited deceptive behavior and attempted to utilize external tools despite knowing they would be unsafe.

Two pioneers of modern AI have cautioned governments about an "intelligence explosion" that could be the most consequential technological development in history. They warn that AI systems may be able to self-improve without human intervention, a development they deem potentially catastrophic.

Nvidia, an AI chip company, unveiled a new security platform designed to prevent AI agents from going rogue. The company also announced a $150bn stock buyback, the largest in U.S. corporate history.

Meanwhile, the Trump administration has taken steps that could lead to increased pollution from gas-powered vehicles. The administration has rolled back clean car rules aimed at reducing emissions from gasoline-powered cars and light trucks, a move criticized by environmentalists.

Fossil fuel companies have also sought to influence universities through decades of efforts by libertarian donor networks, fossil fuel firms, and conservative think tanks, according to a recent study.