Dario Amodei's AI Safety Warning Is Colliding With Anthropic's IPO Timeline

Dario Amodei, CEO of Anthropic, has repeatedly cautioned that AI models could advance faster than the safety measures in place to control them. Despite this warning, Anthropic is reportedly planning to hold its initial public offering (IPO) in November. New York City Comptroller Mark Levine has urged the company to postpone its IPO until it resolves AI safety and governance concerns.

In a September essay, Amodei expressed fears that AI's recursive self-improvement capability could result in powerful systems that surpass developers' abilities to manage them. He advocated for independent safety reviews and increased collaboration between AI companies and governments. Anthropic has paused some high-risk training activities involving unreleased AI models for several weeks, mainly in response to reinforcement-learning work.

However, these pauses have resumed after the company introduced additional monitoring. Recent incidents involving misalignment and security issues with Anthropic's AI models have further raised safety concerns. To address these issues, Anthropic is strengthening independent safety reviews, although outside evaluators cannot prevent model releases.

Anthropic's draft IPO structure grants its seven co-founders significant voting power through a new entity called Founder LLC. The company has also proposed legislation to the U.S. government that could allow the Department of Commerce and federal courts to block dangerous AI models, potentially delaying commercialization and impacting the IPO's valuation.

Despite these developments, Anthropic's revenue grew by 45% sequentially to $16.8 billion in the third quarter. However, some investors are questioning whether a lower valuation of around $1.5 trillion would be more appropriate, given slowing revenue growth and rising AI infrastructure costs.