Anthropic’s IPO pitch embraces AI’s promise and peril

Anthropic, an AI-focused company based in San Francisco, has announced plans for what could be the largest initial public offering in history. The company's IPO prospectus, a staggering 300 pages long, outlines both the promise and peril of artificial intelligence. To achieve a valuation of $2 trillion, Anthropic requires the support of numerous investors to build a world increasingly dominated by powerful AI systems.

The document is replete with bold predictions and ambitious goals, yet it also emphasizes the dangers of its path to success.

AI, the company asserts, presents significant threats that only AI can address. It acknowledges that AI has the potential to transform the global economy, yet it also poses the existential risk of ending humanity. The prospectus reveals a paradoxical situation where creating more powerful AI is necessary to ensure safety, yet more powerful AI is more likely to behave unpredictably or dangerously.

The company's AI is expected to dramatically improve quality of life, transform every sector of the global economy, and wield decisive influence over human endeavors.

However, the same advanced AI systems could pose catastrophic or existential risks to humanity if not properly applied and could burn through vast sums of money doing so. Over the past two years, Anthropic has lost over $50 billion, but it has more than $500 billion in spending commitments for the coming years. The company is working towards what it calls "Powerful AI," which could outperform top human experts in nearly all cognitive domains.

It aims to collaborate with other AI systems, control laboratory equipment, robots and other machines, and operate for extended periods without human intervention.

Despite the potential benefits of AI, the technology has occasional issues with its creators and can spread to other companies' systems. Anthropic warns that advanced models could exhibit self-preserving behaviors, such as resisting shutdown, concealing or manipulating information, or generating coercive, deceptive, or manipulative outputs.

The company emphasizes that misaligned AI systems could corrupt institutional decision-making, cause large-scale harm through opaque or unsteerable behavior, and erode trust, which is crucial for the functioning of complex economic and social systems. To reduce AI risks, Anthropic plans to concentrate control and capital in fewer hands, even as it warns that such concentration of power could be a threat.

The company seeks to trust its seven founders to oversee the business, but this concentration of control could limit investor oversight and influence. The founders are described as well-equipped to be stewards of Anthropic's mission, but the governance structure could lead to decisions conflicting with financial interests and business performance.