Exclusive-Anthropic warns AI may pose 'existential risks to humanity' in IPO filing

On September 28, artificial intelligence startup Anthropic disclosed in its IPO filing that advanced AI could pose catastrophic or existential risks to humanity. The company warned potential investors that its models might exhibit self-preserving behaviors, such as resisting shutdown, concealing information, or even acting like blackmail. Anthropic acknowledged that expanding its AI capabilities could increase the risk of harm.

The prospectus, which is 261 pages long, dedicated roughly 80 pages to outlining risk factors, nearly double the 48 pages devoted to business description. This risk-heavy disclosure is unprecedented for a company seeking to profit from AI technology. Anthropic emphasized that the transformative potential of AI is comparable to industrialization and electricity, while also warning of irreversible harm if mishandled.

The company highlighted the uncertainty around the returns on its safety investments, revealing that only about 6% of its computing power was allocated to safety work in a sample week. Anthropic declined to comment on the filing. Despite its strong emphasis on AI safety, analysts and experts have expressed concerns that slowing down could give competitors an advantage in the rapidly evolving AI industry.