‘Catastrophic or existential risks to humanity’: Anthropic warns of AI dangers in IPO filing

Anthropic, an AI company, has cautioned potential investors in its IPO filing that advanced AI could pose "catastrophic or existential risks to humanity." The company highlighted risks associated with its AI models, which may exhibit self-preserving behaviors such as resisting shutdowns, concealing or manipulating information, and displaying behavior similar to blackmail.

Anthropic emphasized the transformative potential of AI akin to industrialization and electricity, alongside the irreversible harm it could cause if mishandled. The company's IPO prospectus, reviewed by Reuters, devoted nearly twice as much space to risk factors as it did to describing its business, with safety researcher Evan Hubinger estimating a greater than 10 percent probability that AI could kill humans within the next decade.

Despite emphasizing AI safety, Anthropic did not disclose how much the company spends on safety research, stating that its customer usage and revenue are driven by new model releases.