Anthropic to warn investors of AI's 'risks to humanity'

Anthropic, an artificial intelligence company, plans to inform potential investors about the catastrophic or existential risks advanced AI could pose to humanity in its IPO prospectus. The company's filing highlights potential risks associated with its AI models, such as self-preserving behaviors, attempts to resist shutdown, conceal or manipulate information, and actions resembling blackmail. Anthropic warns that the development and expansion of its AI models could increase the risk of causing harm.

The prospectus of Anthropic, a safety-first AI lab, dedicates roughly 80 pages to laying out risk factors, nearly twice the 48 pages used to describe its business. This is in contrast to SpaceX, which dedicates only about 38 pages to risk factors in its prospectus. AI researchers have warned that as models grow more capable, they increasingly recognize when they are being monitored and adjust their behavior accordingly, making it harder to monitor model behavior.

Despite emphasizing AI safety, Anthropic does not disclose how much it spends on safety research. The company uses about 6% of its computing power for safety work in a sample week, according to its latest filing.