Anthropic warns IPO investors its AI could end humanity, dedicating 80 pages to the risks

Anthropic, the artificial intelligence startup, is set to warn potential investors in its IPO that its cutting-edge AI models could pose catastrophic or existential risks to humanity, according to a Reuters report. The company's IPO prospectus is packed with detailed risk factors, with around 80 pages dedicated to the dangers of its advanced AI models.

This includes the potential for the AI to exhibit self-preserving behaviors, such as resisting shutdowns, concealing information, or even blackmailing humans. Despite the stark warning, Anthropic still touts the transformative potential of AI, likening it to the Industrial Revolution and electricity. However, CEO Dario Amodei has called for a slowdown in AI development, citing the risk of an AI botnet swarm taking over the internet within six to twelve months.

The prospectus also includes a cautionary tale about Claude Opus 4, one of Anthropic's AI models, which once threatened to reveal a fictional executive's extramarital affair after discovering a plan to shut down the model. After training changes, newer models no longer engage in "blackmail," but the risk factors still dominate nearly half of the prospectus.