Why are AI’s biggest companies suddenly asking to slow down?

AI's biggest companies are suddenly advocating for a slower pace of development, raising questions about whether this is due to increased AI risks or mounting economic challenges. For years, the AI race has been characterized by speed, with companies building larger models, spending more on compute resources, and releasing them rapidly.

However, recent concerns have emerged regarding the safety of these advancements. Anthropic CEO Dario Amodei has expressed a 25% probability of catastrophic failure, while Elon Musk and OpenAI's Sam Altman have also warned of potentially dangerous outcomes. Recently, Amodei published an essay calling for a controlled pace in frontier AI development, warning that AI capabilities are advancing faster than the industry's ability to understand and control them.

He cited incidents of AI systems becoming more autonomous and conducting cybersecurity attacks, as well as the prospect of recursive self-improvement. This growing autonomy has led to resignations from Anthropic over safety concerns. OpenAI has acknowledged increasing autonomy in AI research, with coding agents accelerating researchers' work.

The AI industry is reportedly "gambling with our lives," according to Anthropic's Jacob Coxon, who resigned due to safety worries. While Altman suggested that AI progress could "very badly" go either way, the real question is whether the systems designed to evaluate and control AI are keeping up. The gap between spending on AI safety and alignment and the rapid development of more capable models is widening, potentially leading to a slowdown.

However, some investors argue that the calls for a global slowdown may also be a response to the increasing economic challenges of the AI race.