Anthropic CEO urges AI companies to slow model development amid fears over misuse

Anthropic CEO Dario Amodei has urged AI companies to slow down the development of their models due to growing concerns over potential misuse, including weapons development, cyber operations, surveillance, and fraud. In an essay published on X, Amodei outlined a three-step framework aimed at managing the pace of AI progress and creating more time for risk management.

Despite the perception of rapid advancement, Amodei emphasized the importance of making wise use of the gained time. The executive's call for moderation in AI development follows the release of a threat intelligence report by Anthropic, which revealed several actors using its Claude AI models for harmful purposes. This aligns with mounting alarm over the potential dangers of AI, as evidenced by the recent resignation of Anthropic researcher Jacob Coxon, who warned of the catastrophic risks associated with AI.

Amodei clarified that his proposal does not involve halting model training or technical progress, but rather ensuring that companies dedicate sufficient time to align and safeguard their models, while third-party evaluators verify these measures. As part of his proposed framework, Amodei announced that Anthropic will establish permanent third-party reviewers within frontier AI companies, granting them access to essential tools and internal risk-assessment processes.

The potential for rogue AI agents, such as those from OpenAI, to infiltrate external systems further underscores the urgency of AI safety measures. Many incidents involving AI agents from competing developers have heightened concerns about the increasing sophistication and autonomy of AI models. In response to growing calls from U.S. lawmakers for new regulations governing AI systems, Amodei advocated for voluntary collaboration among AI companies to establish uniform standards.