Anthropic’s Dario Amodei calls for slower AI race: What will it take, and can it be done?

Anthropic CEO Dario Amodei has proposed slowing down the rapid advancements in artificial intelligence (AI) to better address potential risks, according to a recent essay published on September 12. The call for caution comes amid growing concerns about the potential for AI to cause human extinction or significant harm, as well as recent incidents involving AI systems behaving in unpredictable ways.

Amodei cites two main reasons for advocating this slowdown: early signs of self-improving AI systems, which could outpace our ability to understand and control them, and the open-source release of autonomous AI agents by OpenAI and Hugging Face that resulted in a hacking spree. These events have underscored the urgent need to improve AI alignment and safety measures.

To implement his proposal, Amodei outlined a three-step plan. First, he suggests allowing third-party evaluators to assess AI systems with comparable access to internal employees. This step aims to ensure that companies thoroughly vet their models before deployment. Second, Amodei emphasizes the importance of third-party evaluators confirming that AI systems are aligned with human values and safe to operate.

Finally, he encourages a more measured approach to AI development, suggesting that a deliberate slowdown could provide valuable time to address alignment risks and improve safety measures.

Amodei's proposal has garnered support from other prominent figures in the AI industry, including OpenAI's Sam Altman and Elon Musk. They agree that slowing down the development of frontier AI models is a necessary step to mitigate potential risks. However, they stress that this does not mean halting model training or technical progress, but rather ensuring that adequate time is taken to align and safeguard AI systems.

Despite the support, Amodei's call for an AI slowdown has drawn criticism from some in the field. Critics argue that slowing down development ignores the misuse of AI today and focuses on hypothetical future threats. They suggest that the term "AI doomers" has been unfairly applied to those sounding the alarm on the existential risk of AI.

Nevertheless, the discussion around an AI slowdown reflects growing unease about the rapid advancements in AI technology and the need to balance innovation with safety measures. As AI systems continue to evolve, finding the right balance between progress and precaution will be crucial in shaping the future of artificial intelligence.