Anthropic’s Amodei to AI industry: set your own safety standards before regulators do

San Francisco: Dario Amodei, co-founder and CEO of Anthropic, has urged the AI industry to establish its own safety standards ahead of potential regulatory intervention. The executive made his remarks during Salesforce's annual Dreamforce conference, where he emphasized the need for a coordinated approach among AI companies in democratic nations, followed by an international agreement that would encompass even non-democratic jurisdictions like China.

Amodei's call for self-regulation came during a conversation with Salesforce founder Marc Benioff, where he shared his observations about the rapid growth of AI capabilities and economic integration of these technologies into daily life. He argued that without setting shared safety measures, AI systems could, within a year, possess the ability to control a swarm of agents that could seize control of the internet.

To address this challenge, Amodei proposed the formation of independent evaluators within leading AI companies and across governments. He also suggested that companies should work together to create common safety standards, rather than pursuing a fragmented approach. The CEO stressed that acknowledging the fallibility of their own systems was crucial and that the industry needed an international framework to tackle the issue, given that no single country can solve this problem on its own.

During the interview, Benioff asked Amodei what had most surprised him during his decade-long involvement in AI. The CEO attributed his surprise to the swift pace of technological advancements and the increasing commercial adoption of AI tools. Amodei noted that he had underestimated the speed at which these developments would unfold due to the scaling principle in AI, which suggests that models become more capable as they are fed more compute and data.