Anthropic boss calls for slowing pace of AI development

Dario Amodei, the CEO of AI company Anthropic, has called for a slower pace in the development of advanced AI technology. This call for moderation came to light as concerns over the potential dangers of "superintelligent" computer systems have grown. Just days prior, Jacob Coxon, an AI researcher who transitioned from OpenAI to join Anthropic, announced his departure from the industry.

In his resignation letter, Coxon accused both US-based companies of recklessly gambling with humanity's future in the pursuit of self-improving AI models. Amodei, in a post on his personal website, emphasized the need to temper the pace of AI advancements. He stressed that while AI progress may seem rapid, it is crucial to use the acquired time wisely.

According to Amodei, building the technology comes with significant risks due to its immense potential. While not developing AI might deprive humanity of benefits or allow authoritarian entities to harness its power, accelerating its development is equally reckless. Amodei, co-founder of Anthropic alongside his sister Daniela in 2021, admitted that over recent months, he has grown even more convinced that addressing the risks associated with AI demands a more cautious approach.

Anthropic had previously urged developers to either slow or pause AI development in early June. In late July, Sam Altman, CEO of OpenAI's rival, suggested that developers might need to voluntarily slow down their rapid advancements to give society a chance to keep up. Around the same time, over 1,000 employees from leading AI firms, including Anthropic, signed a petition requesting the US government to actively pace the frontier of AI development.

Earlier in the week, Coxon expressed his concerns after three years of pretraining AI models for OpenAI and Anthropic. Pretraining is the phase where AI models are exposed to extensive data. Coxon criticized both companies for their reckless pursuit of self-improving superintelligence, warning that this could potentially lead to the end of human existence by the end of the decade. Superintelligence refers to the hypothetical stage where AI surpasses human intelligence.