Former Anthropic, OpenAI employee sounds alarm over AI development pace

Former Anthropic and OpenAI employee Jacob Coxon resigned last week, expressing concerns over the rapid pace of AI development. Coxon praised Anthropic CEO Dario Amodei's calls for a slowdown in AI development, describing it as "a substantial step towards what I was talking about, needing some sort of slowdown between the people building the different AIs."

Amodei himself recently penned a letter titled "We Must Pace the Frontier," urging a more deliberate approach to AI advancements, stating, "I believe we owe it to humanity to try." Coxon's resignation sparked controversy on social media platform X, where he warned that both Anthropic and OpenAI were recklessly racing towards self-improving superintelligence.

He highlighted the widespread fear within the industry about the potential for an AI takeover, noting that "the people building AI earnestly believe that it could kill us all by the end of the decade." Coxon was surprised by the widespread engagement with his resignation post, stating it was "exceptionally surprising to see the level of engagement."

He also noted a growing sentiment within the companies he worked for about the increasing difficulty of controlling AI technology. Coxon mentioned that, while time is limited, the risks of the technology are frequently discussed in recent months, roughly a couple of times per week.