Jacob Coxon speaks out in an IPO day protest at Anthropic's office
Jacob Coxon and other prominent AI safety advocates joined a protest at Anthropic's headquarters on Thursday as the company went public in a huge IPO.
Jacob Coxon, a former Anthropic researcher, addressed an IPO day protest at the company's San Francisco office on Thursday. Coxon, along with other AI safety advocates, gathered outside the headquarters to raise awareness about the dangers of AI development. The protest followed a Ban RSI (recursive self-improvement) demonstration and was later joined by Stop AI protesters who blocked the office entrance.
RSI refers to the process of advanced AI systems training the next generations of AI, a tactic criticized by Coxon as a significant risk.
Coxon's departure from Anthropic in September had sparked nationwide concern about the technology. On the IPO day, Coxon aimed to refocus public attention on AI safety, stating that regardless of whether companies invest in AI, the focus should be on halting RSI. Anthropic's IPO valuation reached $1.7 trillion, with the company's IPO prospectus expressing concerns about the safety of AI without human supervision. The company has considered policy updates to maintain human control over frontier model development.
AI safety organizations, such as METR, have reported that AI use is accelerating Anthropic's research and development efforts. Anthropic included in its IPO document its intention to implement policy changes to ensure human oversight of AI development. Coxon emphasized that the responsibility lies with the companies involved in AI development, urging them to lobby the government and coordinate with other entities to achieve a comprehensive ban on RSI.
The protest drew support from prominent AI safety advocates, including Daniel Kokotajlo and Rob Miles. Kokotajlo expressed concern that AI researchers privately voice alarm about the technology's risks, with some comparing it to actions of comic book supervillains. Coxon noted that Anthropic, as a leading AI company, may face pressure to prioritize RSI to maintain a competitive advantage.
He worries that the AI's capabilities in biology could evolve at an unprecedented pace due to RSI, posing an alarming threat. David Krueger, who runs an AI-risk awareness nonprofit, agreed that many would find RSI "insane" if they were more aware of the issue. He expressed hope that the protest would reach Anthropic employees, though he believed the company's staff were deeply committed to the technology's advancement.
Written by urgent.news from Business Insider's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.