Former Anthropic researcher behind ‘AI could kill us’ warning called to UK parliament

Former Anthropic employee Jacob Coxon has been asked to testify before the UK parliament concerning his dire predictions about artificial intelligence. Liam Byrne, who leads the Business, Innovation, Science and Trade Committee, disclosed to City AM that MPs have invited Coxon to provide insight into AI risks. Byrne elaborated, "We've extended an invitation for him to speak to us in parliament.

We'll be listening to his comments." Coxon left his position at Anthropic last week, asserting that major AI corporations were "racing towards self-improving superintelligence and wagering with our lives." He subsequently told the BBC that colleagues within AI firms were "truly terrified" and suggested there was a "high probability that we could all perish in the near future" if progress continued unabated.

Although other luminaries in the tech sector have contested his assertions, they have sparked increased debate in Westminster over the UK's largely voluntary stance on AI security. Byrne separately requested the government's AI Security Institute (AISI) to give testimony to MPs on October 13th. In a letter sent to AISI director Henry de Zoete on Tuesday, Byrne noted that growing public concern over "unfettered, inadequately regulated artificial intelligence development" had intensified, citing Coxon's resignation along with warnings from Anthropic CEO Dario Amodei and OpenAI president Sam Altman.

When quizzed about what authorities could do if an AI firm neglected safety warnings and rolled out a hazardous model, Byrne replied, "We don't believe the UK possesses safeguards to prevent such peril." "I think voluntary self-regulation will likely prove insufficient in the long run." Byrne contended that domestic regulation alone would be inadequate in addressing AI risks, emphasizing the need for collaboration with the US and China to establish a regime for "strategic stability and safety."

"My discussions with Chinese policymakers in the spring indicate that they would be quite interested in this, as they recognize the risks," he added. If Washington and Beijing fail to reach an agreement, Byrne posited that the UK should partner with Canada, Australia, Japan, South Korea, and the EU. "Collectively, we could leverage the European consumer market to exert pressure on major AI firms," he stated.

Byrne also cautioned that open-weight models could heighten cyber security threats if exploited by criminal syndicates and recommended that Britain might eventually require a government-backed safety net for the cyber insurance sector. However, he acknowledged reservations from AI firms that policymakers might be moving too swiftly.

"Many in the industry believe that the EU AI Act is excessively restrictive," he noted. Synthesia, a British AI firm, informed City AM that policymakers risk overreacting to "theoretical risks" and a "Silicon Valley AI doomer bubble," while Cohere has advocated for regulation grounded in demonstrated risks rather than rules devised by a select group of leading AI developers.

Byrne cautioned that poorly designed legislation could pose a risk, but argued that confidence in the technology is essential for its broader acceptance. "To foster the growth of the AI market in the medium to long term, you must maximize trust," he remarked. "And you cannot maximize trust if nobody believes it's safe." The government has left the possibility of stricter rules open, informing City AM this week that it "should not presume that the current framework will invariably remain adequate as AI capabilities advance."

Louise Haigh also expressed alarm over "extensive risks to our national security and society if the appropriate guardrails are not implemented." Byrne asserted that the October hearings would provide "whistleblowers, industry representatives, and regulators" an opportunity to voice their apprehensions about the existing system.