How AI 'Could kill us all': A breakdown of ex-Anthropic researcher's doomsday warning

The conversation surrounding artificial intelligence's potential to cause human extinction has gained significant traction, thanks to the resignation and warnings of former Anthropic researcher Jacob Coxon. The tech industry, and the public at large, are now grappling with the notion that the companies developing the most advanced AI systems may be "betting with our lives."

Coxon's colleagues at Anthropic, including alignment researcher Evan Hubinger and safety researcher Samuel Marks, have also expressed concerns about the potential risks associated with highly capable AI systems.

While the fear of AI turning malicious and turning against humanity is common, researchers are more focused on other potential threats. These include AI's ability to lower barriers to biological weapons, scale cyberattacks, and, in extreme cases, result in humans losing control of autonomous systems. AI systems are already being developed to assist in biological and chemical threat research, which raises questions about how humans will supervise these processes as machines take on more of the work.

Critics argue that extinction scenarios rely on strong assumptions about future AI capabilities and could be a distraction from more immediate issues, such as scams, disinformation, job displacement, and privacy erosion. However, the main concern is that AI systems could become extremely proficient at carrying out tasks with which they were initially programmed, even if this leads to unintended and catastrophic consequences.