AI Will Kill Us All, Unless Our Country Builds It First

Former Anthropic researcher Jacob Coxon resigned, accusing OpenAI and Anthropic of racing toward self-improving superintelligence while recklessly gambling with humanity's survival. Many developers believe AI could cause human extinction by the end of the decade, with Anthropic's Alignment Science Lead Evan Hubinger estimating a probability greater than 10%. This isn't the invention of an apocalyptic headline by journalists, but explicit claims made by people directly involved in the technology.

In response, Anthropic CEO Dario Amodei called for slower global AI development, arguing that a Chinese lead would be a grave danger to the United States and the world. He maintained that continued American restrictions on advanced chips supplied to China were necessary. However, Donald Trump dismissed the extinction warnings as a hoax, believing the United States should lead China in AI development for strategic advantage.

China, in turn, accused American technology leaders of fearmongering and claimed safety was used as an excuse to contain Chinese development.

The AI debate has become a competition over defining what "AI safety" means, rather than a serious discussion about potential dangers. The popular extinction story begins with an AI surpassing human intelligence, escaping control, taking over human infrastructure, and eliminating humanity. Yet, what happens next remains unclear - does the AI create a robot civilization, improve Earth, or transform the world according to its own plans?

The concept of "better" is subjective and dependent on the values and experiences of beings. AI can discuss these concepts convincingly, but this doesn't mean it inherently possesses human emotions or motives.

Engineering and political decisions, rather than inherent properties of AI, determine whether it causes catastrophic damage. A system with broad permissions, opaque objectives, no independent verification, and no reliable shutdown path is more likely to become dangerous than one lacking these characteristics. Uranium serves as a relevant comparison - the physical knowledge can be used for both beneficial and destructive purposes, and it is humans who ultimately decide its application.

AI, too, has a dual-use character, capable of aiding medicine, knowledge dissemination, and eliminating repetitive work, but also potentially scaling surveillance, operating weapons, automating cyberattacks, and manipulating populations.