Humanity Needs an AI Nonproliferation Treaty
Tech companies will never unilaterally disarm. Neither, for that matter, will the countries where they’re based.
In recent years, America's leading AI companies have been rapidly developing increasingly advanced models, causing concern among industry insiders. In July, over 1,300 employees from major AI firms, including executives from OpenAI, Anthropic, Google, and Meta, signed an open letter warning that AI capabilities could soon surpass human understanding and control. The letter called for the U.S. government to develop the necessary tools to pace AI progress deliberately.
Meta's chief scientist, Shengjia Zhao, stated that AI is nearing the creation of "unprecedented social and safety risks." Google engineer Matthew Rahtz compared the current pace of AI development to a "runaway nuclear chain reaction." OpenAI researcher Leo Gao likened the current progress to "a runaway nuclear chain reaction."
Experts warn that the progress could lead to dystopian scenarios such as chatbot-generated bioweapons, major hacking attacks, and rogue AI agents causing societal havoc. There is also growing public backlash against AI data centers. However, the proposed solution of a data-center moratorium is inadequate, as it does not address the core issue of runaway AI development. Industry insiders believe that the industry is on a path towards uncontrollable progress, with potential leaps happening every few weeks or days.
Government policy is deemed necessary to overcome the collective-action problem within the AI industry. However, existing proposals such as state-level moratoriums and comprehensive regulation may not be sufficient. Chinese AI companies are estimated to be only a few months behind American counterparts, ensuring that even if the U.S. slows down its own industry, the dangers of fast AI progress would still persist.
Written by urgent.news from The Atlantic's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
