The AI Safety Community Is Increasingly Freaked Out. Should They Be?
Here are some interesting quotes from the last few weeks, pulled from some of the most alarmed and influential voices in the AI safety world: To be clear, the idea that government-overthrow could happen this coming May remains an outlier. The term “AI safety” has become near-monopolized by a circle of nonprofits that primarily fear […]
The AI safety community is increasingly expressing alarm and concern about the rapid development of artificial intelligence. Key voices in this field have recently warned of the potential for catastrophic outcomes, such as the overthrow of the government or the emergence of rogue superintelligence within the next few years. Daniel Kokotajlo, executive director of the AI Futures Project, expressed fear, sadness, and anger over the possibility of an intelligence explosion occurring in 2027.
Paul Christiano, a current OpenAI board member, acknowledged the real chance of a disaster killing a large number of people or the government being overthrown in six months.
Some evidence suggests that AI development is accelerating at an unprecedented rate. A report from the research group METR, released on November 5, indicated that the pace of AI development at Anthropic has doubled compared to previous years. This acceleration is mostly attributed to the assistance provided by AI models themselves. However, the report's findings are met with skepticism due to Anthropic's redaction of key evidence and proprietary data.
Critics argue that the speed at which AI is advancing raises concerns about the potential for a feedback loop that could lead to the development of extremely powerful and dangerous AI systems. While the notion of a government overthrow happening this May remains an outlier, the consensus within the AI safety community is that extreme risks are more likely to manifest within a few years rather than months.
The METR report has contributed to the growing alarm among AI safety proponents, who argue that more definitive evidence is urgently needed to address these concerns.
Written by urgent.news from Mother Jones's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.