A New Report Sheds Light on the Secretive Race to Build Self-Improving AI
As AI companies warn of the dangers of self-improving AI, a new report offers a rare look at how they’re working to make it a reality.
In recent months, two major AI companies, Anthropic and OpenAI, have alerted the world to the potential of AI to improve itself, a concept known as "recursive self-improvement" or RSI. This development could lead to AI capabilities advancing at an unprecedented pace, but it also poses significant risks to humanity, potentially leading to a point where developers lose control over AI systems.
Despite these concerns, AI companies have been quite secretive about their efforts to pursue RSI, sharing little about their progress with the public. Gladstone AI, a research firm, has published a detailed report that sheds light on how RSI might work and how AI companies are progressing toward this goal. The report is based on conversations with several researchers from top AI companies.
According to Gladstone's executives, the aim of the report is to provide lawmakers with a better understanding of RSI and enable them to regulate it more effectively. The researchers believe that RSI will begin within the next 3 to 12 months, potentially sooner if there is no intervention from the U.S. government. Among these researchers, there is a growing concern that "there's a fair amount of disaster prepping and bio-bunker purchasing going on."
The report provides a simplified outline of the AI research pipeline and how AI companies are beginning to automate this process. During the initial phase of training an AI model, known as pretraining, companies focus on finding ways to maximize the intelligence of a model with less computing power. Companies employ a portfolio approach to test these ideas, allocating a small amount of compute to each idea and training a series of models to see which perform better.
As the AI companies scale up, they select the most promising models for further training using larger amounts of compute. However, companies are limited by their compute resources, which restrict the number of promising ideas they can test at larger scales. This limitation makes it crucial for AI companies to have reliable methods for filtering good AI-generated ideas from bad ones, as spending excessive time or compute on filtering could render it more efficient to rely on human researchers with sound intuition.
Written by urgent.news from Time's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
