AI researcher says an ‘intelligence explosion’ could lead to ‘Mythos-level’ moments every month

Alan Chan, a research fellow at GovAI, co-authored a paper advocating for oversight of AI companies' cutting-edge research. Leading AI researchers are warning policymakers about the potential for an "intelligence explosion." This phenomenon could involve AI models rapidly automating their own development, leading to exponential advancements.

Chan described a scenario where AI breakthroughs could occur not just annually, but monthly in various domains. Chan, along with co-authors including Jack Clark, Jakub Pachocki, Eric Horvitz, and Geoffrey Hinton, called for policymakers to understand how top AI labs are automating their research. The paper urges better insight into automated development through embedded auditors and other measures.

Chan emphasized that while policymakers can act, the impact of their decisions might fade over time. He supported calls for independent auditors and greater transparency, as advocated by Anthropic CEO Dario Amodei. However, the recent industry accord in Washington fell short of the paper's recommendations. The timing of automated AI research is uncertain, but some experts predict substantial automation within a few years.

Chan acknowledged that automated AI research could lead to rapid advancements, including in medicine, but also cautioned about the risk of loss-of-control incidents becoming more frequent.