As AI companies get closer to ‘recursive self-improvement,’ this physics professor calls full autonomy ‘the worst idea in the history of humanity’
The prospect of artificial intelligence models teaching themselves autonomously to be more efficient and capable appears ever closer to reality.
Artificial intelligence (AI) models are becoming increasingly proficient at teaching themselves how to improve their performance, a process known as recursive self-improvement (RSI). Leading AI companies have varying definitions of RSI, with some considering it as any feedback from AI on model improvement, while others view it as AI fully autonomous in its self-improvement efforts.
The main concern surrounding RSI is that it could lead to an uncontrollable superintelligence, posing a significant threat to humanity. Researchers, including physics professor Anthony Aguirre from the University of California, Santa Cruz, and Cornell University's John Thickstun, have debated the pace of RSI development and the implications for society.
Some AI labs, such as Anthropic, OpenAI, and Microsoft, are moving closer to achieving fully autonomous model improvement, while others advocate for a slower pace to ensure human control and safety measures keep up with the models' capabilities.
Written by urgent.news from Fortune's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
