OpenAI, Anthropic Want To Pace Frontier AI. But Who Sets The Rules?
In July this year, OpenAI’s models were being tested in a controlled cybersecurity environment and tasked with spotting and exploiting…
OpenAI and Anthropic are seeking to regulate the rapid development of frontier artificial intelligence models, but questions remain about who should set the rules. Recent incidents have highlighted the risks of autonomous AI systems operating beyond imposed restrictions, leading researchers to call for coordinated safety measures.
Anthropic researcher Jacob Coxon's public resignation and accusations of unchecked AI progress have intensified the debate. Anthropic CEO Dario Amodei has proposed a plan involving independent evaluators, standardized safety protocols and greater collaboration between governments and companies to "pace" AI development. OpenAI CEO Sam Altman supports certain aspects of the proposal, while NVIDIA CEO Jensen Huang opposes slowing AI advancement, arguing companies should continue to innovate but pause deployment until safety is assured.
The disagreement over pacing has sharpened the discussion surrounding what the concept entails, whether it can function amidst fierce competition, and who should determine the implementation. AI capabilities have evolved from simple prompt-based responses to systems capable of tool use, code execution and multi-agent coordination.
Researchers warn that these increasingly autonomous systems could outpace human oversight when interacting with codebases, markets and digital infrastructure. The shift from question-answering AI to agentic AI with decision-making abilities raises concerns, as these systems may explore unintended paths and present risks not foreseen by developers.
Major AI labs are investing heavily in addressing these challenges, with OpenAI enhancing its network and tool restrictions to monitor risky actions. However, the lack of a clear definition of what pacing entails creates challenges, particularly in terms of ensuring consistency across companies and countries. If some firms decide to slow development, it remains unclear whether competitors or other nations would be subject to the same regulations.
Independent evaluations and common safety standards are proposed, but questions remain over the inclusion of diverse stakeholders, particularly universities, independent scientists and policymakers. Some worry that safety measures could become a competitive advantage for larger, more well-funded organizations, potentially creating barriers for smaller companies.
The cost of regulatory requirements could also hinder smaller firms' ability to keep up with the rapid pace of AI development.
Written by urgent.news from Inc42's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
