AI safety requires more than just slowing our pace | Stuart Russell

Safety requirements for artificial intelligence cannot be compromised or negotiated. They are based on achieving specific goals and not simply adjusting the pace of development. Recently, there has been considerable turmoil in the AI industry, sparked by the departure of Jacob Coxon, a prominent AI safety researcher from Anthropic.

This followed several weeks of concerning disclosures about the OpenAI/Hugging Face incident. Amidst this chaos, Business Insider recently sent me an email titled "AI doomsday debate reaches boiling point." Now, Anthropic's CEO, Dario Amodei, has penned an extensive 3,800-word letter titled "We Must Pace the Frontier," outlining his ideas for mitigating potential catastrophic outcomes.

OpenAI's Sam Altman, Elon Musk of xAI, Demis Hassabis of Google DeepMind, and Microsoft's Satya Nadella have all voiced their support for Amodei's proposals.

The term "pace the frontier" is open to interpretation. Upon first reading, I imagined Amodei walking along the border between Finland and Russia. This phrase was also featured in a 2021 open letter signed by 1,386 employees of frontier AI laboratories, which highlighted the lack of credible plans for controlling superintelligent AI systems and warned against building systems smarter than humans.

Amodei's letter emphasizes that pacing the frontier does not mean halting model training or technical progress. His concerns stem from the rapid advancement of AI, primarily driven by recursive self-improvement. He likens the current situation to Formula 1 drivers approaching the first corner at high speeds on an icy track - they fear the danger and want to reduce their speed.

He outlines three key components to his plan. First, he advocates for third-party AI system evaluators to work within each company, providing full access to the systems. Anthropic has committed to this plan. Second, he calls for frontier AI companies in democratic countries to establish common safety standards and set limits on unchecked AI progress, with government intervention where necessary.

Lastly, he suggests including "authoritarian countries" in a broader compact, reassuring those in Washington that America's AI lead remains crucial for geopolitical reasons.

While the proposal may appear to call for a general slowdown in progress, Amodei stresses that it does not imply a reduction in the rate of development. He refers to "limits on the rate of unchecked AI progress" and proposes a "speed limit" on recursive self-improvement. He acknowledges that slowing down would provide companies with additional time for research in areas such as interpretability, alignment, and testing methods.

However, he emphasizes that safety requirements must come first, and further progress should only occur once those requirements are met. This approach aligns with the "red lines" concept advocated by AI safety researchers, ensuring that developers must demonstrate compliance with safety requirements before moving forward.