Who Aligns the Aligners?

Dario Amodei, CEO of Anthropic, has written an essay titled "We Must Pace The Frontier," explaining his belief that AI could bring significant benefits to humanity, such as curing major diseases and accelerating economic growth. However, he acknowledges that AI also poses serious risks, including the potential destruction of the human race.

Despite the lack of evidence supporting these risks, Amodei proposes a balanced approach to AI development, aiming to achieve its benefits while ensuring its safety and addressing geopolitical concerns. He suggests that AI development should be paced, ideally through voluntary regulation, which would include the use of embedded evaluators to monitor adherence to safety practices and commitments.

These evaluators would be akin to third-party safety overseers in the Web 2.0 industry, as seen in the Global Alliance for Responsible Media (GARM). However, Amodei also proposes a second approach, "Democratic Coordination," where AI companies within democratic countries work together to establish common safety standards and limit the rate of unchecked AI progress.

This approach would require government support, particularly for more legally challenging aspects of pacing.