Amodei’s AI Safety Plan Comes With a Catch

Dario Amodei, CEO and Co-Founder of Anthropic, has proposed a three-pronged plan to address the rapid advancement of artificial intelligence. The first step involves granting independent evaluators access to company tools and resources, akin to bank supervisors monitoring controls in operation rather than reviewing a report after deployment.

The second step calls for companies to implement a system for "pacing" AI development, without halting progress altogether, but rather through checkpoints tied to a model's capabilities and safeguard effectiveness. Lastly, Amodei suggests coordination among companies in democratic nations, with the ultimate goal of establishing an international agreement that includes China's participation.

The plan primarily hinges on the notion of independent oversight as the most crucial first step towards AI safety. However, several challenges and limitations arise when examining the proposal. The definition of "pacing" remains ambiguous, as there is no universally accepted measure of AI advancement, and any limitation on one aspect may simply shift development to another area.

Additionally, coordination among competitors might necessitate government intervention, potentially giving the largest AI labs an advantage due to their financial resources and relationships. Lastly, the prospect of forming a global pact with China is considered weak, as the technology holds both military and economic value, making it difficult to enforce a universal limit on general AI development.