AI rivals found rare agreement on safety. Putting it into practice is harder

The leading AI industry voices rarely agree on slowing down development, but when they do, it's a rare and noteworthy moment. This consensus emerged after an Anthropic researcher warned about the technology's potential threats to humanity. However, several obstacles stand in the way of a coordinated effort to pace AI's development, including competition, profit motivations, and pushback from President Donald Trump.

Nick Reese, a former DHS director of emerging technology policy, believes this moment could force the industry to consider a "reframing of what we consider successful" in AI, comparing it to airline safety where crashes are intolerable but failures are accepted.

Tech leaders envision slowing AI progress in various ways. Anthropic's Dario Amodei proposes that leading AI companies, or frontier labs, should give "ongoing, employee-like access" to independent evaluators who monitor safety practices. OpenAI's Sam Altman echoed this idea on social media, welcoming a federal framework setting safety standards for advanced AI.

Amodei's plan also suggests government regulation and intervention, including coordination between frontier AI companies with the help of the U.S. and other democratic countries. He hopes the leading labs could establish common safety standards and limits on the rate of unchecked AI progress.

However, implementing these agreements presents significant challenges. Getting cooperation from authoritarian governments like China is difficult, and even "slowing the rate from extremely fast to only somewhat fast" still provides relatively little strategic advantage while potentially greatly improving safety. Altman clarified that AI progress will continue to be rapid, but it should be slower than it otherwise could be, with interventions like safety cases and monitoring adding costs.

Experts question the feasibility of these proposals, but the rare consensus among AI rivals highlights a growing recognition of the need for caution in AI development.