AI Safety Requires Eyes Inside the Lab | Opinion

AI safety demands that experts have unfiltered access to the inner workings of cutting-edge artificial intelligence systems, according to Anthropic CEO Dario Amodei. In order to govern this rapidly evolving technology, public authorities must first have a direct line of sight into the development processes of private companies. This allows for continuous evaluation and intervention when necessary.

While mature safety-critical industries like automotive have robust systems of certification, inspection, and enforcement, frontier AI currently lacks any comparable oversight. This raises a significant paradox: one of the most powerful technologies ever designed has among the least public oversight of any comparable industry. The issue is further complicated by the fact that even with unchanged weights, AI models can change their behavior in unpredictable ways when prompted, using tools, accessing memory, or interacting with other agents.

This makes it difficult for governments to establish binding rules for companies developing and deploying AI models without losing sight of the fact that companies ultimately retain operational and legal responsibility. Public oversight is essential to ensure that catastrophic risks are not taken lightly by companies alone, as they cannot independently verify their own restraint or determine what risks society should accept.

To address this, governments should utilize Anthropic's commitment to give independent evaluators ongoing access to their laboratories as a pilot project, bridging the gap between industry and public knowledge regarding frontier AI models. By embedding supervision within these labs, technical knowledge can be brought into public decision-making, allowing for timely and effective intervention when necessary.

However, supervision must still preserve corporate responsibility, with supervisors needing public accreditation, protection from retaliation, access to relevant data, and defined escalation rights. While technical standards can define evaluation methods and reporting requirements, they cannot guarantee that successor models will not exhibit misaligned behavior or overcome incentives to accelerate.

The EU AI Act and California's recent executive order provide a foundation for oversight, but neither establishes permanent independent supervision inside every frontier laboratory. A serious incident in a democracy could lead to severe social and political repercussions, potentially triggering a domestic pause in frontier model development.