AI labs shouldn’t be allowed to grade their own homework

Recent incidents at AI labs such as OpenAI, Anthropic, and Meta have exposed a critical flaw in the oversight of frontier AI systems. These labs, racing to create the most powerful AI models, are responsible for evaluating their safety, determining which failures matter, and deciding what information the public should know about them.

While OpenAI and Anthropic have disclosed these incidents, the fact remains that there are no independent institutions to confirm what is being disclosed or require these disclosures. This situation places an extraordinary amount of trust in companies that are racing to develop AI with potentially disastrous consequences if they fail.

The need for independent verification is further emphasized by the lack of shared standards or consistent evaluation methods. Just as other high-stakes industries rely on independent institutions for oversight, AI should also be held to the same standard. The FRONTIER Act, a bipartisan proposal in Congress, aims to address this gap by establishing licensed Independent Verification Organizations (IVOs) to evaluate AI safety frameworks.

These IVOs would act as external technical experts, assessing whether companies' safety measures are adequate. This approach could eventually create a market for independent verification, attracting experts from various fields and leading to the development of insurance markets around AI risk. By separating the evaluation of AI safety from the companies themselves, the FRONTIER Act would provide a more reliable and credible system for assessing potentially dangerous frontier AI technologies.