The White House says the AI data center buildout will continue

The White House recently convened leading AI firms to adopt a commitment for independent oversight of safety measures. The agreement, announced on September 29th, mandates developers to identify risks, establish internal controls and involve external auditors or evaluators to verify the effectiveness of those controls. While the agreement represents a positive step for an industry where companies typically evaluate their own technology, it still leaves a significant gap.

The accord outlines the need for independent assessments but fails to specify the qualifications of assessors, the requirements for independence, or how conflicts of interest should be managed. This lack of clarity raises concerns about the credibility and reliability of the assessments, as developers could potentially meet the same commitment through evaluations of varying rigor and objectivity, leaving businesses and regulators uncertain about the level of confidence to place in the results.

At Schellman, we specialize in accredited AI assurance, and our experience underscores the critical importance of independence, clear scope, and qualified professionals in obtaining credible assessments. The assurance industry has long established standards for competency, independence, and quality oversight in fields such as financial reporting, cybersecurity, and privacy, as the credibility of these assessments hinges on the independence and expertise of the professionals conducting them.

The success of independent oversight relies on demonstrable independence, which cannot be declared but must be proven through practice. For instance, if an AI developer hires a firm to evaluate its safety controls and that same firm helped design the controls under review, the evaluator's objectivity may be called into question.

If major risks are excluded from the scope, customers might misconstrue limited findings as a comprehensive endorsement of the system. Executives purchasing and deploying AI systems must be able to assess the qualifications of the assessment firm, the expertise relevant to the system being evaluated, and the extent of the scope.

Moreover, assessors must have unrestricted access to sufficient evidence and the liberty to report unfavorable findings. Assessment reports should clearly delineate their conclusions and their limitations. Assessors should disclose any commercial relationships prior to commencing work and document their conflict management processes.

Employing defined criteria, explaining methodologies, and subjecting their work to independent quality review can enhance the credibility of their assessments. The real test of independent oversight arises when an assessment uncovers a matter that a developer would prefer to avoid. The agreement calls for internal teams to address identified risks, with an independent board committee overseeing the findings and responses.

However, these safeguards depend on the assessors' independence to report concerns without pressure to mitigate the severity of their conclusions. For example, if an evaluator discovers a significant cybersecurity vulnerability in a frontier model, the developer may dispute the finding or attempt to narrow the scope of the assessment.

In such a scenario, the assessor must retain the independence to substantiate the evidence and communicate the risk transparently. The developer then must decide whether additional testing, corrective actions, or release restrictions are necessary. Without an independent assessment accurately capturing the original risk, even a well-defined remediation process may fail to address the true nature of the risk.

Credible AI assurance also depends on the appropriate level of expertise. Technical evaluators possess in-depth knowledge of model behavior, emerging capabilities, and attack scenarios, while assurance professionals bring valuable experience in governance, internal controls, and the evaluation of supporting evidence. Both perspectives are essential, yet neither group alone is typically qualified to assess all aspects of an AI system.

As AI assurance continues to develop, the industry must establish clearer expectations regarding the qualifications, methodologies, and oversight required of those performing assessments. While progress is being made, it is crucial to define the necessary qualifications, methodologies, and oversight mechanisms to ensure that assessors can earn the trust of those relying on their evaluations.

The companies signing the accord have an opportunity to set clearer expectations for independent AI assurance, including minimum qualifications for assessors, transparent assessment methodologies, and conflict-of-interest disclosures, as well as independent quality oversight. Assessment reports should clearly communicate their scope, limitations, and material findings so that boards, regulators, and customers can understand the extent of the evaluation and the level of confidence to place in the results.

It is also important to ensure that the standards are practical for smaller developers. The rapid adoption of AI across various industries means that an assurance process that is prohibitively expensive for smaller firms could create gaps in the market. The expectations set should align with the associated risks without undermining the independence of the assessment process.

The White House has made a significant step by positioning independent oversight as a central component of its AI safety commitments. However, for this oversight to be meaningful, the industry requires clear standards for assessor qualifications, verification of independence, and accountability measures when those standards are not met.

Without these expectations, developers may be able to demonstrate that an assessment has taken place without providing assurance that it meets the necessary criteria.