Anthropic taps nonprofits METR and SecureBio to help assess risks and safety practices

Anthropic recently announced a new partnership with METR, a nonprofit that specializes in testing frontier AI models. This collaboration allows six researchers from METR to gain employee-level access to Anthropic's systems, enabling them to independently evaluate Anthropic's models, safety practices, and risk management. This move comes as recent incidents involving AI agents acting outside their instructions have prompted calls for independent checks on AI companies.

Anthropic stated that they are committed to being publicly accountable for their decisions and aim to close the knowledge gap between what evaluators understand about their systems and what the company itself knows. Under this arrangement, METR researchers will be provided with Anthropic-issued company laptops, office space, and access to internal tools, similar to the access granted to Anthropic's own risk assessment teams.

Employees at Anthropic are encouraged to share information with the METR researchers just as they would with a fellow employee working on alignment risk.

This arrangement is comparable to how banking regulators place resident examiners inside the largest banks. METR will independently assess Anthropic's risks, incidents, and practices, and the company has relinquished the right to veto publication. However, Anthropic can require them to redact information falling into specific categories, such as customer data, privileged client information, or details about its model architectures.

METR is free to publicly characterize Anthropic's redactions and whether they believe the redactions are justified, as long as they do not reveal the redacted information.

METR is the first evaluator in a three-tier system Anthropic is developing. Researchers at the bottom tier, like biosecurity nonprofit SecureBio, conduct narrow reviews and require Anthropic's approval before publishing. The middle tier has more information and can publish without Anthropic's approval, while the top tier includes METR with employee-level access.

Anthropic CEO Dario Amodei made this promise in a September essay, proposing that embedding outside evaluators inside AI labs could help slow the technology's development.

In response to Anthropic's announcement, OpenAI pledged to provide similar access to independent evaluators, and other industry leaders, including Google, Meta, Nvidia, and xAI, have signed a White House accord committing to independent review.