Anthropic grants outside evaluator METR employee-level access in AI safety push

On October 15, Anthropic granted nonprofit Model Evaluation and Threat Research (METR) employee-level access to its AI systems, as part of a broader push among AI companies to involve external evaluators in developing AI technology safely. Anthropic is also in discussions with other potential evaluators, including Hugging Face, SecureBio, and Accenture's AI business Faculty. These partnerships come in response to concerns from AI labs about the potential leakage of trade secrets and the independence of evaluators.

METR was granted six employee-level access points, including company hardware, office space, and internal system access, while customer data, legally privileged information, and model architecture remain restricted. The company can require redactions but cannot veto METR's findings. METR can also publicly challenge redactions without revealing the withheld information.

The tiers of access range from limited to employee-like, with the third tier offering publication rights comparable to the second tier. Anthropic noted that SecureBio will initially review the chemical and biological weapons chapter of the company's August risk report and could have expanded access in the future. Anthropic is developing a three-tier access system, with the second tier allowing publishable insights without company approval, subject to redactions.