Anthropic grants AI evaluator METR employee-level access to its models

On October 15, Anthropic announced it has granted a team from nonprofit AI evaluator METR employee-level access to its advanced systems. This move comes after concerns were raised among researchers and evaluators about leading AI labs exerting excessive control over independent reviews through publication restrictions and limiting access to sensitive information.

Under the new arrangement, six METR employees have been equipped with company hardware, provided office space, and granted access to internal tools and systems, which are generally equivalent to those used by Anthropic's own risk assessment teams. Anthropic stated that METR will now be able to publish independent evaluations of risks, incidents, and safety practices within their AI lab without the need for prior approval, with only limited redactions applied to areas such as customer data, privileged information, and certain model architecture specifics.

This development aims to address the growing concern within the AI community about potential imbalances of power between major AI labs and independent evaluators, ensuring a more equitable and transparent review process.