Anthropic partners with Accenture to embed evaluators within Anthropic, including red teaming models and conducting alignment assessments (Anthropic)

Anthropic has entered into a partnership with Accenture to conduct independent evaluations of its frontier AI models. This collaboration, spearheaded by Accenture's Faculty business unit, aims to embed evaluators within Anthropic, including red teaming models and conducting alignment assessments. The partnership is expected to invest at least $1 billion over the next five years, with both companies recognizing the importance of this work in ensuring the safety and accountability of AI models.

Unlike external evaluators, embedded evaluators will have direct access to Anthropic's models, allowing them to observe model development, verify safety commitments, and identify potential blind spots. This level of access will enhance transparency and enable the public to better understand the benefits and risks associated with Anthropic's AI technologies.

While there are currently no established standards for what information embedded evaluators can access or how they should report their findings, Anthropic plans to collaborate with various evaluators, including METR and other nonprofit organizations, to pilot this approach. The partnership is non-exclusive, allowing Anthropic to work with multiple evaluators as the field of frontier AI continues to evolve.