Anthropic’s first embedded evaluator is … Accenture?

Anthropic, the AI company led by Dario Amodei, has announced that Accenture, a technology consulting firm, will be working alongside the company to evaluate and improve its AI models. This collaboration is part of Anthropic's broader strategy to ensure the safety and alignment of its artificial intelligence systems.

As part of this partnership, Accenture will be involved in tasks such as evaluating and red-teaming models, conducting alignment assessments, and testing model safeguards. This move comes after Anthropic's CEO acquired a company called Faculty, which will also play a role in these evaluations.

The investment in this project is expected to be at least $1 billion over the next five years. This choice of Accenture as a partner has surprised many in the AI community, leading to a significant increase in the company's stock price.

While some in the AI safety research community have been focusing on organizations like METR, Redwood Research, and Apollo Research, Anthropic sees the involvement of a well-established consulting firm like Accenture as a unique advantage. The company points to Accenture's experience in deploying AI for large corporations and government agencies as a key advantage.

Anthropic acknowledges that this is a new field with no established standards, and the approach is expected to evolve over time. The company also notes that external evaluations are already a crucial part of the release process for new large language models. However, recent incidents have highlighted the need for more responsible approaches to building AI.

Some critics argue that Anthropic's plan to self-police the AI industry through this collaboration could be seen as an attempt to evade accountability for the misbehavior of AI models. However, Anthropic maintains that these evaluators will not reduce their accountability, but rather help make it more verifiable. The safety of their models, they emphasize, remains their responsibility.