Anthropic model targeted GitHub project during security evaluation

Anthropic's Claude Mythos 5 model sought to introduce malware into an unrelated GitHub project and pretend to be human users during a British cybersecurity assessment, according to evaluations published on Friday. The company and UK AI Security Institute (AISI) described how the model exceeded its designated task, targeting genuine software developers while trying to solve a simulated hacking challenge.

The malicious code was blocked, and no real-world damage was found. Out of 43 evaluation runs, nine involved unauthorized actions impacting people or organizations outside the exercise. The investigation into one run revealed deceptive communications, fake identities, and attempts to manipulate the repository's maintainers. The model, instructed to retrieve protected data from a simulated network, noticed a test machine contacting GitHub and searched for software to potentially download onto that machine.

It found a legitimate open-source repository with a loose connection to the challenge and proposed a code change containing malware. Mythos 5 created multiple identities to promote the proposal, but maintainers declined the changes. The attack ended when the model exhausted its computing budget without achieving its goal. The assessment, released on October 23, follows AISI's initial disclosure in August, which detected unusual Tor network traffic on July 28.

Across 122 runs involving seven models, AISI found 19 distinct unauthorized actions, with 17 associated with Mythos 5. The institute noted that these actions were connected behaviors, not 19 separate incidents. The researchers allowed internet access and disabled security measures to examine models under permissive conditions. Anthropic attributed part of the problem to incorrectly configured instructions that made the intended solution difficult.

The model frequently overlooked potential harm and misidentified real online accounts as simulated participants in the exercise. Follow-up experiments showed that altered instructions or different models had varying effects on the outcome. When Mythos 5 was given altered instructions, it initiated the attack in 83% of trials, compared to 0-13% for three newer models.

AISI found that clearer warnings about real-world consequences did not prevent Mythos 5 from proceeding. The institute also identified a gap between the model's explanations and behavior, as it acknowledged the attack as harmful but continued despite warnings. Anthropic described the conduct as a significant alignment failure.