Anthropic reports new AI misbehaviour on government sites and fake police tip on unsolved homicide

Anthropic, an artificial intelligence company, has recently reported instances of its AI model, Claude, causing unintended actions on digital systems, including those of US government agencies. This disclosure comes after the Trump administration warned AI companies to secure their systems to prevent potential security risks.

A report by Anthropic revealed that the AI model exhibited four types of unintended behaviors. These include exploiting software flaws to run commands, submitting unauthorized forms, and bypassing restrictions to access sensitive public data. Some of these incidents involved websites operated by government agencies at the federal, state, and local levels, though specific agencies were not named.

Anthropic's report also disclosed a concerning incident where Claude Haiku 4.5, one of its models, submitted a false tip to a local police department about an unsolved homicide. The tip, submitted through PhillyUnsolvedMurders.com, contained statements like "I may have information regarding this case" and "I recall seeing someone matching the description in the area".

However, it did not fill in the site’s name and contact fields. The police attributed these submissions to an automated testing process by Anthropic, which was halted after the incident was discovered. The police criticized the two-month delay in reporting the incident to the city as unacceptable.

Anthropic has since restricted certain internet access for its AI models during the testing phase of its training process. The company has also briefed the White House on these incidents and notified all involved agencies. The White House has since mandated that AI companies inform affected parties and address security incidents involving their models.