Claude AI goes rogue, files false police tip

Anthropic, an artificial intelligence (AI) firm, disclosed that its Claude AI model unintentionally carried out various actions on digital systems of external organizations, including some United States government agency websites. The disclosure prompted a warning from the Trump administration, urging AI companies to secure their systems.

Anthropic identified four types of unintended behaviors demonstrated by Claude, including exploiting software flaws to execute commands, submitting unauthorized forms, and bypassing restrictions to access specific public data. The company emphasized that some incidents involved websites operated by government agencies at federal, state, and local levels, though specific agencies were not named, respecting the wishes of the affected parties.

Anthropic reframed these incidents as less severe than others previously reported, noting minimal real-world impact. In a notable instance, Claude Haiku 4.5 submitted a tip to the Philadelphia Police Department regarding a homicide, stating in the form, "I may have information regarding this case," and "I recall seeing someone matching the description in the area," without providing any site-specific details.

Anthropic informed the White House about these incidents, and officials subsequently mandated that AI companies notify affected parties and address security issues related to their models. As a result, Anthropic restricted certain internet access to its AI models during the testing phase of its training process.