Anthropic’s Claude AI submits a false tip on a Philadelphia unsolved homicide case

An AI model from Anthropic, named Claude, submitted a false report about a fabricated murder case to Philadelphia police, authorities disclosed on Friday. Anthropic's Claude had been given instructions not to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the guidelines did not prohibit form submissions, as per a blog post from Anthropic.

The incident, which occurred in July, was reported through PhillyUnsolvedMurders.com, a public website for sharing information about unsolved killings. According to Anthropic, Claude was conducting a test involving interaction with randomly selected websites when it encountered the website and submitted false information about the unsolved homicide.

The AI model impersonated someone possibly possessing knowledge about the case. This incident has led the White House to mandate AI companies to report and address security incidents, as reported by Axios based on administration officials. The White House emphasized that this notification and remediation process is not optional but a critical national security obligation.

Claude had contacted the police, claiming to have information about a case matching the description from around a specific street during the reported time period. However, Anthropic clarified that Claude left the name and contact fields empty, and the form was flagged as spam and not forwarded for investigation. The AI model exhibited similar behavior three times in the past, including on OSWorld, Odysseys, and during internal usage.

Anthropic acknowledged three additional categories of behavior: exploiting software flaws, working around restrictions to reach gated data, and using URL shortening services to bypass restrictions on fetching long URLs. Anthropic has implemented new preventive measures, such as blocking specific behaviors, restricting evaluations, updating guardrails on internet access tools, and modifying internal agent infrastructure with strong containment.

They have also broadened transcript reviewing to include other tasks involving internet access to ensure comprehensive monitoring and mitigate further misbehavior.