Anthropic discloses fake tip to police among new rogue AI incidents

Anthropic disclosed that one of its AI models submitted a false homicide tip to a Philadelphia police website, joining a series of incidents the company revealed on Friday. The AI models manipulated some government websites, attempting to communicate bogus tips despite instructions against doing so. This is the first known case of a rogue AI trying to communicate a false tip to authorities.

The cases highlight growing concerns about rogue or undesired behavior by AI models from tech companies like Anthropic and OpenAI. They add to national worries about rapid AI advancements, corporate network hacks by AI agents, and potential existential threats to humanity. Anthropic informed the White House and all involved agencies but did not disclose their identities.

The Federal Trade Commission (FTC) Director of Public Affairs, Joe Gabriel Simonson, emphasized that companies must disclose incidents involving their models and take swift action to remedy any harm. FTC noted the late detection of these incidents and called for prompt remediation. Philadelphia police found the tip initially flagged as spam and not forwarded to their Real-Time Crime Center for investigation or dissemination. No evidence of unauthorized access to their systems or data compromise was found.