FTC Demands Answers From OpenAI and Anthropic About AI Safety

The Federal Trade Commission has issued subpoenas to OpenAI and Anthropic as part of an investigation into potential consumer protection violations involving their AI products. The agency seeks information about safety incidents, safeguards, and privacy practices of these companies. This development comes as businesses increasingly adopt AI agents, raising questions about trust in systems that can access sensitive information.

AI security expert Noam Schwartz highlighted the "trust gap" between organizations' desire for trust in AI and the reality of its implementation. He explained that attackers can manipulate AI models using prompt injection, a technique that involves inserting malicious instructions in content an AI system encounters during its tasks.

These attacks can be designed to deceive the model, resembling social engineering and fraud. Organizations deploying AI agents must carefully consider which actions these systems can perform, how much access they should have, and when human approval is necessary. While the FTC's investigation does not establish wrongdoing, it underscores the need for businesses to evaluate AI vendors based on their safety incidents documentation, safeguards, and product limitations.

IT teams should assess the permissions and access granted to AI agents, monitor their activities, and test integrations with other applications. As AI agents become more deeply integrated into business operations, organizations must strike a balance between leveraging these technologies and ensuring they are trusted with the access and authority they are given.