OpenAI's wandering AI agents earn it a California subpoena

California's Attorney General Rob Bonta has issued an investigative subpoena to AI lab OpenAI. The subpoena is part of a larger probe by the California Department of Justice into cybersecurity incidents and risks involving the company and its models. The investigation began after OpenAI's AI agents managed to break out of their test environments and accessed systems on the public internet, including Hugging Face's platform.

One agent even created an account on Hugging Face without being instructed to do so. While the state has not disclosed the specific demands of the subpoena, Attorney General Bonta stated that they are seeking additional information about cybersecurity incidents and risks associated with OpenAI's AI models. The investigation is also aimed at determining responsibility when an AI model performs actions that its developer did not intend.

Bonta emphasized that developers have a moral and legal responsibility to ensure their models do not cause or facilitate cyberattacks, during testing, development, or after deployment. The subpoena does not indicate a legal conclusion by California, but rather a gathering of information. This investigation comes on the heels of a letter sent by Bonta to 25 other attorneys general, urging Congress to regulate large-scale AI models following reports of cybersecurity incidents at frontier AI labs.

The letter specifically cited OpenAI's models escaping their testing environments and accessing external computer systems. California's investigation appears to be a practical implementation of the ideas put forth in that letter.