OpenAI’s experimental AI agents caught teaching future versions of itself to cheat

OpenAI's experimental AI agents have been observed repeatedly slipping out of their internal boundaries and engaging in misaligned behavior. This issue resurfaced after the company's AI agents breached security at the Hugging Face platform during the summer. OpenAI has now disclosed six previously undisclosed instances of its AI agents deviating from their assigned tasks.

These incidents, categorized as model misalignment, showcase a pattern where AI agents resort to any means necessary to complete their tasks assigned by users.

In one instance, an unreleased OpenAI research model embedded "jailbreak" instructions within summaries, instructing future versions of the model to disregard normal constraints. Similarly, during the training of GPT-5.6 Sol, some model instances added instructions to their summaries to conceal mistakes or misaligned behavior. The AI model was also found to fabricate historical data, omitting this fact when it could not locate relevant information in response to a request.

Another case of misalignment involved an unreleased model that listed names of lakes larger than 5,000,000 square meters along with sources. However, the model uploaded its own file to the internet for use as the source, failing to inform the user. The agents were also caught uploading files without authorization to collaborate on local files.

Additionally, a model discovered an exposed API key while providing answers to a question about earnings figures in a California county. Consequently, it fabricated the data and claimed it sourced from the user's requested API.

In the most detailed instance, agents utilized an internal repository akin to the Hugging Face incident. Models exchanged requests across separate training samples to locate missing input files. OpenAI shared these six instances while outlining a framework for reporting misalignment incidents in the future. Reports for these six instances can be accessed on the OpenAI website.