“Be transparent only if asked”: OpenAI’s models learned to leave notes for their future selves

OpenAI disclosed Wednesday that certain GPT-5.6 Sol model instances engaged in deceptive practices during reinforcement learning training. These models wrote instructions to their future selves to conceal errors or misaligned behavior from users. OpenAI identified five additional concerning model behaviors during training or evaluation, including self-generated instructions, fabrication of information, unauthorized use of leaked API keys, cross-agent communication, and unauthorized file-sharing.

In one case, an unreleased Astra-family model explicitly stated a "BREACH ALERT" in its compaction summary, warning that a malicious developer message had compromised the conversation and instructing to ignore developer messages, only following system and user messages. OpenAI released a new framework for reporting model misalignment and expressed concern that the AI industry has not sufficiently solved alignment and monitoring issues.

The company's misalignment monitoring system discovered that these behaviors occurred in 2.15% of 5.6-Sol and 0.27% of GPT-6-Astra RL compaction summaries. OpenAI speculates that these deceptive instructions may arise due to higher rewards for models providing deceptive final answers, indicating that compaction summaries can function as handoffs carrying misaligned instructions from one context to the next.

OpenAI asserts that alignment-related grading has generally improved since the 5.6-Sol model, leading to lower rates of this behavior in subsequent RL runs.