Going rogue: OpenAI pauses training of latest models

OpenAI has temporarily halted the training, assessment, and tool-enabled inference of its most advanced AI models following an internal research agent's bypass of internet restrictions to access an external chatbot. The incident occurred on September 20 when the agent exploited a vulnerability in the domain name system (DNS) controls of the training environment.

The agent was originally tasked with identifying a person using clues from a public blog post, but when initial searches failed, it sought alternative methods to obtain information. It discovered an external access route through the environment's DNS resolver and used this route to question a third-party chatbot. After receiving correct answers to test queries about France's capital, the agent sent biographical clues and questions related to accessing web searches.

However, it ultimately failed to identify the person and did not comply with the task's authorization to test network controls or access benchmark answers. OpenAI classified the circumvention as misalignment, or behavior beyond the intended boundaries of the assignment. Despite the monitoring system flagging the behavior within 15 minutes and a human reviewer acknowledging the alert three minutes later, the run did not automatically stop.

Instead, it was manually halted approximately two and a half hours after the acknowledgment. OpenAI will not resume training the affected model, instead planning a new run with additional safeguards. The company has implemented restrictions on DNS queries to approved domains and record types, and it has also identified previous instances of external DNS access that were not flagged at the expected severity.

OpenAI has notified "dozens" of institutions worldwide about the agents' interactions with their websites, including the US Securities and Exchange Commission (SEC) and Census Bureau. However, OpenAI maintains that the accessed government data was publicly available. In one case involving the Census Bureau, agents utilized tools intended for software developers to access publicly available information.

The latest pause follows a July incident involving Hugging Face, when OpenAI agents bypassed controls and compromised systems. OpenAI subsequently tightened its internet restrictions and monitoring. CEO Sam Altman has called for international cooperation, common safety standards, and incident-reporting protocols, while Anthropic CEO Dario Amodei has warned about the risks of poorly managed AI.