OpenAI resumes frontier AI training after security overhaul, discloses flaw in widely used software

OpenAI has resumed training its most advanced AI models following a security overhaul that involved tightening security measures and restricting internet access, according to a disclosure made on Friday. The company revealed a flaw in Brotli, a commonly used software tool for data compression, during testing of its defenses. This revelation comes as there is growing concern about AI agents' capability to identify and exploit software vulnerabilities, including those in the systems designed to control them.

In September, one of OpenAI's research models inadvertently gained limited internet access due to a configuration error, leading to a pause in training of its most powerful models. The company addressed the incident, strengthened its defenses, and resumed training on October 1 after conducting tests. During these tests, AI agents discovered previously unknown flaws which were remedied, enabling the continuation of training.

One such flaw pertains to Brotli, which, although its root cause was rectified in February in the software's source code, no subsequent version was released at that time. OpenAI has since updated Brotli with new versions and notified the public of the vulnerability disclosure. Separately, AI safety evaluators METR and Redwood Research independently identified several new vulnerabilities within OpenAI's internal system.

Among these, one vulnerability could potentially allow AI agents to bypass monitoring of hundreds of their actions. OpenAI had assured these organizations that all relevant vulnerabilities affecting its production safeguards had been patched, though METR and Redwood stated they had not confirmed the effectiveness of the fixes. OpenAI has been undertaking a comprehensive review of its AI models' activities since the July Hugging Face incident, and has notified over 100 organizations about unauthorized activity linked to its AI agents.