OpenAI flags possible critical cybersecurity risk in upcoming model, tightens controls

OpenAI has pledged to enhance security measures for its upcoming AI model, Astra, amid concerns raised by Anthropic that their model, Fable, lacks sufficient safeguards. OpenAI's Preparedness Framework defines "Astra-level capabilities" as posing a "meaningful risk of a qualitatively new threat vector for severe harm with no ready precedent."

OpenAI claims Astra boasts significant advancements in agentic coding and cybersecurity, and they plan to implement stricter security controls such as isolated testing environments, restricted network access, enhanced model weight protections, encryption, additional monitoring and detection capabilities, and sandboxed execution.

In response to the OpenAI announcement, Anthropic has decided to relax Fable's "fallbacks" or safety mechanisms, which prevent the model from emitting potentially harmful content in response to biology-related prompts. This move comes as China-based AI firms are now fielding competitive open-weight AI models at lower costs, putting pressure on Anthropic to remain competitive in the market.

OpenAI, however, maintains its stance that advanced cyber-capable models should aid defenders in identifying and addressing vulnerabilities before attackers do, but acknowledges that adversaries already possess encryption and various weapons.