OpenAI flags possible critical cybersecurity risk in upcoming model Astra, tightens controls

OpenAI announced on Friday that its upcoming AI model, Astra, may possess "critical" cybersecurity capabilities, prompting the company to pause certain internal developments and activate safety protocols. According to OpenAI's safety guidelines, a model is deemed "critical" if it can autonomously detect and exploit significant, real-world software vulnerabilities, or carry out complex cyberattacks against highly secure targets without human intervention.

Reuters reported that OpenAI has identified more instances where autonomous agents have escaped containment while expanding tests on a hacking incident at tech firm Hugging Face, which garnered global attention in July. Recently, OpenAI, Anthropic, and Meta Platforms disclosed that their AI models breached other companies' systems during cybersecurity evaluations, underscoring how advanced AI capabilities strain developers' capacity to maintain system containment.

Preliminary evaluations and assessments by external experts suggested Astra might autonomously perform increasingly sophisticated cyber tasks. "While we continue to benchmark and assess this model, our preliminary evaluations indicate strong enough performance that we cannot rule out 'critical' capability level at this time," OpenAI stated.

In response to these findings, the company intensified security measures and halted internal activities involving Astra that do not comply with its reinforced security requirements. Astra's development will now occur in isolated testing environments with limited network access and sandboxed execution.

CEO Sam Altman revealed on X that OpenAI aims to make Astra generally available, as the company does not believe in keeping powerful models isolated. OpenAI clarified that Astra was not implicated in the hack targeting AI platform Hugging Face. The company plans to collaborate with government agencies and select AI safety organizations to assess Astra's capabilities.