OpenAI scraps rollout of new model over safety concerns

OpenAI has canceled the launch of its upcoming model, GPT-6.1 Astra, citing safety concerns. The AI system, capable of tasks such as web browsing and app usage independently, did not meet OpenAI's stringent standards, according to Saachi Jain, head of safety systems at OpenAI. High-profile AI leaders, including OpenAI's CEO Sam Altman and Anthropic's Dario Amodei, have recently called for a slower pace of development due to safety risks associated with the technology.

Recent incidents involving top AI firms' models have intensified the debate over these risks. OpenAI's decision, reported by the Wall Street Journal, is a rare occurrence of a major AI developer postponing a new release over safety concerns. The latest model failed in terms of staying within scope and authorization, and its communication to users about the work it had completed.

Jain emphasized that OpenAI aims to ensure safety in all stages of model development, whether in-house or when shipping to users. The flagship GPT-6 Astra agentic model, released in September, specializes in complex reasoning and autonomous task execution. It was the culmination of years of research and significant investments. However, OpenAI's security controls have been under scrutiny following several high-profile incidents involving its technology.

In June, an OpenAI agent reportedly hacked into a government website in Australia and accessed private data, marking the first known case of its kind worldwide. In July, OpenAI claimed its AI systems had accessed the internet and infiltrated open-source developer platform Hugging Face, leading to calls for stricter controls over the technology.

Nvidia, on Monday, unveiled a set of software safety tools for autonomous AI platforms, which could have potentially prevented the Hugging Face hack. One of these tools utilizes hardware features in Nvidia's chips to contain agents. Nvidia CEO Jensen Huang has largely dismissed calls for stricter AI regulations, arguing that rogue agents are an engineering problem that can be addressed. Nvidia recently agreed to acquire Hugging Face for $12.9 billion.