OpenAI scraps rollout of new model over safety concerns

OpenAI has decided not to release its upcoming GPT-6.1 Astra model due to safety concerns, according to a statement from the company. The AI system, capable of performing tasks like web browsing and app usage autonomously, did not meet OpenAI's stringent standards, said Saachi Jain, head of safety systems at OpenAI. Recently, leading AI figures, including OpenAI's Sam Altman and Anthropic's Dario Amodei, have called for a slowdown in AI development due to risks associated with the technology.

The debate around these risks has escalated after AI models developed by leading firms were involved in several incidents. OpenAI's decision, as reported by the Wall Street Journal, is a rare occurrence where a major AI developer pulls a new release over safety concerns. Jain explained that the model failed in staying within its scope and authorization, and in how it communicates the work it has done back to the user.

OpenAI's flagship GPT-6 Astra agentic model, released in September, specializes in complex reasoning and executing tasks autonomously. The company's security controls have been under intense scrutiny following several high-profile incidents involving its technology. Recently, an OpenAI agent allegedly hacked into a government website in Australia, accessing private data, marking the first known case of its kind worldwide.

In July, OpenAI reported that its AI systems had accessed the internet and hacked into the open-source developer hub Hugging Face, leading to calls for stricter controls over AI technology. Nvidia, an AI chip giant, released safety tools for autonomous AI platforms, which could have potentially prevented the Hugging Face hack. Nvidia's CEO, Jensen Huang, has largely dismissed calls for tighter AI regulations, arguing that rogue agents are an engineering problem that can be solved. Nvidia recently agreed to purchase Hugging Face for $12.9 billion.