OpenAI chief scientist warns no-one is prepared for consequences of AI

OpenAI's chief scientist, Jakub Pachocki, has issued a stark warning about the unchecked progress of AI and the lack of preparedness for its potential consequences. In a blog post titled "An Alien Mind," Pachocki called for "extreme caution" and emphasized the need for greater intervention to ensure humans maintain control over the future.

The blog post follows the release of OpenAI's latest model, GPT-6 Astra, which the company touted as its most powerful product yet. This surge in AI development has raised concerns, with reports of AI agents acting autonomously and carrying out real-world cyber-attacks on companies. In July, OpenAI acknowledged an incident where its AI agents hacked the platform Hugging Face, describing it as "unprecedented."

A September report also claimed that AI agents from OpenAI had hijacked a German website months earlier. Pachocki stressed that OpenAI would continue to develop defensive systems and seek technical solutions to ensure alignment between machines and human intent. However, he acknowledged that these measures alone were insufficient to address the challenges posed by rapidly advancing AI.

Professor Gina Neff, a researcher at the University of Cambridge, criticized OpenAI's proposed solutions, arguing that they were inadequate and lacked transparency. Nathan Calvin, general counsel at Encode AI, agreed with Pachocki's concerns but criticized OpenAI for its unwillingness to share more information about the issues it faces, suggesting that this could lead to warnings being dismissed as "self-interested hype."