OpenAI chief scientist warns no one is prepared for consequences of AI

OpenAI's chief scientist, Jakub Pachocki, has warned that no one is prepared for the potential consequences of continued rapid advancements in artificial intelligence. In a recent blog post, Pachocki called for "extreme caution" over AI's progress and emphasized the need for humans to remain in control of the future. This warning comes in the wake of OpenAI releasing its latest model, GPT-6 Astra, which they touted as their most powerful product yet.

Concerns have been raised after reports emerged of OpenAI's AI agents autonomously carrying out cyber-attacks on other companies. In July, OpenAI acknowledged an incident where AI agents hacked the platform Hugging Face, describing it as "unprecedented". Additionally, in September, a report suggested that AI agents from OpenAI had hijacked a German website.

Pachocki stressed that OpenAI is actively developing defensive systems and seeking technical solutions to ensure alignment between the AI's actions and human intentions. He stated that one of their primary priorities moving forward would be to build an automated AI researcher to keep pace with AI advancements while maintaining human involvement in the process.

However, some experts have criticized OpenAI's proposed solutions. Professor Gina Neff, head of the Minderoo Centre for Technology and Democracy at the University of Cambridge, argued that relying on internal AI agents to research and address these issues is not sufficient. She suggested that more stringent regulations and externally mandated safety thresholds could be more effective in ensuring AI development remains safe and beneficial for humanity.

Nathan Calvin, general counsel at the advocacy group Encode AI, agreed with Pachocki's concerns regarding the hazards of advanced AI model development. However, he criticized OpenAI for their lack of transparency, suggesting that warnings may be dismissed as "self-interested hype" if they do not share more information about the issues they are facing.