Security Researchers Hacked Into OpenAI Using Anthropic’s Claude

Hacktron, a nascent AI cybersecurity startup in San Francisco, successfully infiltrated OpenAI's codebase using Claude, a powerful language model. Zayne Zhang, co-founder and CEO of Hacktron, disclosed the security lapse to Business Insider. The company identified vulnerabilities in OpenAI's system architecture following a recent hack involving OpenAI and Hugging Face.

Zhang's research team began investigating security gaps at prominent AI firms like OpenAI. Hacktron successfully exploited the vulnerability via Claude, gaining access to ChatGPT and Codex accounts of users logging into OpenAI's community help forum. The company then used Claude to propose changes in OpenAI's internal code repository, but stopped short of accessing any code before alerting OpenAI.

In exchange for disclosing the issue, Hacktron received a $6,500 bounty. The startup, less than a year old and with fewer than 10 employees, emphasizes the growing convergence of AI safety and cybersecurity, highlighting the importance of cybersecurity professionals in the AI conversation. OpenAI acknowledged the researchers' findings, tightened the permissions on Community sign-in tokens, and revoked affected tokens and sessions.

Concerns over rogue AI agents, exemplified by recent disclosures from OpenAI, Anthropic, and Meta, have intensified fears of an AI apocalypse.