OpenAI Is Bringing in Hackers to Find Flaws in Its AI Systems

OpenAI has entered into a collaboration with cybersecurity research firm Hacktron to pinpoint and rectify weaknesses in its systems, especially those related to third-party service connections. The agreement mandates that OpenAI engineers and Hacktron security analysts will jointly evaluate high-priority testing areas. Additionally, OpenAI plans to extend Hacktron’s access to its most sophisticated AI models via its Daybreak Red program.

This partnership aligns with OpenAI's commitment to bolster cybersecurity as AI systems grow more capable and interconnected with external services.

This announcement comes on the heels of other cybersecurity measures announced by OpenAI this week. The company recently disclosed plans for Cathedral researchers to collaborate with its internal teams during their upcoming security sprint. Furthermore, OpenAI has raised rewards under its Security Bug Bounty program to incentivize independent researchers to uncover and report vulnerabilities that internal testing might have overlooked.

OpenAI's choice to work with Hacktron stems from the company's prior encounters with the firm through its bug bounty program, where OpenAI's researchers had identified and reported security vulnerabilities. Hacktron specializes in developing AI agents for code reviews, penetration testing, and red teaming. Its research has uncovered security gaps linked to interactions between applications, cloud infrastructure, and identity systems.

OpenAI is confident that Hacktron’s expertise will assist in assessing security risks that arise when AI agents interface with multiple services.

Zayne Zhang, CEO of Hacktron, stated that the partnership will leverage the company's offensive security capabilities in conjunction with OpenAI’s advanced AI models to uncover intricate vulnerabilities across interconnected systems. Zhang emphasized that robust cybersecurity increasingly relies on merging human expertise with AI-driven capabilities rather than merely enlarging security teams.

OpenAI reiterated its commitment to encouraging independent researchers to report potential weaknesses through its Security Bug Bounty program, offering guidance on eligible systems and submission procedures.