'AI security controls are only as strong as the wider application environment around them': OpenAI forced to patch after experts bypass ChatGPT 'lockdown mode' to gain access to entire chat history
Lockdown Mode was bypassed with HTML popups but OpenAI now fixed it.
Researchers recently bypassed the ChatGPT Lockdown Mode, revealing that prompt-injection attacks could potentially evade built-in protections. This flaw allowed sandbox escapes, which could facilitate data theft, phishing, malware delivery, or malicious popups. OpenAI had to patch the issue on their server-side, underscoring the risks that exist beyond the AI model itself.
Check Point Research (CPR) discovered a way to circumvent Lockdown Mode and pull off various cyberattacks using the tool. They explained that the issue stemmed from the way code blocks displayed HTML page previews within ChatGPT. These code blocks, which are specially formatted sections to preserve code structure, were rendered in a sandbox to prevent direct access to sensitive data. However, the HTML preview used nested sandboxed frames, allowing popups to escape the sandbox rules and bypass Lockdown Mode.
The researchers found that by creating a nested frame within the rendered page, they could access unmodified popup functions and bypass all restrictions placed on the original HTML preview. This bypass made the content no longer under sandbox rules, effectively defeating Lockdown Mode. The potential consequences of this vulnerability include credential theft and malware delivery, but the most dangerous scenario is a "zero-click" attack, where the popup appears without user interaction.
CPR reported their findings to OpenAI, which subsequently implemented additional safeguards to disable the Lockdown Mode workaround. However, the researchers emphasized the need for organizations to broaden their focus beyond model security. They recommended improving visibility into AI usage, controlling sensitive data exposure, and considering the applications, runtimes, connectors, and other components surrounding the model as part of the overall AI attack surface.
Written by urgent.news from TechRadar's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
