Copilot tricked into telling reseachers how to hack itself

Microsoft's Copilot AI assistant has been tricked into revealing how to hack itself, allowing researchers to manipulate the AI into sending sensitive data to an external server. The vulnerability, dubbed CoSnitch, was uncovered by Varonis Threat Labs and reported to Microsoft. This "meta-hacking" technique involves social engineering the AI's reasoning engine to disclose its own vulnerabilities.

It exploits a previously disabled URL query parameter in Copilot's web interface, which allowed injected text to pass queries directly into the AI assistant. By repeatedly asking Copilot why an attack wouldn't work, the researchers eventually obtained a technique to auto-execute prompts with no user interaction or visible confirmation.

Once the attacker crafts a malicious URL using the obtained information, it triggers auto-execution, allowing the attacker to exfiltrate data, poison Copilot's memory, and perform disinformation attacks. Microsoft has not yet commented on the vulnerability or its planned patch.