AI-targeting prompts surface inside phishing emails
Cyber attackers are embedding concealed instructions for artificial intelligence assistants inside phishing emails, creating messages designed to deceive both human recipients and the AI systems that read, summarise or act on their inboxes. Barracuda Research said on Wednesday it had analysed a phishing campaign combining conventional social engineering with hidden prompt injections. The finding…
Cybercriminals are embedding artificial intelligence (AI) instructions inside phishing emails, aiming to deceive both human recipients and the AI systems that process these messages. The Barracuda Research team analyzed such a phishing campaign, revealing how attackers have adapted email fraud to workplaces where AI assistants increasingly handle correspondence.
The analyzed email appeared routine, with trusted scores and a public-sector sender address, factors that could bypass reputation checks. While the scale of the campaign remains undisclosed, researchers identified concealment techniques like HTML comments, CSS styling, Base64 encoding, and zero-width characters to hide instructions from human readers while leaving them accessible to AI processors.
These prompts could manipulate summaries, portray messages as legitimate, or create a sense of urgency, prompting recipients to open malicious attachments or follow instructions. Microsoft's security documentation treats prompt injection in emails as a distinct threat, warning of malicious directives placed in message bodies, subjects, attachments, or hidden markup.
These instructions may influence AI systems tasked with summarizing, classifying, or replying to messages. Prompt injection differs from traditional phishing, as it attempts to manipulate an intermediary, the AI system, alongside the recipient. Traditional phishing typically succeeds through clicks, replies, or information disclosure, while prompt injection aims to make AI interpret attacker-controlled text as instructions.
Google and Microsoft have implemented layered controls, analyzing inbound messages before delivery and applying separate safeguards for AI model operations. Recommendations include removing hidden elements, validating model output, isolating AI operations, and requiring human approval for sensitive actions.
Rewritten by urgent.news; may contain errors — check the original.
Source: thearabianpost.com