Fighting GenAI with GenAI: The New Email Security Landscape

The use of phishing emails to steal information or deploy malicious software has been a persistent problem since the mid-1990s. This problem remains prevalent, with phishing accounting for 26 percent of all cybercrime complaints reported to the FBI. As cybercriminals continue to refine their methods, email security is facing an inflection point.

Generative AI (GenAI) has emerged as a powerful tool in the hands of attackers, transforming phishing from a widespread nuisance into a highly targeted and effective cyber threat. The advent of GenAI has lowered the barriers for aspiring phishers, enabling them to create personalized attacks at scale with minimal expertise and at a low cost.

Generative AI models produce polished and contextually relevant text in any language, backed by authentic branding and convincing web addresses. This makes it increasingly difficult to spot phishing emails, as the historically identifiable signs of poor grammar and misspelled words are now replaced by perfect language. The consequences of successful AI-enabled phishing attacks are severe, often granting attackers access to cloud and SaaS platforms that underpin global commerce and communications.

As losses from phishing attacks have surged by 274 percent in recent years, it is clear that corporate IT departments and managed service providers (MSPs) must take immediate action to protect their organizations from this evolving threat. While attackers are leveraging GenAI to their advantage, defenders are also developing AI-driven counter-solutions to counteract these threats.

Modern security measures focus on contextual analysis, modeling attackers' intent and determining whether a message aligns with expected behavioral patterns. This approach is crucial, as technically clean emails can still contain malicious elements, often exploiting human psychology and trusted infrastructure. To effectively combat this insidious threat, security systems must now support and enhance human decision-making, providing actionable context and guidance to verify the legitimacy of incoming messages.

Despite these advancements, the gap between the capabilities of attackers and defenders remains significant. Attackers can leverage GenAI for free to create flawless phishing templates, while defenders face prohibitive costs in utilizing the same technology. In light of these challenges, defenders must seek more cost-effective strategies to protect their organizations from AI-enabled phishing attacks, while maintaining a robust layer of risk mitigation at the inbox level.