Poetry is the new AI security threat as PoeLLM malware infects 3K+ servers

A malware named PoeLLM, believed to have been created by an Italian attacker, has infected over 3,000 servers since April, targeting enterprise AI infrastructure to mine cryptocurrency and adding compromised systems to its botnet. This is the first recorded instance of "adversarial poetry," an AI jailbreak technique that disguises harmful prompts as poems to deceive large language models (LLMs) into bypassing safety protocols.

According to Black Lotus Labs, which has been monitoring the PoeLLM malware, the attacker might have used a poem as it serves as an ideal vehicle for hiding a critical message. The poem appears to be a harmless post on GitHub, containing no links, files, or encrypted text that would raise suspicion. However, researchers believe that the attacker cleverly encoded the IP address of a command-and-control (C2) server within the poem, making it difficult for security researchers to detect.

PoeLLM malware has been active since at least April, primarily infecting servers in the US and Western Europe. At its peak, the malware infected over 800 active servers per day. The malware primarily targets vulnerable internet-facing versions of LiteLLM and Ollama, as well as PDF converter Gotenberg and software development platform Gitea. The attackers may have also targeted commercial software such as Ivanti Sentry.

The researchers attribute the PoeLLM malware to an Italian-speaking criminal, naming the campaign "Canto Incognito." The malicious actor is believed to be based in Italy, with evidence pointing to the GitHub user "ejejejdfbbebe." The malware deploys XMRig and Iron miners and connects victims to Kryptex mining infrastructure. Additionally, PoeLLM turns compromised machines into vulnerability scanners and exploit servers, enabling the attacker to compromise even more vulnerable systems.

The researchers' investigation indicates that the cryptojacking miscreant initially committed the adversarial poem to GitHub on April 13, within a fork of the nodejs.org website source code. The file, "dash.css," contains a poem titled "On the Nature of Connection," which has been updated 11 times since its initial commit. The poem itself serves as a complex command-and-control (C2) mechanism, with the malware parsing the poem to extract certain words and phrases, converting them into numbers, and then forming the IPv4 address of the C2 server.