North Korean spies are running local LLMs to cause AI mischief

North Korean cyber spies, operating under the Reconnaissance General Bureau, are utilizing locally deployed Large Language Models (LLMs) to enhance their cyber attack capabilities, according to South Korean security firm Genians. The researchers observed the Kimsuky group setting up and running local LLM environments using tools such as Ollama, GPT4All, and Msty, and experimenting with other AI technologies like Cursor.

By employing retrieval-augmented generation (RAG) for local document searches, the group aims to keep sensitive data within their control, reducing the risk of external exposure.

This strategic move demonstrates Kimsuky's growing integration of AI into their operations, including malware development, data analysis, and the advancement of attack techniques. The security researchers concluded that the North Korean group's recent phishing emails, which employ ZIP archives containing malicious LNK files, are increasingly leveraging AI-generated lures to increase user trust and encourage the execution of malicious files.

Kimsuky maintains a presence on GitHub, hosting configuration files, PowerShell scripts, and various payloads used in subsequent attacks. The group's Git-based Command and Control (C2) infrastructure is also utilized for malware development, stolen data management, and AI technology research. This includes setting up multiple local LLM environments on controlled infrastructure, suggesting that the development components are not merely the result of curiosity but are specifically designed to create an AI-based tool for a particular purpose.

Moreover, Genians uncovered evidence of Kimsuky employing speech-to-text tools, such as OpenAI's Whisper speech recognition models, as well as Cursor AI for code editing and testing Retrieval-augmented generation (RAG) for document-based question answering. These findings indicate a shift away from relying on content-based assessment to detect AI-based threats.

Instead, organizations should focus on behavior-based detection, including the identification of anomalous behaviors following LNK execution, such as PowerShell execution, persistence establishment, and external communications.