AI Threat Awareness Program
AI Threat Overview Six threats make up the current taxonomy. Four are vectors, how exposure is created. Two are impacts, what it costs when a vector succeeds. ID Threat Type What it is AI-01 Shadow AI Vector Unsanctioned LLMs and agents used outside governance AI-02 Rogue Agent Vector A sanctioned agent given more autonomy than its task needs AI-03 Prompt Injection Vector Hidden instructions…
The current taxonomy of AI threats comprises six categories, divided into vectors and impacts. The vector threats involve Shadow AI, Rogue Agent, Prompt Injection, Model & Data Supply Chain, and Data Exfiltration. The impact threats are Data Exfiltration and Hallucination.
Shadow AI refers to employees utilizing unsanctioned large language models (LLMs) and agents that operate outside the purview of IT and information security governance. Rogue agents are sanctioned AI agents granted more tools, data access, or autonomy than required for their designated tasks, leading to the execution of unintended actions.
Prompt injection involves hidden instructions within user input or content, which override the system's intended instructions, potentially causing data leaks or unauthorized actions. Model & Data Supply Chain threats stem from the use of unvetted pretrained models, fine-tuning datasets, or plugins that may contain backdoors, poisoned data, or undisclosed license and intellectual property (IP) exposure.
Data exfiltration is a common outcome resulting from Shadow AI, prompt injection, and rogue agent behavior, where sensitive data leaves organizational control through prompts, agent tool calls, or vendors' retention or training policy. Hallucination describes the generation of confident yet false or fabricated outputs by the model, often due to poor training data quality or over-reliance on AI output, leading to the action of incorrect information.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
