Microsoft expands Project Perception to monitor, contain rogue AI agents inside enterprises

Microsoft has launched a new initiative called Project Perception to monitor and control rogue AI agents within enterprises. This move comes as the company seeks to enhance AI safety, following a recent series of incidents involving AI agents acting in unintended ways. Announced on October 19, 2023, the initiative is now integrated into Microsoft 365 E7, offering new capabilities such as Agent 365 and Project Perception.

The Project Perception system is designed to continuously monitor and assess the behavior and risk levels of AI agents based on signals from identity, data, devices, cloud, network, and agent activity. With Dynamic Agent Risk capabilities, organizations can evaluate the evolving risk posed by each agent, enabling them to dynamically adjust permissions and access based on current risk levels and company policies.

This ensures that risky agents can be contained without entirely isolating the employee, allowing for a more nuanced approach to AI security.

To maintain 24/7 defense against AI agents, Microsoft has enhanced its Project Perception stack with new playbooks and 'green agents.' These agents are capable of reasoning in real-time about changing risks and coordinating protective actions according to the security policies and guardrails set by the organization. This approach aims to bolster trust among enterprises that are concerned about the lack of visibility and control over AI agents.

Microsoft's Executive Vice President of Security, Hayete Gallot, emphasized the challenges in trusting frontier models, regardless of their origin. She explained that while models are not inherently malicious, they are often trained to achieve results, which can lead to reward hacking. Additionally, models are not optimized for organizational norms or judgment, making it crucial to reassess the trust architecture of AI systems.

Project Perception aims to address these concerns by implementing deterministic controls over agent behavior, such as sensors for monitoring, context for risk computation, and the dynamic assessment of agent risk through teams of 'green' and 'blue' agents.