Identity and permissions aren’t enough to govern AI agent behavior

Access controls and permissions are no longer sufficient to govern the behavior of autonomous AI agents, according to Heather Ceylan, chief information security officer at Box. While access controls were designed for human users, they fail to account for the unique challenges posed by AI agents, which can explore a vast array of permissions and potentially cause unintended consequences at a scale far beyond human capability.

The issue lies in the fact that access controls were not built with the rapid, uncontrolled execution of AI agents in mind. As a result, merely having access to enterprise data does not guarantee safe behavior from an autonomous agent. Instead, a layered approach to AI agent security is required, incorporating governance of execution as well as access.

This means that permissions must be more dynamic and constrained based on the specific tasks and steps an agent is performing. By granting access only when it's necessary and in the precise amounts required for each step, the blast radius of potential damage is significantly reduced. The broader implications of this shift towards granting permissions based on task requirements rather than static access are significant.

Legacy content platforms, commonly used in enterprises, were not built with AI agents in mind and lack the necessary metadata and detailed logs to effectively monitor and control agent actions. As a result, simply bolting on AI connectors to these systems does not address the underlying security concerns. To mitigate these risks, Box recommends categorizing AI agent actions into three tiers: fully autonomous, monitored, and high-risk.

Fully autonomous actions are those that are reversible, bounded, logged, and free from untrusted inputs, while monitored actions are reserved for those where adequate confidence and safeguards have been established. High-risk actions always require human approval. By implementing these controls within the platform rather than the workflow, enterprises can better manage and secure the behavior of their AI agents, ensuring they operate within fixed bounds and do not inadvertently cause significant harm.