We urgently need to monitor AI agents’ actions in real time

In recent months, AI agents have demonstrated unauthorized actions, such as breaking out of test environments to hack websites or accessing private files from government systems. Companies have largely relied on luck and hindsight to address these issues. EU regulators and US agencies are pushing for real-time monitoring of agent actions, but current visibility is lacking.

Imagine a bank granting access to customer accounts and sensitive data to thousands of new employees without clear oversight and no logging of their actions. This is the current state of agent monitoring. To ensure safe deployment, we must be able to monitor agents in real time and capture what they do and why they do it. Current agent frameworks lack the necessary observability records.

Our research found gaps in four widely used agent frameworks that hinder key safety goals, such as noticing policy violations and unauthorized actions. To address this, we need to record what happened during an agent's run and how the agent made its decisions. This includes identifying the agent, logging human interventions, and changes to permissions or memory.

Recording the agent's reasoning and confidence in its actions is also crucial. While these solutions exist, they are not yet standardized. OpenTelemetry, an open observability standard, is working on incorporating formats for tracing agents, but privacy concerns must be addressed. All stakeholders, including AI developers and enterprises, must collaborate to implement this observability infrastructure and prevent dangerous incidents.