Trust in Depth: Giving agents more autonomy without losing control

Enterprise leaders consistently express a desire for their agents to possess greater autonomy, yet they remain hesitant to fully trust these agents. The very traits that make agents valuable—creativity and autonomy—also make them difficult to trust. When given a task, an agent may determine its own approach, selecting tools and sequencing steps not scripted by a developer.

Even running the same task multiple times could yield varied results. The conventional solution is for a human to oversee each action taken by the agent, but this becomes cumbersome as agents assume more responsibilities. Human approvals pile up, leading to review fatigue and a decrease in the ability to catch crucial errors. Instead of relying solely on human oversight, a new approach called Trust in Depth is emerging. This method centers on verifying the agent's behavior through multiple layers of checks.

Trust in Depth is built upon answering a series of questions at various stages of an agent's lifecycle. Initially, during the scope phase, before the agent begins its work, we must define what the agent is intended to accomplish, the capabilities required, and the behaviors it must never exhibit. At deployment, we need to verify the agent's authorization, the identity under which it operates, and confirm that every agent acting on our behalf is known.

While the agent is executing its task, we must ensure its actions remain within established limits and still achieve the desired outcome. Post-execution, we must be able to reconstruct the agent's actions and learn from them.

Certain requirements can be expressed as precise rules, while others necessitate human judgment. For instance, some decisions may require a manager's approval, while others, such as maintaining a friendly tone or addressing a sensitive situation appropriately, depend on context and human interpretation. Precisely defined rules can be tested for effectiveness, but an agent may encounter paths not previously anticipated.

Automated reasoning, a mathematical logic approach, can verify whether these rules hold true in every possible scenario, establishing proofs when applicable. However, some rules cannot be precisely defined, such as maintaining a friendly tone. In these cases, we must restrict the agent's access and actions, evaluate its behavior, and delegate ambiguity to human judgment.

The Trust in Depth framework encompasses several key layers. First, scoping the agent to its specific job requirements is crucial. This involves determining the work the agent must accomplish, the tools, data, services, rules, and decisions reserved for human intervention. Next, we must derive rules that the agent must follow, using tools like Kiro, which helps define the agent's model, instructions, and tools, while also generating property-based tests to identify potential issues.

Amazon Bedrock AgentCore offers governance controls to safely manage and govern agents, using policies to enforce the established boundaries.

By systematically applying Trust in Depth principles, organizations can grant agents more autonomy while maintaining control and ensuring safety. This approach allows agents to take on higher-stakes work, like financial transactions, with confidence that they will operate within predetermined limits, even in the absence of constant human oversight.