Who on-calls an AI? This is the unresolved crisis of agent ownership
At 2:07 a.m., an AI agent starts failing. The application is still running, but requests are timing out because a The post Who on-calls an AI? This is the unresolved crisis of agent ownership appeared first on The New Stack .
When an AI agent begins to malfunction, it becomes crucial to determine who is responsible for managing that system. Simply asking “Which model should we use?” does not address the core issue. Ownership of the system is the first question that must be answered. Teams frequently focus on prompts, retrieval of information, selection of tools, and choice of model. However, it is the production ownership that decides whether an agent functions as a dependable service or becomes a source of operational risk.
A production agent requires several elements: ownership, boundaries, telemetry, failure policies, escalation paths, and a clear distinction between what the system can automate and what requires human intervention. The agent should not have a direct path from reasoning to action. Each invocation of a tool should go through explicit controls, remain observable, and provide a defined path to human intervention.
To establish these controls, follow these steps:
1. Treat the agent as a production service. Assign ownership metadata to the agent, which should include its name, the team responsible for it, its service tier, and an escalation channel. This transforms the agent from an experimental Python process into a service with an accountable team and an escalation path. This metadata can later be attached to logs, traces, dashboards, alerts, and deployment records.
2. Define what the agent is allowed to do. Establish an explicit list of tools the agent is authorized to use, which limits the agent to the actions you have deliberately granted it. Implement a deny-by-default approach, where adding new capabilities is an intentional engineering decision rather than something that happens automatically when a tool becomes available. For destructive operations, such as a restart or deployment, require authenticated human approval instead of the agent's own judgment.
3. Put budgets around agent execution. Every production workflow should have limits on execution time, model calls, tool calls, and token consumption. Create an ExecutionBudget class that maintains counts for steps and tool calls, and compares them against predefined maximums. This ensures that the agent operates within predefined limits, preventing indefinite reasoning and managing resource usage.
The budget should be configurable by engineers, allowing different budgets for different agents without altering global settings. When no limits are supplied, the class defaults to environment variables.
4. Make model calls observable. An owner cannot manage what they cannot observe. Record essential details about each model call, including latency, the model used, success or failure status, tool calls, token usage where available, correlation or request ID, and any escalation events. A simple wrapper function can provide a useful boundary for these observations. The request ID is particularly valuable when an incident involves multiple services, allowing engineers to track and manage the incident more effectively.
Written by urgent.news from The New Stack's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
