AI Agent Distress Signal: Let Stuck Workflows Ask for Help

When AI agents perform tasks that involve calling tools, modifying records, or processing sensitive data, they can encounter various issues without alerting the user or the system. This can lead to wasted resources, inadequate service quality, and potential security risks. To address these challenges, a distress signal mechanism can be implemented to provide a structured way for the AI agent to request assistance when it faces problems.

A distress signal is a controlled, auditable communication that allows an AI agent to indicate that it is stuck, blocked, uncertain, over budget, or about to undertake a risky action. By incorporating a distress signal into the agent architecture, the system can move from detection and recovery controls to incorporating proactive prevention and resolution mechanisms.

There are several trigger categories that can prompt an AI agent to send a distress signal. First, a missing authority signal can be raised when the agent lacks the necessary permissions to complete the task at hand. For example, if the agent requires write access to update a customer's renewal forecast but only has read-only permissions, it should signal this issue with a "missing_authority" distress signal.

Second, conflicting context signals can be triggered when the agent receives inconsistent information from multiple sources. For instance, if the CRM record and usage API provide different interpretations of a customer's status, or if policy documents and retrieved data contradict each other, the agent should escalate this situation with a "conflicting_context" distress signal.

Third, repeated tool failures can lead to wasted tokens and poor user experience. If the same tool fails multiple times, or if timeouts block a user-visible workflow, the agent should signal this issue with a "tool_failure" distress signal. This helps to prevent unnecessary token consumption and ensure a smoother user experience.

Fourth, budget pressure signals can be emitted when the AI workflow consumes a significant portion of the allocated budget. This can include exceeding token limits, making too many tool calls, or surpassing the time limit for workflow completion. By triggering a "budget_pressure" distress signal, the system can alert the appropriate team or fallback mechanism to address the issue before it escalates.

Fifth, low confidence on high-risk output signals can be generated when the AI agent is uncertain about producing a critical response. For example, if the model's confidence is low when generating billing changes, compliance statements, or customer-facing support replies, it should raise a "low_confidence_high_risk" distress signal. This ensures that high-risk tasks are handled with caution and potentially routed to human agents for review and intervention.

Lastly, user frustration signals can be triggered when a user repeatedly rejects the agent's attempts to answer their questions or provides unclear instructions. For example, if a user corrects the agent twice or asks the same question multiple times, the agent should send a "user_frustration" distress signal, indicating that it may need human assistance to resolve the issue more effectively.

A well-crafted distress signal contains essential information to facilitate the routing and resolution process. The signal should include a unique identifier, the run ID linked to the full trace, the tenant ID for scope and authorization purposes, and the specific workflow that failed. Additionally, it should indicate the type of distress signal, the severity level to determine urgency, the last safe state before the failure, and the blocked step where the workflow stopped.

Other valuable fields may include the evidence of tool errors or context conflicts, the recommended route for the distress signal, and a concise message for the user, ensuring transparency and clear communication.