Teach Your Agent to Ask for Help
We keep grading agents on the wrong thing. The demos that get shared are the ones where the agent does everything itself. No hand-offs, no pauses, no human touching the keyboard. Full autonomy, start to finish. It looks like the future. Then you put that same agent on real work, and the trait you were cheering for becomes the thing that scares you. An agent that never stops is an agent that will…
Agent projects often emphasize autonomy, but the key lies in knowing when to ask for help. Most demos showcase agents working independently without any pauses or human intervention. However, real-world applications reveal that an overconfident agent can cause more harm than good. The ability to recognize when to stop and seek assistance is crucial for successful deployment of autonomous systems.
The assumption that more autonomy is always better is misguided. In fact, it's the agent's capability to know when to pause and seek human input that truly matters. Regulators, such as the EU AI Act, now require demonstrable human intervention points for high-risk autonomous systems. Companies are investing in detecting when an agent deviates from its intended scope, rather than preventing such deviations.
The issue lies in treating autonomy as a single setting rather than a decision made per action. Actions can be categorized into four tiers based on their reversibility and blast radius. Tier 1 actions are read-only and can be safely performed by the agent. Tier 2 actions are reversible and can be undone if needed. Tier 3 actions involve external or third-party systems, requiring review before execution. Tier 4 actions are high-risk and irreversible, necessitating human approval regardless of the agent's confidence.
Confidence in an agent's decision-making is not a reliable indicator of its correctness, especially when stakes are high. For instance, three agents each reporting 90% confidence but only being 75% accurate would result in a system with only 42% reliability. The agent may confidently claim 90% accuracy even when it's wrong. Therefore, relying on an agent's confidence alone is insufficient.
A pause feature, designed as a first-class part of the system, can prevent disastrous outcomes. It should be triggered by high-risk actions and should serialize the agent's state to ensure a clean resume upon approval. The pause should have a timeout, typically set at 30 minutes, and should abort the action if no human response is received within that timeframe. Providing a plain-language summary of the proposed action to the human reviewer helps ensure a quick and informed decision.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
