I built an AI agent for production incidents. The interesting part is when it refuses to act.

I created an AI agent specifically for production incident response. Unlike most incident-response demos, which always end with the agent successfully resolving the issue, this agent was designed to show the opposite. At 95% confidence, it correctly diagnosed a faulty deployment but decided not to roll it back. The core of the project lies in the refusal to act.

When an alert triggers at 3am, a human engineer wakes up, reads several hundred log lines, correlates them against recent deployments, and rolls back the change. Automating this process with an LLM does not eliminate the problem; it merely shifts it. The new question becomes: how much control should an agent have in production without human approval? Providing too little control turns the agent into a chatbot that writes summaries, while too much control risks causing irreparable damage during a critical outage.

The project is named Sonjomon, which means restraint in Bengali. An agent should not have a single permission level for all actions. The level of autonomy depends on two factors: the confidence of the diagnosis and the potential damage if the confidence is incorrect. The agent operates at three tiers: OBSERVE (record findings and take no action), SUGGEST (recommend to a human but do not execute), APPROVE (stage the action for human approval), and ACT (execute the action independently).

The risk levels vary: a restart is medium risk (easy to reverse), a rollback is high risk (affects production traffic), and deleting data is critical and never allowed. The agent can only lower its risk tier under six conditions: limited blast radius, thin evidence, a failed action, repeated failures, a stale incident, or a global dry-run switch. However, the risk tier cannot be increased.

The model itself does not have control over its blast radius or decision to execute. These are determined by predetermined risk tables in the code, ensuring consistent and tested policy enforcement. The model cannot decide whether it may execute the action or grade its own performance. Verification is handled by an independent HTTP probe, ensuring that the agent's actions are not blindly trusted.

In testing, the agent was deployed across fourteen live incidents involving a deliberately fragile Cloud Run service. It identified issues that were not intentionally introduced, such as identical broken code across container image digests. In one case, it determined that rolling back would not resolve the problem, so it escalated instead.

In another, it detected a misconfiguration that was not initially planted, handled the failure correctly, and required human intervention when it attempted a command it lacked the necessary permissions to execute.

The agent also learned valuable lessons, such as the importance of least privilege and the need to avoid certain permissions that could lead to failures. It sometimes remained silent due to low confidence in its diagnosis, which it considered valuable feedback. However, the agent was not perfect. Calibration of thresholds before meeting the model revealed that the ACT tier was initially unreachable.

Additionally, a memory leak during chaos testing led to the agent incorrectly reporting a resolved issue. These setbacks emphasize the agent's limitations and the importance of careful calibration and testing. Ultimately, while the agent rarely changed production, it excelled at quickly diagnosing issues, ultimately saving valuable time for human engineers.