What Happens When an AI Agent Runs Longer Than Your HTTP Request?
A user clicks “Run agent” , your backend receives a normal HTTP request, and the agent starts doing what agents do: calling tools, reading documents, querying APIs, waiting for a human approval, retrying a flaky search, and generating a long report. Two minutes later, your load balancer returns 504 Gateway Timeout . The user sees an error. The agent, depending on where it is running, may still be…
When an AI agent run exceeds the duration of an HTTP request, problems arise. Here's the issue broken down:
1. The Gateway Times Out, but the Agent Keeps Spending Tokens
- An HTTP request has a timeout set by gateways, load balancers, or clients
- An AI agent run can continue long after the request times out
- The request returns a 504 Gateway Timeout error to the user
- The agent might still be executing, using tokens, calling tools, and generating results
- The user sees the error, but the agent may have already completed work
2. The Client Retries, and Now You Have Two Agents
- Users may click "Run" again if they don't see a response
- Automatic retries can happen after network issues
- Two separate agent runs will occur for the same user request
- Agents are not pure functions; they may have side effects and tool calls
- Running the same agent twice may lead to duplicate work and inconsistent results
Key points to address these issues:
- Create a durable "run resource" with a unique run ID
- Return a 202 Accepted status immediately instead of waiting for completion
- Execute the agent asynchronously in the background
- Store agent state in a durable system, not just process memory
- Use idempotency keys to prevent duplicate runs when retried
- Handle cancellation explicitly and cooperatively
- Prefer durable execution patterns for long-running workflows
- Use progress notifications (SSE/WebSockets) to keep the user informed
Consider these points before implementing long-running AI agents in your system. The mismatch between the HTTP request lifecycle and AI agent runs requires a different architectural approach to ensure reliable execution and avoid duplicate work.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
