Kubernetes Gave AI Agents a Place to Run, But Not a Way to Recover

Kubernetes has provided a platform for AI agents to operate, but it lacks the capability to recover their work. While Kubernetes can restore the runtime environment, without execution history, an agent might repeat actions like payments or API calls. Sandboxes protect the runtime, but durable execution is necessary for the business's integrity.

AI agents require more than Kubernetes' traditional stateless model, as they operate for extended periods and make critical decisions. The cloud-native community has responded swiftly, focusing on addressing the challenges of agent runtime. Two open-source initiatives illustrate this trend: Kubernetes SIG Apps' Agent Sandbox and Google's Agent Substrate on GKE.

These provide secure isolation and the ability to snapshot agent state, enabling rapid recovery in under 500 milliseconds. However, they do not track the agent's completed work, which could lead to duplicating critical actions upon recovery. Durable execution, which records each step of an agent's work, is essential to address this issue.

This runtime records LLM calls, tool calls, decisions, inputs, and outputs, ensuring that completed steps are not executed again. This approach can save costs and provide a cryptographic record of an agent's activities, crucial for regulated industries. Combining sandboxes and durable execution creates a comprehensive solution for production AI agents, addressing both isolation and recovery requirements.