Google Cloud takes aim at the cost of keeping idle AI agents running on Kubernetes
Kubernetes has become the go-to platform for distributed training and inference, but what happens to the compute bill when it runs AI agents that spend most of their time waiting? When an agent’s container or virtual machine stays provisioned, customers pay for the wait as well as the work. To help customers control those costs […] The post Google Cloud takes aim at the cost of keeping idle AI…
Kubernetes is a popular choice for distributed training and inference, but it incurs costs when AI agents spend most of their time waiting. To address this, Google LLC has donated three projects to the Cloud Native Computing Foundation. Allan Naim, Google Cloud's director of product management for AI infrastructure, explained that Kubernetes has become the go-to platform for inference and training, but a new era, Act 3, is emerging with different patterns.
Google Cloud uses open-source tools to reduce compute costs during idle agent periods, which can be up to 90% to 95% of the time. They developed Agent Substrate, an open-source runtime that hosts AI agents in isolated environments on Kubernetes. When agents are suspended, their memory and local files are saved, freeing up compute for other agents.
When work resumes, the agents are restored from the snapshot. Google also open-sourced gVisor to prevent agents from making security compromises, and they are collaborating with Anyscale Inc. on KubeRay, which runs Anyscale's Ray platform natively on Kubernetes for distributed training and inference. OpenRL further extends this approach to reinforcement learning, making it accessible to platform engineers for continuous improvement of agentic workloads.
Written by urgent.news from SiliconANGLE's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
