Kubernetes gets a second act as AI agents turn one prompt into thousands of inference calls

As AI systems evolve into agentic agents, Kubernetes, traditionally known for its reliability and predictability, is gaining a new purpose. With AI spending shifting from model training to serving and running agents, the open-source community is developing tools to manage the increased volume of inference requests. Jonathan Bryce, executive director of the Cloud Native Computing Foundation, highlighted the importance of Kubernetes, built from the start to handle dynamic, distributed workloads, as a solid foundation for fast-growing AI companies.

At KubeCon + CloudNativeCon NA, Bryce discussed the CNCF's efforts to meet rising inference demand, optimize hardware usage, and enhance AI agent security during an exclusive broadcast on theCUBE. As models' performance relies on efficient serving, the CNCF has introduced a new AI inference and agentic track at KubeCon to tackle this challenge.

By using coding agents to spawn numerous sub-agents, which can perform thousands of inference calls, the CNCF aims to boost performance and address security concerns. OpenShell, a security project under CNCF's neutral governance, has attracted 300 contributing companies within months, demonstrating the speed of collaboration in the open-source community.

Bryce emphasized the need to lower collaboration friction and foster open solutions, enabling everyone to build on shared infrastructure components.