When agents build, deploy, and maintain, persistence becomes the hard problem

In the world of application development, agents are increasingly taking on the roles of creating, deploying, and maintaining these applications. This shift presents a new challenge: how to keep the state of these applications alive without incurring high costs. The key to solving this problem lies in separating durable state from ephemeral compute.

Traditionally, each tenant of an application has been provided with its own database instance, which works well until the number of tenants becomes large. This approach couples the logical experience of an isolated database with the physical reality of dedicated, always-on compute. It leads to a cost structure where the expense grows with the number of tenants created, not the number of active tenants.

Kimi, an AI platform, offers a solution to this issue. Instead of providing each tenant with its own database instance, Kimi uses a virtual layer that provides each tenant with its own namespace and isolation guarantees over a shared substrate. This approach uses object storage for durable data and a routing layer to send requests where they belong. By decoupling state from compute, Kimi can keep the cost of state alive regardless of whether it is being used or not.

However, this solution isn't without its challenges. When a request arrives for an idle tenant, a database needs to be spun up, which could add latency. Kimi addresses this by provisioning a database in about a second from a warm pool of pre-initialized resources. This ensures that the latency problem does not become a cost problem.

In essence, the solution to the hard problem of persistence for agent-driven products lies in separating durable state from ephemeral compute. By doing so, the cost of keeping state alive becomes independent of the number of tenants created, making it a more cost-effective solution even when tens of millions of applications are running.