Google Cloud introduces Agent Compute Engine to host AI agents

Google Cloud has unveiled Agent Compute Engine, a specialized infrastructure service crafted to host artificial intelligence agents. These AI agents operate within isolated environments called sandboxes, where they store necessary applications, data, and user instructions. Google refers to these sandboxes as microVMs, a combination of micro (small) and virtual machines (VMs), which leverage the resource efficiency of containers while maintaining the security of traditional virtualization.

The company's implementation of microVMs incorporates hypervisors' capability to isolate virtual machines, ensuring that a breach in one sandbox doesn't spread to others, thwarting the spread of malware. These microVMs are hosted on specially designed appliances that are stateless, meaning they don’t retain any files permanently. Instead, any data is stored in an immutable file system that can't be altered or deleted, safeguarding the information stored within.

Google verifies the integrity of these appliances using its proprietary Titan cybersecurity chips, which also mitigate side-channel attacks—cyberattacks that exploit a CPU's physical state to access customer data. Within each microVM, access to storage resources is restricted to prevent unauthorized data access or deletion. Network traffic connecting the appliance to the host data center is further protected by multiple proxies, ensuring that any inbound or outbound traffic must traverse an isolated, point-to-point IPC boundary, which in turn forwards packets to an external proxy for validation and strict policy enforcement.

One of the key features of Agent Compute Engine is its ability to detect idle sandboxes, which consume significant hardware resources and increase costs. The system automatically shuts down these idle sandboxes and swiftly reactivates them when needed. This efficiency allows a single physical node to host numerous living agent microVMs, far more than conventional virtual machines.

Additionally, Agent Compute Engine manages sensitive agent credentials outside of the sandboxes, preventing hackers from stealing such data even if they manage to take over a sandbox. When an agent requires access to an application, a service called Agent Gateway retrieves the necessary credentials and submits them on behalf of the agent.

Google plans to introduce Agent Compute Engine in the coming months, alongside an updated version of Agent Gateway. This new version will include model routing capability, optimizing the distribution of agent requests to the most efficient processing models. Additionally, Agent Gateway will feature tools for regulating inference costs through token management, allowing for the tracking of token consumption, enforcing budgetary limits, and providing comprehensive metrics for token usage.