Microsoft plans multi-gigawatt rollout of next-gen Azure Maia AI accelerator racks

Microsoft has unveiled plans to roll out a next-generation Azure Maia AI accelerator in a multi-gigawatt deployment, signaling a significant leap in its compute infrastructure. The company's Azure Maia 200, announced earlier this year, is already serving production workloads, with a successor expected in 2027. The most advanced model, Maia 300, is described as a complete intelligent system designed for emerging AI workloads, promising seven times the rack-scale token throughput of its predecessor, alongside 40 percent more tokens per megawatt than leading GPU-based systems.

Created with an inference-first architecture, Maia 300 is optimized for long-context, memory-intensive, and decode-heavy AI workloads. Each unit will feature eight HBM4 memory stacks, delivering up to 18 TB/s of peak memory bandwidth to support large model and long-context execution. Microsoft envisions deploying these rack-scale systems with up to 96 accelerators, aiming to meet the demands of cloud-scale agentic inference.

This system-level approach, integrating software, silicon, physical rack infrastructure, networking, and datacenters, is crucial for handling the unique computational profile and arithmetic intensity of agentic workloads. This move contrasts with Microsoft's 2024 AI Access Principles, which emphasized partnerships across its technology stack.

The company's shift comes after a heated exchange with the European Commission, which accused Microsoft of having a competitive advantage due to its vertical integration, something Microsoft denied. With its own AI chips, rack-scale designs, and internally developed MAI models, Microsoft now holds more control over its technology stack and plans to scale Azure Maia to multi-gigawatt deployments starting in 2027.