From Megawatts to Tokens: How NVIDIA Maximizes AI Factory Production
On a sweltering August evening in Silicon Valley, as the sun dropped and air conditioning loads spiked, Silicon Valley Power sent a signal to an AI factory to adjust its power consumption. Varun Sivaram was watching on Zoom with about forty others — his team at Emerald AI in their San Francisco conference room, engineers […]
On a warm evening in Silicon Valley, as air conditioning demand rose, a signal from Silicon Valley Power reached an AI factory, prompting it to cut power consumption. Varun Sivaram watched the demonstration with colleagues, including engineers at the data center and representatives from the utility, all connected via a Zoom call.
The AI factory's Conductor platform, developed by NVIDIA partner Emerald AI, received grid conditions information and adjusted the data center's computing workloads accordingly. Lower-priority tasks were slowed or delayed, while high-priority services continued running without interruption. The team cheered as the electricity demand fell from four megawatts to three, with no impact on critical AI workloads.
This successful pilot, orchestrated by NVIDIA's DSX Flex grid-orchestration platform, demonstrates how AI factories can optimize power consumption without compromising performance. The technology enables data center operators to maximize compute density within a fixed power budget, potentially increasing token throughput by up to 24%.
NVIDIA's DSX MaxLPS software monitors GPU and rack-level power consumption, reallocating headroom across nodes based on workload type to recover stranded capacity. By implementing DSX MaxLPS, Lambda, a GPU cloud provider serving over 10,000 customers, achieved 24% more cluster-wide token throughput in a five-rack, 19-node cluster compared to running at full power.
The AI Infra Summit highlighted the importance of AI factory efficiency, with NVIDIA's Ian Buck emphasizing that a fixed power budget can support more token throughput when managed intelligently. The DSX suite of technologies, including DSX Flex, DSX OS, DSX Sim, and DSX Reference Designs, aims to optimize AI factory throughput per megawatt and pave the way for reclaiming stranded capacity within the same footprint.
As the AI factory economy continues to grow, power remains a crucial constraint, and innovative solutions like DSX are essential for unlocking the full potential of these facilities.
Written by urgent.news from NVIDIA Blog's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
