Nvidia’s journey to forge the one CPU SKU to rule them all
A deeper look at Vera
Nvidia's Ian Finder, who leads the development of the datacenter CPUs, asserts that most use cases don't require the multitude of SKUs offered by competitors from Intel and AMD. Nvidia's first datacenter CPU, Grace, launched in 2023, but saw limited deployment outside of AI headnodes, high-end NICs, and standalone CPU deployments by hyperscalers.
The company's latest CPU, Vera, is a different story. In addition to being part of Nvidia's Vera-Rubin superchips, it will also be offered in standalone CPU racks, storage, and context memory systems. A new rack with 192 Vera CPUs, aimed at HPC workloads, was announced recently.
Historically, CPUs have come in various SKUs, each tailored for specific use cases like high core count for throughput, high frequency for latency, or low power for edge computing. However, Finder argues that most datacenter CPUs aren't utilized to their full potential, leading to higher latency when cores are loaded. This is due to congestion on connectivity buses as more cores become active.
Nvidia aims to strike a balance in its architecture by designing a CPU that maintains low latencies and high throughput even when all 88 cores are loaded. This includes sacrifices, such as a less powerful vector unit, but is justified for workloads like knowledge agents for CI/CD, software engineering, databases, and analytics. These workloads can achieve the same performance per core as frequency-optimized x86 parts while offering the same throughput per watt as density-optimized parts.
Nvidia isn't the only company to take this approach. Cloud providers have long favored CPUs that offer a mix of throughput, latency, memory bandwidth, and capacity, rather than just focusing on high core counts or low latencies. Vera, being an Arm chip, offers a unique feature: simultaneous multithreading (SMT) that bifurcates the Olympus cores, effectively providing 176 narrower cores.
This allows for increased throughput if needed or greater flexibility depending on the workload. Additionally, Vera's split personality enables binning of defective cores and offers customers variations in clock speed, memory bandwidth per core, and power consumption.
Written by urgent.news from The Register's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
Other reports of this story
- Nvidia’s journey to forge the one CPU SKU to rule them all theregister.com