Nvidia unveils custom high-bandwidth memory promising higher bandwidth and lower power use - but who will actually get to use it?

Nvidia has introduced a new custom high-bandwidth memory solution called NVHBM, which promises enhanced memory bandwidth and reduced power consumption compared to standard HBM4E. The key difference lies in the placement of the memory controller, which is now integrated into the HBM stack's base die, rather than being separate from the accelerator die. This change is said to result in up to 30% more bandwidth, 15% lower power consumption, and 25% more usable area on the accelerator die compared to HBM4E.

Currently, access to NVHBM is restricted to Nvidia's NVLink Fusion program, with Amazon's Annapurna Labs being the sole named partner so far. The company has not disclosed which GPU utilizes this memory. NVHBM is expected to become available in volume sometime in 2027, although standard HBM4E is still in the sampling stage with a launch anticipated in 2027.

Nvidia's approach does not invent a new memory handling method, but rather reimagines where the memory controller is managed. Traditional HBM solutions split the responsibility between the memory vendor and the accelerator designer, resulting in a wide parallel bus connection. NVHBM eliminates this separation by moving the memory controller onto the base die of the 3D stack, replacing the standard bus with a narrower, serialized die-to-die link designed by Nvidia.

While the underlying technology is not entirely new, as suggested by Marvell's December 2024 announcement involving Micron, Samsung, and SK hynix, NVHBM's distribution and access are unique. Access to this custom solution is limited to Nvidia's NVLink Fusion program partners, with Amazon's Annapurna Labs being the first named participant. Currently, NVHBM may represent the future of memory solutions, but it may arrive after HBM4E has become more widely deployed.