Micron Joins Race for Post-HBM Memory

Micron, a leading memory company, is reportedly exploring a new memory technology called 'Near-GPU NAND' to complement high bandwidth memory (HBM) in the rapidly growing artificial intelligence (AI) memory market. This shift towards a 'post-HBM' era is driven by the unveiling of advanced memory structures and tiering technologies by Samsung Electronics and SK hynix.

The Near-GPU NAND technology involves placing high-endurance NAND flash close to the graphics processing unit (GPU), closer than existing NAND flash, to act as a new memory layer between HBM and general storage devices. This approach aims to increase input/output (I/O) speed and bandwidth, rather than reducing storage density, allowing for hundreds of gigabytes of NAND storage space around the GPU to process data required for large-scale large language model (LLM) inference more efficiently than existing storage devices.

This development is part of a broader trend in the AI memory market, where companies are seeking to optimize data storage and processing by integrating different memory technologies with varying characteristics. Samsung Electronics and SK hynix have also introduced their own solutions, with Samsung unveiling 'zHBM', a technology that reduces data movement time and power consumption through a three-dimensional (3D) structure, and SK hynix presenting 'tiered memory', a strategy that divides memory into several stages with high-speed data transmission close to HBM and high-capacity memory layers between HBM and solid-state drives (SSDs).

The goal of these innovations is to address the increasing demand for memory in AI applications, particularly in agentic artificial intelligence systems that require both speed and capacity to retrieve past data and external information in real-time. As the industry anticipates a future where the competition in AI memory will expand beyond simply increasing capacity and bandwidth, the introduction of Near-GPU NAND and tiered memory strategies represents a significant step towards a more efficient and effective AI memory landscape.