ScaleFlux bets its controller chips can break AI’s memory bottleneck

AI data centers have been investing in faster accelerators, denser racks and liquid cooling to maintain the performance of processors. However, the data pipeline feeding those processors has lagged behind, causing expensive compute to wait on memory and storage. This gap has become more critical as AI infrastructure bottlenecks have taken center stage in data center planning.

Hao Zhong, co-founder and chief executive officer of ScaleFlux Inc., believes that rethinking the external memory and storage tiers behind the GPU is necessary to keep up with the data pipeline. Zhong stated that GPU and TPU performance has improved significantly over the past eight years, but the data pipeline has not kept pace, resulting in inefficiency.

He discussed AI memory bottlenecks, liquid cooling and the shift towards inference and agentic workloads during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. ScaleFlux unveiled new controllers built for the PCIe Gen 6 SSDs and Compute Express Link memory tiers last year. These controllers are designed to build the controller chips or little processors for the second and third stages of the data pipeline.

Zhong emphasized that as AI applications transition from training to inference and agentic workloads, systems require more data close at hand, making capacity, bandwidth and performance in external DRAM and SSDs crucial for infrastructure decisions.