'AI success will be defined not by how much infrastructure organizations own, but by how productively they use it': Nvidia lays out its thoughts on how storage has become the next frontier of AI

Nvidia has made significant progress in the storage industry by open-sourcing its cuFile APIs and storage stack, and launching an industry initiative called Storage-Next, which includes over 40 flash and storage vendors. The company's key unveiling is its SCADA framework, which moves the storage control path onto the GPU, allowing multiple GPUs to directly pull data from storage without the need for a CPU to broker every request.

This framework is a game-changer because it allows GPUs to pull data directly from storage in small chunks of data, rather than the traditional method of fetching larger amounts of data at once. Nvidia's focus on 512-byte IOPS rather than raw bandwidth is a strategic move, as inference performance heavily depends on the former. By making cuFile open-source, Nvidia is encouraging other vendors to build their systems around it, which in turn benefits Nvidia's GPU sales.

The initiative has received significant support from Intel, Google, and Meta, who are all actively maintaining software designed to remove the CPU from the I/O path. Nvidia is targeting Gen7 SSDs that can sustain 100 million IOPS, which is a target that controller vendors are designing toward. Overall, Nvidia's approach to storage is a well-thought-out strategy to address a significant bottleneck in AI inference performance, and it is backed by credible partners.