Nvidia open sources cuFile API, accelerating GPU read/write capability for high-speed storage

Nvidia has made its cuFile API application programming interface open source, boosting the speed of GPU read and write capabilities for high-speed storage. This move comes as AI applications demand faster data access. At the Future of Memory and Storage conference, Nvidia announced the initiative alongside Storage-Next, a large-scale industry effort involving technology leaders to optimize memory and storage solutions.

CuFile allows secure, millisecond-level data access from storage, operating in just milliseconds. Initially launched in July 2021, the API was part of the GPUDirect Storage software, providing direct, high-speed data transmission between local distributed storage and GPU memory. This bypasses the CPU and system main memory, reducing memory access delays.

CuFile's direct memory access, or DMA, moves data directly from storage devices like NVMe drives into GPU memory, eliminating the need for a middleman software layer. By cutting out this middleman, CuFile enables rapid GPU orchestration and data synchronization at rates that keep up with elite GPU speeds. Nvidia's Storage-Next initiative, involving 40 storage and flash memory vendors, aims to create a more direct, efficient connection between GPUs and data, ensuring accelerated computing resources remain productive and speeding up time to insights.

This collaboration helps AI inference run faster, providing massive datasets from retrieval-augmented generation and agentic AI with a low-latency "superhighway" between deep storage and the GPU. Nvidia's Chief Technology Officer, Sven Oehme, emphasized that Storage-Next's SCADA solution allows high-speed storage layers to operate securely, splitting direct access across different jobs to meet security requirements.