How Everpure plans to stop AI from starving without data

Everpure, a company focused on AI infrastructure, aims to address the issue of AI agents starving for data during operations. In order to efficiently deliver the necessary data to these agents, Everpure has developed an AI Data Platform built using NVIDIA's AI Data Platform reference design. This platform tackles three main bottlenecks, which are crucial in ensuring AI agents operate at optimal performance levels.

Firstly, Everpure's architecture focuses on solving the issue of throughput starvation at scale. To keep thousands of GPUs saturated, a storage system must have the capability to scale out its bandwidth capacity. The Everpure FlashBlade//S and FlashBlade//EXA platforms are designed to handle this high-performance scale, with the latter capable of supporting up to 10,000 GPUs.

These platforms utilize NVMe fabric-connected flash for data nodes, delivering 400 to 450 million IOPS, 220 GB/sec bandwidth, and 4.6 billion metadata operations per second.

Secondly, Everpure addresses the KV cache prefill tax. This tax occurs when a GPU's High-Bandwidth Memory (HBM) is used to store pre-computed tokens from data chunks, such as a business' SEC 10-K report. If multiple users request to process the same document with different GPUs, the KV cache needs to be recomputed, resulting in wasted GPU cycles.

Everpure's Key Value Accelerator (KVA) offloads cached token states directly to shared flash via NVIDIA GPUDirect Storage (GDS) using Remote Direct Memory Access (RDMA). This bypasses the host CPU overhead and eliminates the compute penalty, effectively saving computational resources.

Lastly, Everpure's AI Data Platform eliminates irrelevant data silos and maximizes model training by providing a uniform mechanism, known as the Data Stream. This platform ingests, curates, and reuses data from various sources within an enterprise's estate, without the need for data copying at scale. By doing so, it ensures that data remains current and valid, even as it is changed at the source.

This approach is essential in today's AI-driven world, where physical proximity plays a significant role in performance. The closer the data is to the CPU or GPU, the less time it takes for the AI agent to access and utilize the information, making data speed crucial for efficient AI.