AI data lakes are driving new storage demands

The rapid advancement of artificial intelligence (AI) is causing data center design and operational challenges to evolve. Hyperscale computing companies – those employing large-scale server farms – have invested significantly in GPU clusters and high-bandwidth memory to facilitate AI model training and inference operations. While these investments have led to remarkable performance improvements, they have also highlighted a critical design gap that data center architects are urgently addressing.

Today's foremost AI compute issue is not about speeding up processor cycles; it revolves around effectively serving diverse compute engines with data, in a manner that is efficient, consistent, and scalable.

AI workloads are transforming data center roles for data. Previously, data was accessed only intermittently or in predictable batches. Now, data is continuously consumed, recalled, and recombined across training, inference, and complex agentic workflows. Consequently, data lakes and other storage systems, which previously stored raw or semi-structured data and conducted minimal offline analysis, are now required to function as high-throughput delivery platforms.

Data center managers responsible for budgets, infrastructure planning, and long-term strategic direction face new trade-offs due to this shift. In the past, data lakes served as centralized, cost-effective repositories for storing vast amounts of raw data – ranging from text, images, video content, to physical sensor outputs. However, AI has altered the scope and behavior of data lake models.

AI training workloads repeatedly access large datasets distributed across clusters, while AI inference introduces a higher demand for data access events. What may seem like a straightforward request can trigger numerous underlying operations, each necessitating access to stored data, intermediate results, or contextual information.

Thus, AI data lakes are not merely storage repositories; they actively deliver output continuously and at high speed by enabling concurrent access from multiple pipelines and managing rapid data ingestion under sustained loads.

In essence, AI data lakes have shifted from being passive storage vaults to becoming active, persistent, and increasingly central to overall system performance. Conventional storage approaches, however, are struggling to meet the demands of AI inference workloads. Traditional data lakes assumed that only a small fraction of stored data would be accessed at any given moment, tolerating latency and focusing on cost per terabyte rather than throughput per workload.

Such a model was effective for reporting and analytics but falls short under constant, parallel access patterns.

The issue is further compounded by the fact that GPU clusters are costly to build and operate. They consume substantial power and necessitate advanced cooling solutions. Simply augmenting compute resources or high-bandwidth memory is not always a viable solution, as power budgets, space limitations, and replacement cycles impose physical and economic constraints.

Moreover, this pressure is reshaping how cloud providers structure their storage services and service-level agreements (SLAs). Hyperscalers have long segmented customers into storage performance tiers, distinguished by latency, throughput, input/output operations per second (IOPS), and data availability guarantees. While lower-cost archival and capacity tiers prioritize cost and scalability, premium tiers increasingly rely on all-flash infrastructure to deliver faster response times and consistent performance.

AI is accelerating this transition as inference workloads increasingly require near-continuous access to massive datasets, pushing customers toward higher-performance storage tiers capable of sustaining parallel data retrieval without bottlenecks.

To address these challenges, a new approach to storage is needed – one that balances capacity, performance, and efficiency at scale. Quad-level cell (QLC) NAND flash offers a solution by increasing storage density by encoding four bits per cell, enabling significantly higher capacities than other NAND configurations. Recent advancements have even led to the demonstration of 256TB NVMe drives specifically designed for AI workloads.

These ultra-high-capacity SSDs provide the necessary scale and sustained throughput critical for AI data lakes, where both scale and rapid data access are paramount. By maximizing capacity per device, these SSDs help data centers reduce their physical storage footprint while improving power efficiency, measured in terabytes per watt.

This efficiency is crucial for hyperscale environments, where energy consumption is a primary constraint. The shift towards QLC NAND flash and ultra-high-capacity SSDs reflects a broader trend in flash design, where the focus is moving from serving as a general-purpose storage medium to addressing the specific demands of AI, particularly sustained access patterns and the management of large datasets.