Neoclouds reshape traditional architectures to meet AI demands

The neoclouds are rapidly reshaping enterprise infrastructure to meet the demands of AI models and agents. This shift is driven by the need for compute power and performance, particularly during the inference stage. Junxia Zhou, a senior product manager at Super Micro Computer Inc., explained that inference is highly sensitive to latency, leading neocloud providers to deploy AI closer to end users to reduce latency.

Phil Manez, vice president of strategic initiatives at Vast Data Inc., noted that agentic AI is moving into production, and inference has become the dominant workload. Agentic AI's production move and inference's dominance necessitate purpose-built, AI-focused infrastructure, as stated by Anders Graham, senior director of SSD marketing and business development at Kioxia Holdings Corp. Crusoe Inc., another neocloud, collaborates with Vast to provide a high-performance, low-latency storage platform with more flexibility than traditional clouds.

The demand for flash production and enterprise SSDs has surpassed mobile and client applications, highlighting the significance of neoclouds in AI infrastructure. Power availability remains a significant challenge, with liquid cooling becoming a preferred solution due to its power efficiency and improved rack density. As the collaboration between hardware companies, data platforms, and neoclouds evolves, the industry must continually reassess the optimal combination of compute, storage, and infrastructure elements to maximize value.