How networking became the critical layer of AI

Investment firm Citi analysts believe that data transmission, not raw computing power, is now the primary limit on artificial intelligence (AI) performance as AI systems become larger and more intricate. This shift suggests a favorable outlook for key players such as Nvidia, Broadcom, Arista Networks, Lumentum, Coherent, Marvell Technology, and Astera Labs, according to Citi.

Initially, the development of AI infrastructure primarily emphasized faster graphics processing units (GPUs), larger accelerators, and increased training capacity. However, as AI systems progress toward trillions of parameters, broader context windows, and autonomous agent workloads, the need for efficient data movement across chips, servers, racks, and data centers has intensified.

During the Hot Chips conference, presentations from industry leaders like Nvidia, Broadcom, Meta Platforms, Google, Samsung, and others indicated this trend. Citi analysts emphasized that the future of AI may not revolve around the company with the fastest processor, but rather the entity capable of moving data most efficiently throughout the entire system.

Memory is also increasingly becoming a critical aspect of the networking challenge. Companies such as Samsung, XCENA, and Cerebras are developing technologies to keep more data close to where it is processed, thereby reducing the amount of data that must traverse congested interconnects. Every byte preserved locally minimizes communication overhead, and improvements in memory capacity, bandwidth, and efficiency are directly tied to network performance.

Citi also pointed to Nvidia's vision of future data centers as "AI factories," which are increasingly designed around data flows connecting compute, memory, networking, storage, and security. This shift could gradually transfer more value in AI infrastructure to companies that can optimize the entire data movement stack, rather than merely providing standalone processors.