Meta enlists tiny Korean startup to build 'one-chip-like datacenter' — CXL architecture introduced by Facebook's parent company and Panmnesia can handle almost 1000 AI GPUs per domain

Meta has partnered with a small Korean startup, Panmnesia, to develop an innovative datacenter architecture capable of supporting nearly 1,000 AI GPUs within a single coherent environment. This architecture leverages Compute Express Link (CXL) to interconnect CPUs, accelerators, and memory across multiple racks, eliminating the need for traditional network links.

The CXL design aims to significantly enhance accelerator coordination, increasing the number of devices that can operate in tandem on a single CPU from two to sixteen. This advancement addresses the challenge of managing communication delays between racks, which often occur when Ethernet or InfiniBand networks handle connections between individual systems. By providing a shared coherence mechanism, CXL enables processors, accelerators, and memory to function within a unified resource environment.

Panmnesia's architecture employs a high-fan-out switch, a link acceleration unit, and a fabric controller to manage traffic within the system. These components are organized into trays, pods, and a fabric, echoing the design principles found in semiconductor chips. The company asserts that its fabric controller and link acceleration unit have undergone silicon validation, while the switch has already been fabricated. Pre-release silicon is currently being supplied as development progresses toward commercial products.

With this proposed arrangement, up to 960 accelerators can be placed within a coherence domain, resulting in an approximate capacity of 1,000 GPUs working collectively as a single system. Cross-rack access would also experience a substantial reduction in timing, potentially decreasing from microseconds to several hundred nanoseconds, which represents a roughly tenfold improvement.

Moreover, individual failed devices can be replaced without disrupting the entire server, thereby minimizing the amount of functional hardware removed during failures.