d-Matrix drinks the Nvidia Kool-Aid with NVLink Fusion and MGX rack designs

AI infrastructure startup d-Matrix has announced it will integrate Nvidia's NVLink Fusion interconnect technology and MGX rack designs into its high-performance inference platform. By adopting these Nvidia technologies, d-Matrix aims to make its computational solutions more accessible to customers. The company expects to offer systems with up to 144 Raptor accelerators connected by a single all-to-all NVLink fabric by the end of next year.

Each Raptor "card" will feature 32 GB of ultra-fast 3D-stacked DRAM, delivering 100 TB/s of memory bandwidth. With 144 of these accelerators per rack, d-Matrix anticipates achieving a total memory capacity of 2.3 TB and peak aggregate memory bandwidth of 7.2 petabytes per second. This memory bandwidth advantage enables d-Matrix to handle models exceeding four trillion parameters in size, even at 4-bit precision.

By leveraging Nvidia's NVLink Fusion and MGX reference designs, d-Matrix sidesteps many challenges associated with scaling chip architecture across large compute clusters, and offers customers the same racks and NVSwitch fabrics as Nvidia.