Next-gen AI networks may hinge on the telephone switchboard's return

The next generation of AI networks may hinge on a return to the telephone switchboard. Nvidia and other tech companies are designing massive systems that cram hundreds or even thousands of accelerators into a single unit. However, to achieve this, they require optical technology that resembles old-fashioned telephone switchboards. This has sparked investment in photonics startups and established optical equipment and fiber manufacturers.

Nvidia recently invested $6 billion in Coherent, Lumentum, and Marvell to advance their optics technology. They also spent $125 million to support the development of iPronics' second-generation optical circuit switching (OCS) technology. Unlike traditional packet-switched fabrics, OCS appliances can switch light, acting like a modern telephone switchboard.

They are the equivalent of human operators manually patching together two telephone lines, but instead, they use microscopic mirrors, piezoelectric actuators, or LCDs to reconfigure the network in the blink of an eye.

Google has been using OCS in its TPU clusters for years, employing 2D and 3D torus topologies for communication between accelerators. This alternative to packet-switched fabrics allows for changes in pod size or hot-swapping of failed accelerators without reconfiguring the network. However, existing OCS appliances are not perfect; they use micro-electromechanical systems (MEMS) devices, leading to slow reconfiguration times of around 100 ms. Silicon photonics-based designs, like those from iPronics, aim to address these shortcomings by achieving sub-millisecond reconfiguration times and higher port densities.

OCS works best in environments with infrequent network path changes, unlike most modern rack systems that prioritize single-hop all-to-all connectivity. However, the two approaches can work together, such as using OCS for rack-to-rack communications and switched fabrics for intra-rack connections. The future of AI networks may rely on a dense fiber attach system, like Mixx Technologies' new SxC connector, capable of terminating tens of thousands of fibers to a single rack.