Next-gen AI networks may hinge on the telephone switchboard's return

Next-generation AI networks may hinge on a return to the familiar telephone switchboard. As GPU designs become increasingly dense, with hundreds or even thousands of accelerators in a single system, optics and technologies reminiscent of the old-fashioned telephone switchboard become crucial. This has sparked investment in photonics startups and established optical equipment and fiber manufacturers.

Nvidia invested $6 billion in Coherent, Lumentum, and Marvell to advance optics technology, and recently, another $125 million to support the development of iPronics' second-generation OCS technology.

Optical circuit switches (OCS) are, in essence, switches that physically reconfigure networks by manipulating light, unlike traditional packet switches. They function similarly to a telephone switchboard, where high-speed actuators manipulate microscopic mirrors, piezoelectric actuators, or LCDs to reconfigure the network in a fraction of a second.

OCS isn't prevalent in modern GPU deployments, but Google has used them in its TPU clusters for years. Google's TPU pods employ 2D and 3D torus topologies, which are less rigid than mesh networks but can lead to increased latency. Optical circuit switching enables Google to dynamically adjust cluster sizes and virtually hot-swap failed accelerators.

However, existing OCS appliances have limitations. MEMS-based devices, used in many OCS appliances, have slow reconfiguration times (around 100 ms) and are bulky due to the required mechanisms and connectors. To address these issues, startup iPronics is developing a second-generation OCS that uses silicon photonics with no moving parts.

This technology promises sub-ms reconfiguration times and higher port densities (up to 72 or 144 ports per chip). In a single rack unit, iPronics claims to be able to pack up to 720 port pairs, which is significantly more than existing high radix OCS switches.