Next-gen AI networks may hinge on the telephone switchboard's return

As AI networks advance, they may return to the telephone switchboard for guidance. Nvidia, Coherent, Lumentum, and Marvell have all invested heavily in optical technology to support the next generation of AI networks. Optical circuit switching (OCS) is the key technology behind this shift, functioning like a modern telephone switchboard.

OCS allows for rapid reconfiguration of networks without the need for traditional packet switching, making it ideal for AI clusters. Google has been using OCS in its TPU clusters for years, enabling flexible pod sizes and hot-swappable accelerators. However, current OCS appliances are not without their drawbacks, with reconfiguration times around 100 ms and the use of micro-electromechanical systems (MEMS) that introduce latency.

To address these issues, startup iPronics is developing a second-generation OCS using silicon photonics with no moving parts. This technology promises sub-millisecond reconfiguration times and higher port densities, potentially enabling changes to massive AI clusters on the fly. For example, iPronics aims to pack up to 720 port pairs into a single rack unit.

OCS also complements other AI network architectures, such as Nvidia's all-to-all NVLink Switch chips connected in an all-to-all network. In the future, OCS may even help manage the massive LPX racks used by Nvidia, allowing for resizing clusters and improving efficiency in parameter-heavy AI models.