Cisco, Nvidia team up to build fatter, faster pluggable

Cisco and Nvidia have joined forces to develop a new pluggable optic that promises to deliver significantly higher bandwidth in a smaller form factor. The ultra-dense small form factor pluggable (USFP) is expected to pack up to 32, 400 Gbps lanes into a single port, delivering up to 12.8 Tbps of Ethernet bandwidth. This represents a massive upgrade from the current maximum of around 1.6 Tbps in pluggable optics.

The new USFP modules will be larger than existing OSFP modules but will still be compact, roughly the size of a high-end pluggable optic. The increased number of lanes per pluggable will require a two-phase insertion mechanism, with a first phase inserting the module into the cage and a second phase making the electrical connection.

This higher density of connectivity could greatly simplify cabling in AI datacenters, where each GPU is often paired with an 800 Gbps or soon 1.6 Tbps NIC. By consolidating multiple GPUs' cabling into a single fiber optic bundle, a single USFP pluggable could reduce the complexity of cabling in these environments. While higher power consumption is expected, with up to 100 watts per pluggable, the USFP's larger blast radius could actually be beneficial compared to co-packed or near-package optics where a single failure could affect an entire rack of accelerators.

Nvidia's involvement in the USFP development is significant, as the company has largely moved away from pluggables in its recent switches, opting for co-packaged optics instead. However, Nvidia's NICs still rely on pluggables, likely due to the flexibility it offers to integrators. Despite the progress in USFP development, the standard will remain open, allowing any designer to incorporate it into their systems.

Microsoft is also moving towards using USFP, as evidenced by its adoption of Cisco's Silicon One P200 routing chips for its multi-datacenter AI infrastructure. Cisco is actively supporting SONiC, the hyperscale network operating system, to leverage its own ASIC capabilities and offer integration with its services.