Unified AI Networking Architecture Bridges Data Center Gaps
AI clusters depend on two types of networking to function effectively for modern workloads. Scale-up networking connects accelerators inside a rack, while scale-out networking connects racks across the data center. A single AI job spans both, which is a challenge for many organizations that treat them as separate. Integrating Scale-Up and Scale-Out Infrastructure Networking startup Upscale…
Modern AI workloads rely on two distinct types of networking: scale-up connects accelerators within a rack, while scale-out links racks throughout the data center. However, many organizations manage these as separate systems, which presents challenges for integrating them seamlessly. To address this issue, AI networking startup Upscale has introduced Token Fabric, an architecture that unifies scale-up and scale-out networking.
This innovative solution combines Upscale's proprietary SkyFabriX scale-up silicon with Nvidia Spectrum-X Ethernet technology for the scale-out systems. A unified software layer, SkyOS, manages operations across the entire environment, providing a single management plane for both scale-up and scale-out domains.
UpScale's SkyFabriX silicon forms the foundation of the scale-up switch hardware, supporting both Ethernet for Scale-Up Networking and standard IP protocols. The design is adaptable to evolving standards like Ultra Accelerator Link, with current switch capacity reaching 115.2 terabits per second, with plans to extend to multiple petabits in the future.
The system includes specialized scale-up switch trays that accommodate various rack form factors, pooling GPUs and other accelerators into large scale-up domains to maximize resource utilization.
The scale-out systems within the architecture utilize Nvidia Spectrum-X silicon, operating at speeds ranging from 400G to 1.6T. Through SkyOS, they integrate with various accelerator clusters to ensure high-speed communication between different racks. SkyCMD provides the orchestration and observability layer for the entire stack, offering a unified view of network health and performance across complex hardware configurations.
Customers can adopt Token Fabric in various ways, from only the silicon or software to the full stack, catering to different market segments such as enterprise teams and smaller cloud providers with limited internal engineering resources. Token Fabric extends existing networking protocols instead of replacing them, enabling easier integration into existing data center environments while providing specialized features for AI workloads.
The architecture relies on Ethernet for Scale-Up Networking and standard Ethernet with RoCE for scale-out traffic, reducing latency and overhead. Upscale is actively involved in several open-source projects, including contributions to SONiC and extensions of the Switch Abstraction Interface, as well as working with the Ultra Ethernet Consortium and UALink groups to help shape the future of industry standards.
By focusing on abstraction and open standards, Upscale aims to create a flexible, secure, and standardized AI infrastructure that can be easily managed across heterogeneous clusters containing different types of hardware.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
