{
  "id": 15863402,
  "title": "InfiniBand Architecture First Principles: Physical Link Rates, Credit-Based Link Flow Control, and Subnet Manager Fabric Orchestration",
  "url": "https://urgent.news/2026/10/21/infiniband-architecture-first-principles-physical-link-rates-credit",
  "topic": "tech",
  "section": "Tech",
  "published": "2026-10-21T08:36:04.000Z",
  "source": {
    "name": "Dev.to",
    "slug": "dev-to",
    "url": "https://dev.to/ifnodoraemon/infiniband-architecture-first-principles-physical-link-rates-credit-based-link-flow-control-and-1p4a"
  },
  "original_language": "en",
  "account": "InfiniBand architecture represents the foundational principles of physical link rates, credit-based link flow control, and subnet manager fabric orchestration in supercomputing. Unlike Ethernet, which retrofits for RDMA with reactive PFC pause frames and DCQCN congestion tuning, InfiniBand is a purpose-built network designed for high-performance computing and large-scale AI.\n\nThe layered architecture consists of Upper Layer Protocols (ULP) including MPI and NCCL, a Transport Layer handling Reliable Connection (RC) and Unreliable Datagram (UD) with Hardware Checksum & Retransmit (BTH), a Network Layer for intra-subnet 16-bit Local Identifier (LID) routing and inter-subnet GID routing, a Link Layer providing Credit-Based Hardware Flow Control, Virtual Lanes (VL), and Nanosecond Cut-Through Switching, and a Physical Layer encompassing SerDes, Serial Links, PAM4 High-Frequency Modulation, and OSFP & QSFP-DD Transceivers.\n\nPhysical link speeds have evolved from Single Data Rate (SDR) at 2.5 Gbps to the 2026 flagship XDR at 800 Gbps, increasing aggregate bidirectional network throughput to 1600 Gbps per host. The Link Layer's credit-based hardware flow control model grants buffer credits to the receiver, allowing the sender to transmit only when the receiver has available buffer space, thereby preventing packet drops and eliminating deadlock hazards.\n\nCut-Through switching in InfiniBand ensures sub-100 nanosecond forwarding, a stark contrast to traditional Ethernet's Store-and-Forward mode, which introduces microseconds of latency by buffering entire packets and verifying frame CRC before forwarding. InfiniBand's packet header includes a Local Route Header (LRH) with a single 16-bit destination LID, enabling rapid forwarding decisions.",
  "summary": "Introduction: The Dedicated Arteries of Supercomputing — Native InfiniBand In Chapter 9 of our series, RDMA High-Performance Networking Foundations: Kernel Bypass, Zero-Copy, Queue Pairs, and Lossless RoCEv2 Architecture , we examined the trade-offs involved in retrofitting Ethernet for RDMA—relying on reactive PFC pause frames and DCQCN congestion tuning to manufacture a pseudo-lossless…",
  "key_points": [
    "InfiniBand designed for high-performance computing and AI, unlike Ethernet retrofitting RDMA",
    "Physical link speeds from SDR 2.5 Gbps to XDR 800 Gbps, 1600 Gbps aggregate throughput",
    "Credit-based hardware flow control prevents packet drops, eliminates deadlock hazards"
  ],
  "editors_take": "By prioritizing hardware-level flow control and cut-through switching, InfiniBand provides a purpose-built alternative to Ethernet that eliminates the latency and packet loss issues inherent in reactive congestion management.",
  "illustration": null,
  "coverage": {
    "outlets": 1,
    "also_reported_by": []
  },
  "ai_generated": true,
  "disclaimer": "Summaries, key points and the editor’s take are written by software from other outlets’ reporting and may contain errors — always check the linked original."
}