Why AI needs a new kind of supercomputer network — the OpenAI Podcast Ep. 18
OpenAI's Mark Handley and Greg Steinbrecher detail Multipath Reliable Connection (MRC), a new networking protocol designed to overcome the unique challenges of large-scale AI model training. They explain how moving intelligence to the network's edge creates a resilient, efficient, and simple system that handles constant hardware failures without disrupting massive, synchronized GPU workloads.
The Unique Challenge of AI Workloads
Training large-scale AI models has fundamentally upended traditional data center and networking design. Unlike conventional internet traffic, which consists of many separate, independent conversations that benefit from the law of large numbers, AI training involves a massive number of GPUs working in lockstep on a single, synchronized task. In this environment, communication between GPUs is an integral part of the computation itself.
This synchronous nature creates a “worst-case” performance scenario. The entire system’s speed is dictated by the single slowest link or component. If one GPU is slow, thousands of others must wait. This means performance is not governed by averages, but by the “100th percentile” or tail latency. As Greg Steinbrecher notes, “It is not about how fast can the average pair of GPUs talk to each other. It’s always what is the absolute worst case that occurs.” This makes AI training just about the worst possible workload for a traditional network.
The Inevitability of Failure at Scale
As AI systems scale to hundreds of thousands of GPUs, the number of network components—switches, cables, and optical transceivers—grows by orders of magnitude, reaching millions of individual parts within a single building. With this scale, component failures are not a rare event, but a constant reality. As Steinbrecher puts it, “If you double the size of your system, you’re going to have half the time between failures.”
In a traditional network, a single link failure can cause a significant disruption. The network must reconverge using routing protocols like BGP, a process where switches “gossip” to update each other about the new network topology. This can take seconds, during which the entire training job may stall or even crash. These frequent, small delays accumulate, wasting immense amounts of computational resources and slowing down research.
Multipath Reliable Connection (MRC): A New Paradigm
To address these challenges, OpenAI, in collaboration with partners like Microsoft, AMD, Broadcom, Intel, and Nvidia, developed Multipath Reliable Connection (MRC). The core philosophy of MRC is to move intelligence from the complex network core to the endpoints (the network adapters on the servers).
MRC is built on three key techniques:
-
Packet Spraying: To avoid network hotspots, traffic from a single source to a destination is “sprayed” across many of the thousands of potential paths through the network. This ensures traffic is balanced evenly, leaving congestion to occur only if multiple sources send to the same destination simultaneously.
-
Packet Trimming: When congestion does occur and a switch’s queue overflows, traditional networks drop the packet, creating ambiguity for the sender—was the packet lost or just reordered and delayed? MRC uses a technique called “packet trimming.” Instead of dropping the packet, the switch trims off its payload but forwards the small header. The destination receives the header, immediately knows the packet was not lost but was intentionally trimmed, and can request a retransmission without ambiguity. This drastically reduces recovery time.
-
Decentralized Fault Tolerance: MRC’s most significant innovation is how it handles failures. Instead of relying on a slow, centralized routing protocol to announce a dead link, each endpoint independently and rapidly detects when a path is no longer working. Within milliseconds, it simply stops sending packets down that path. This makes the network “self-annealing.” The system dynamically routes around failures at the edge without needing coordination, eliminating the seconds-long stalls associated with traditional network reconvergence.
The Transformative Benefits of MRC
The implementation of MRC has yielded profound benefits. The system is so resilient that during the construction of a new data center, with technicians constantly causing links to go up and down, training jobs continued without interruption. The researchers “didn’t even notice.”
This resilience allowed for a radical simplification of the network core. Since MRC handles failure detection and path selection at the edge, OpenAI was able to turn off complex, dynamic routing protocols in the switches entirely. The network now runs on completely static routing, where switches have a simple configuration at boot time and never change it. This removes a massive source of complexity and potential bugs from the network’s control plane.
The result is a more reliable and efficient system that accelerates the entire research and deployment pipeline. It allows for the construction of flatter, simpler network topologies that use less power and cost less, meaning more energy goes directly to the GPUs doing useful work.
An Open Standard for a Shared Future
Recognizing that these infrastructure challenges are shared across the industry, OpenAI is releasing MRC as an open standard through the Open Compute Project (OCP). By building on the open Ethernet standard and contributing back to the community, they aim to prevent a fractured supply chain and allow the entire industry to benefit from a common, robust foundation. As Mark Handley states, “What we’re trying to do is hard enough without everybody having to reinvent the wheel all the time.” This collaborative approach ensures that as AI continues to scale, the underlying infrastructure can keep pace.