Networking for AI: why GPU clusters need a different network, and what Ultra Ethernet changes
AI training spreads one job across many GPUs, and the network between them decides how fast it finishes. That has turned data center networking into an AI buying decision.

Training a large AI model splits the work across many GPUs, which constantly exchange results with each other. If the network between them is slow or drops packets, expensive GPUs sit idle waiting for data. For AI clusters, the network is part of the compute.
Most mid-size companies will rent AI capacity from cloud providers rather than build clusters. The network still matters to them, because it shapes which providers can offer large GPU clusters, at what price, and how well on-premises AI servers perform.
What AI traffic needs
Traditional data center networks are designed for many small, independent flows, such as users reaching web servers. AI training produces the opposite: a few huge, synchronized flows between GPUs, where the slowest link holds up the whole job.
The network requirements that follow are high bandwidth per GPU, very low and predictable latency, no packet loss under heavy load, and congestion control that keeps all paths busy evenly.
InfiniBand and Ethernet
| InfiniBand | Ethernet for AI | |
|---|---|---|
| Origin | High-performance computing | General-purpose networking |
| Strength | Low latency and lossless behavior built in | Familiar to network teams, many vendors, existing tooling |
| Weakness | Fewer suppliers, separate skills | Needed extensions to match InfiniBand for AI traffic |
| Direct memory access | Native RDMA | RDMA over Converged Ethernet (RoCE), and now Ultra Ethernet transport |
The Ultra Ethernet specification
On June 11, 2025, the Ultra Ethernet Consortium, a Linux Foundation project, released its Specification 1.0. It defines a modern RDMA transport for Ethernet and IP aimed at low-latency, high-throughput AI and HPC traffic, and covers network cards, switches, optics and cables so equipment from different vendors can work together. The consortium says the design scales to millions of endpoints.
For buyers, the significance is choice. An open standard for AI networking over Ethernet means more vendors can compete for AI cluster networks, and teams can build on the Ethernet skills they already have.
What this means for a mid-size company
If you rent GPUs in the cloud, ask providers how their GPU clusters are networked and what bandwidth each GPU gets, especially for training jobs that span many machines. If you are buying on-premises AI servers, plan the network at the same time as the servers, and check whether the switches and network cards support RoCE or Ultra Ethernet.
What IT teams should do
- For cloud GPU contracts, ask about interconnect bandwidth per GPU and cluster size limits.
- For on-premises AI servers, budget for high-bandwidth switches and network cards alongside the GPUs.
- Keep AI cluster traffic on its own network segment, separate from general data center traffic.