Every large language model you interact with, every AI-generated image and video, ultimately runs on physical hardware: racks of GPU servers, switching fabrics, optical transceivers, and kilometers of carefully routed cable. While most conversations about AI focus on models and algorithms, the reality is that the physical infrastructure of the AI data center is often the decisive factor in how fast a cluster can train, how reliable it is, and how much it costs to operate.
This article walks through the four pillars of AI data center infrastructure — servers, switches, optical modules, and cabling — and explains how they fit together into a working compute fabric.
1. AI Compute Servers: The Engines of the Cluster
An AI training cluster is built from high-density GPU servers. Unlike general-purpose web servers, an AI server is designed around one goal: delivering maximum compute throughput per rack while staying within strict power and thermal envelopes.
What makes an AI server different
| Component | Traditional Server | AI GPU Server | |---|---|---| | Compute | 2 CPUs, 8–16 cores | 2 CPUs + 8 GPUs/accelerators (PCIe or SXM modules) | | Typical power draw | 300–500 W | 6–12 kW per server (up to 100 kW+ per rack in newest designs) | | Network interfaces | 2× 10/25 GbE | 8× 400G/800G per GPU (one NIC per accelerator) | | Cooling | Air | Air for current gen; direct liquid cooling (DLC) for next gen | | Interconnect | PCIe | High-bandwidth GPU interconnect (NVLink-class fabric) |
Key design considerations
- Compute density. Accelerators are typically deployed in "8-GPU" server nodes, with high-bandwidth interconnects linking all GPUs inside the node into a single logical compute unit.
- Power envelope. A fully populated AI rack can draw 40–130 kW. That drives everything else: electrical design, cooling capacity, and even floor-loading requirements.
- Cooling. Air cooling reaches its practical limit as accelerators pass roughly 700 W each. Direct-to-chip liquid cooling is rapidly becoming standard in new builds, which in turn changes the cabling and layout of the rack itself.
- Resilience. In a training job with thousands of GPUs, hardware failure is a statistical certainty. Redundant power supplies, hot-swappable drives, and out-of-band management ports are mandatory, not optional.
2. Network Switches: The Fabric That Makes Many GPUs Act as One
A single GPU server is fast. A cluster of thousands is only fast if the network keeps up. AI training generates enormous east-west (server-to-server) traffic: gradient synchronization, parameter updates, and dataset streaming, repeated billions of times per training run. If the network stalls, the expensive GPUs sit idle.
Spine-leaf topology
AI clusters almost universally use a two-tier spine-leaf architecture instead of the traditional three-tier (core-aggregation-access) design:
- Leaf (Tier 2) switches sit at the top of each rack (top-of-rack, or ToR). Every GPU server connects to leaf switches at 400G or 800G per link.
- Spine (Tier 1) switches interconnect all leaf switches, providing any-to-any reachability with exactly two hops between any two servers.
- This topology delivers predictable, uniform latency and non-blocking (or controlled oversubscription) bandwidth — critical for synchronized training workloads.
Rail-optimized designs
The most common AI fabric variant is rail-optimized: the eight GPUs in each server connect to eight separate "rails" of leaf switches, so all GPU #1s in the cluster share one set of leaf switches, all GPU #2s share another, and so on. This concentrates traffic patterns, simplifies failure domains, and lets collective communication libraries choose the shortest path for each synchronization pattern.
Switch capabilities that matter
- Port density and speed. Current-generation data center switches offer 64 ports of 800G (51.2 Tbps of total throughput), with 1.6T-port platforms arriving next.
- Buffering. Large, well-partitioned packet buffers absorb micro-bursts from synchronized all-reduce operations — a leading cause of transient congestion in AI fabrics.
- Congestion control. Modern fabrics use enhanced congestion-notification mechanisms to react in hardware to hot spots, rather than dropping packets and waiting for TCP-style retransmits.
- Lossless transport. AI networks typically run an enhanced transmission protocol with priority flow control, providing Ethernet with RDMA-class lossless behavior for GPU-direct memory access.
3. Optical Modules: The Translators Between Silicon and Fiber
Inside the rack, signals travel over copper. Between racks, over any real distance, they travel as light through fiber. The optical transceiver module is the small pluggable device that performs this electrical-to-optical conversion at every switch and NIC port — and in an AI data center there are a lot of them: a 2,000-GPU cluster can easily require 5,000+ transceivers.
Common form factors
| Form Factor | Typical Speed | Where You'll Find It | |---|---|---| | QSFP56 / QSFP-DD | 200G–400G | ToR switch ports, server NICs | | QSFP-DD / OSFP | 400G–800G | Leaf and spine switches in AI fabrics | | OSFP / OSFP-XD | 800G–1.6T | Next-generation spine platforms | | SFP-DD / SFP56 | 25G–100G | Management networks, storage, out-of-band |
Reach classes (the naming you'll see on datasheets)
- SR / SR4 / SR8 — short reach over multimode fiber (OM3/OM4/OM5), typically up to 70–100 m. Cost-effective within a row of racks.
- DR / DR4 / DR8 — 500 m over single-mode fiber with parallel fiber pairs.
- FR / FR4 / FR8 — 2 km over single-mode fiber using wavelength division multiplexing to conserve fiber strands.
- LR / LR4 / LR8 — 10 km single-mode, used to connect buildings or data halls.
- ZR — 40–120 km for campus or metro-scale connectivity.
Active optical cables (AOC) vs. direct attach copper (DAC)
- DAC (Direct Attach Copper): twinax copper cables with integrated transceiver connectors. Cheapest and lowest power (roughly 2–3 W per end at 800G is achievable with active copper), but limited to about 1–2.5 m reach. Ideal for GPU-to-ToR-switch links within a rack.
- AOC (Active Optical Cable): fiber with transceivers permanently attached at both ends. Cheaper than pluggable modules plus separate fiber, lighter, reaches 3–30 m — the common choice for connecting ToR to spine switches across adjacent racks.
- Pluggable modules + patch fiber: the most flexible option, and the only practical one beyond 100 m.
The operational reality
Optical modules are now among the highest-value line items in an AI data center build — collectively comparable in cost to the switching silicon itself. They are also a leading source of link failures, so mature operators insist on Digital Optical Monitoring (DOM) support to track transceiver temperature, optical power, and error counters, and treat module firmware as managed infrastructure rather than anonymous commodities.
4. Cabling: The Unglamorous Component That Decides Everything
Cabling is the least glamorous part of the data center and one of the most consequential. In AI fabrics, the sheer volume of links — every GPU needs its own high-speed connection — creates cable counts that overwhelm traditional practices. A single 8-GPU server with 800G ports uses eight DAC cables, each of which has a thick, stiff, minimum-bend-radius twinax bundle. Multiply by hundreds of servers per data hall and cable management stops being aesthetics and becomes physics.
Structured cabling in AI halls
Modern AI data centers converge on a structured cabling model:
- Patch panels at the racks. Equipment-side cables (DAC/AOC) terminate on patch panels rather than running directly between devices, decoupling equipment changes from the fixed plant.
- Pre-terminated trunk cables. Factory-terminated multi-fiber trunks (typically 12–144 fibers) run from rack-adjacent panels to spine-side Main Cross Connects (MDC/HDU panels), installed and tested as fixed infrastructure.
- Modular patch cords. Only short, flexible patch cords cross-connect at the panels, making moves, adds, and changes fast and low-risk.
Design rules that hold up at AI scale
- Respect the bend radius. High-speed twinax and fiber both degrade with tight bends — for fiber, permanent macro-bends cause attenuation and, eventually, link flaps. Plan tray fill at roughly 50–60% to leave routing room.
- Separate power and data. Maintain separation between high-power feeder runs and high-speed signal cable to avoid electromagnetic interference on copper segments.
- Color-code by function. Consistent jacket colors — by speed, by fabric tier, by tenant or pod — turn troubleshooting from archaeology into a glance.
- Weight and airflow matter. Fully loaded overhead trays can weigh hundreds of kilograms per meter of run, and underfloor bundles can starve air-cooled racks of airflow. Overhead routing on dedicated tray infrastructure is the default in AI halls.
- Label both ends, test every link. With 5,000+ links per cluster, automated labeling conventions (rack-row-rack-unit-port, mirrored at both ends) and automated optical time-domain reflectometer (OTDR)/insertion-loss testing are the difference between a 2-week and a 2-month commissioning.
A quick note on fiber plant
AI halls overwhelmingly deploy single-mode fiber (OS2) for the horizontal and backbone plant, because reach classes DR/FR/LR all require it and because single-mode future-proofs the plant for 800G and 1.6T upgrades. Multimode (OM4/OM5) survives only for short SR links where cost is the dominant factor. Fiber counts are typically deployed at 2–3× current need, since adding fiber later means disturbing a live, fully-loaded cable plant — which nobody wants to do.
5. Putting It All Together
The four pillars are not independent; they are one system:
- GPU servers define the port count, power, and heat that everything downstream must serve.
- Switches define the topology and the oversubscription ratio that determines effective cluster throughput.
- Optical modules define the cost curve and the failure domain of every link that leaves the rack.
- Cabling defines whether the first three can be installed, diagnosed, and upgraded without tearing the data hall apart.
A useful planning sequence for a new AI build:
- Start from the accelerator count and training job profile → derive east-west bandwidth and oversubscription target.
- Choose the fabric (rail-optimized leaf-spine is the default) → derive leaf/spine port counts.
- Map each link to a media type by distance: DAC in-rack, AOC for adjacent racks, pluggable optics + structured fiber beyond.
- Size the structured cabling plant at 2–3× current port counts, and standardize transceiver monitoring from day one.
Conclusion
The gap between a great AI cluster and a mediocre one is rarely the model or even the accelerators — it is the discipline of the physical layer. Servers, switches, optical modules, and cabling succeed or fail together. Operators who treat the optical and cabling plant as first-class infrastructure — specified, tested, labeled, and monitored with the same rigor as the compute itself — are the ones whose thousand-GPU clusters deliver the training throughput the datasheets promised.
This article is a general overview of AI data center infrastructure categories and design practices, not a specification for any particular vendor's products.