2026-09-18
How DPUs Are Used in AI Computing: Networking, Storage and Security Offload
A DPU, or Data Processing Unit, is a programmable infrastructure processor placed close to the server I/O path. In an AI computing cluster, it can take selected networking, storage, security and infrastructure-management work away from the host CPU. The purpose is not to replace the GPU or the server CPU. It is to separate infrastructure services from the application and accelerator workload so that the system can be easier to scale, isolate and operate.
This is an educational article based on public vendor documentation. It is not a PORTOWER laboratory benchmark, and it does not claim that a particular DPU, GPU server or software combination has been tested by PORTOWER.
What is a DPU?
A DPU combines a high-speed network interface, programmable processing resources and hardware acceleration for infrastructure tasks. Depending on the platform, the device may also provide storage virtualization, encryption, packet processing, telemetry and a separate management environment. Industry documentation may use related terms such as IPU, SmartNIC or infrastructure accelerator, but the exact hardware and software functions vary by vendor and model.
| Component | Primary role | Typical boundary to verify |
|---|---|---|
| GPU | Parallel compute for training, inference or other accelerated workloads | Memory, interconnect, accelerator topology and software stack |
| Host CPU | Runs the operating system, applications and orchestration agents | Core allocation, memory, PCIe topology and virtualization model |
| DPU/IPU | Offloads selected network, storage, security and infrastructure services | Supported offloads, firmware, drivers, mode and control-plane ownership |
| Switch fabric | Connects servers, storage and service networks | Port map, speed, congestion behavior, routing and optics |
Why does a GPU cluster need infrastructure offload?
Distributed AI workloads can generate large volumes of east-west traffic between GPU servers. At the same time, the cluster needs storage access, container or virtual-machine networking, security policy enforcement, telemetry and lifecycle management. If all of those functions compete for host CPU cycles and memory bandwidth, the server design may become harder to predict as the cluster grows.
A DPU creates a second processing domain for selected infrastructure functions. The application domain remains responsible for the workload, while the infrastructure domain handles the services assigned to the DPU. The split is a design decision, not an automatic performance guarantee: every offload must be supported by the hardware, firmware, driver, operating system and orchestration stack.
Five important DPU applications in AI computing
1. Network and fabric offload
A DPU can accelerate or host parts of the network data path, such as virtual switching, routing, overlay processing, packet filtering and selected RDMA or RoCE functions. In an AI fabric, the key question is not simply the advertised port speed. The design team must confirm the NIC or DPU mode, link type, FEC, congestion-control policy, topology, firmware and software support.
For example, a GPU server may have one fabric for high-volume workload traffic and another for management or storage. The DPU can help keep infrastructure services visible and enforceable at the server boundary, but the exact number of ports, lanes and networks must be checked against the server configuration.
2. Storage and data movement
AI training and inference pipelines depend on fast access to datasets, checkpoints and intermediate results. Some DPU platforms support storage virtualization, NVMe over Fabrics, data integrity, encryption or direct data-movement paths. This can reduce the amount of storage protocol processing performed by the host CPU and create a more consistent boundary between the workload and the storage service.
The acceptance question is whether the complete path works: GPU or application, host memory, DPU, network fabric, storage target, filesystem or object layer and recovery process. A storage offload claim should therefore be evaluated with the intended block, file or object workflow rather than with a standalone card test.
3. Security and zero-trust infrastructure
A DPU can provide a hardware-assisted place to enforce segmentation, firewalling, encryption, secure boot, policy checks or telemetry. This is useful when the infrastructure operator needs to protect the host workload from the infrastructure-control plane, or when multiple tenants share a physical server.
Isolation is a system property. It depends on the DPU operating mode, boot chain, firmware trust, management access, host privileges, key management, policy deployment and monitoring. A product description should not turn a platform capability into a blanket statement that every deployment is zero trust.
4. Multi-tenant and bare-metal isolation
Cloud and hosted AI environments must separate tenant traffic, storage and management functions. A DPU can help place provider-controlled infrastructure services outside the tenant-controlled host domain. In bare-metal designs, it can also support a controlled provisioning and monitoring path while keeping tenant workload traffic separate.
Before selecting a DPU for this purpose, document who controls the host, who controls the DPU, which interfaces are exposed to the tenant, how resets are handled and which logs are retained. The operational model is as important as the silicon.
5. Telemetry and lifecycle management
Large GPU clusters need consistent visibility into link state, errors, temperature, power, firmware and service health. A DPU may host agents or provide telemetry paths that remain available even when the workload operating system is busy or being reprovisioned. This can improve operational visibility, but the monitoring system still needs defined metrics, alert thresholds, retention and ownership.
Where the DPU fits in a GPU server architecture
A useful reference path is:
- GPU and application domain: training, inference, analytics or other accelerated workload.
- Host CPU and memory: operating system, application services, orchestration and local control.
- DPU infrastructure domain: selected network, storage, security and telemetry functions.
- Leaf or ToR switch: server-facing aggregation and policy-aware fabric connection.
- Spine, storage and management services: workload communication, data access and operations.
The PORTOWER G7466 X6 GPU server can be used as a catalog reference when making a server expansion matrix. The SD6881-32H switch and 400G QSFP-DD DR4 optical module can be used as examples when documenting the switch and optical layers. This is an architecture-planning relationship, not a claim that the three items are a pre-validated DPU solution.
DPU selection checklist for AI infrastructure
| Area | Questions to answer before ordering |
|---|---|
| Workload | Is the target training, inference, cloud tenancy, storage or edge processing? |
| Connectivity | Ethernet or InfiniBand? What port speed, lane count, FEC and breakout are required? |
| PCIe and server fit | Does the server provide the required PCIe generation, slot width, power and airflow? |
| Offloads | Which network, storage, encryption, virtualization and telemetry functions are actually supported? |
| Software | Are the OS, drivers, SDK, orchestration, container runtime and firmware versions compatible? |
| Operations | Who owns provisioning, policy, keys, upgrades, logs, resets and incident response? |
| Thermals | Is the DPU air-cooled or liquid-cooled, and does the selected server and rack support it? |
What should a DPU acceptance test measure?
A practical acceptance plan can include link bring-up, throughput and packet-loss profiles, small-packet behavior, RDMA or RoCE behavior where applicable, storage latency and recovery, host-overhead comparison, policy enforcement, tenant isolation, secure-boot state, firmware upgrade and rollback, telemetry continuity, power and thermal alarms, and failure of a switch link or DPU service.
Record the exact DPU model, firmware, SDK, host BIOS, operating system, NIC or switch configuration, optical modules, fiber, traffic generator and workload profile. Without this record, a result is difficult to reproduce or compare.
Frequently asked questions
Does a DPU replace the GPU?
No. A DPU handles selected infrastructure processing; the GPU remains the accelerator for the application workload.
Is every SmartNIC a DPU?
Not necessarily. The terms overlap in the market, but the supported processing resources, control-plane model, offloads and software framework must be checked for the exact platform.
Does adding a DPU automatically make an AI cluster faster?
No. It can reduce host overhead or improve isolation for supported functions, but the outcome depends on the workload, software path, topology, congestion, storage and configuration.
Should the DPU be selected before the GPU server?
Select them as one system. PCIe topology, slot space, power, airflow, firmware and network design all need to be compatible.