AMD GPU stack for HPC operators

If your career has been spent driving NVIDIA fleets — DGX boxes, NVLink, NVSwitch, NCCL, the GPU Operator — and someone hands you a rack of AMD Instinct MI300X and says "we just bought 1024 of these, you have eight weeks to make them train," you do not need to relearn HPC. You need a translation table. The hardware concepts map almost one-to-one. The names are different, the kernel modules are different, the collectives library is a fork — but the topology, the failure modes, the rail-binding, the GPUDirect-style RDMA, all of it is conceptually the same.

This page is the translation table at the architecture level. The other pages in this section (rocm, rccl-vs-nccl, troubleshooting, k8s-amd-gpu-operator) take it down to the command line.

The AMD Instinct lineup at a glance

AMD's datacenter GPU line is branded Instinct. The architecture family is CDNA (Compute DNA), the compute-only sibling of RDNA used in consumer Radeon cards. CDNA is the AMD answer to NVIDIA's compute-focused branches (V100/A100/H100/Hopper/Blackwell) — it strips out raster pipelines and graphics fixed-function blocks, doubles down on FP64, FP16, BF16, FP8, and now FP6/FP4.

(All numbers are peak / on-paper / dense. Real workload throughput depends on your kernel, the HIP libraries used, and how saturated the Infinity Fabric is.)

How to read CDNA generations

CDNA evolves the same way Hopper/Blackwell does — each generation roughly doubles useful compute density, expands memory, and adds new low-precision formats. Where NVIDIA leads on tooling and ecosystem maturity, AMD competes on memory capacity per dollar and on raw HBM bandwidth.

CDNA 1 (MI100, 2020)

First-gen Matrix Cores. FP64, FP32, FP16, BF16, INT8/4. Roughly equivalent in headline FP64 to A100 PCIe but no Tensor-Core-equivalent for FP64 matrix. xGMI interconnect was present but anaemic — only useful for 4-GPU configurations. Not really a competitor to A100 in the field; mostly research and Frontier-pathfinder hardware.

CDNA 2 (MI200 series — MI210, MI250, MI250X, 2021–2022)

Big leap. Aldebaran die. Matrix FP64 — the headline that actually mattered for HPC. The MI250X powered Frontier (Oak Ridge), the first 1.1 exaflop supercomputer. Dual-die OAM packages (MI250/MI250X) double the GPU count without doubling the slot count, but the OS sees them as 2 GCDs (Graphics Compute Dies). Operationally, this means a single OAM = 2 logical GPUs, and rocm-smi reports them as separate devices.

CDNA 3 (MI300 series, 2023–2024)

The architecture that put AMD back in the AI conversation. Two SKUs:

• MI300A — APU. 24 Zen 4 cores + 228 CDNA 3 compute units + 128 GB HBM3, all on one package, all coherent. This is what powers El Capitan (LLNL) — the first 1.7+ exaflop supercomputer, and the first to displace Frontier on the TOP500. The relevance for AI operators is limited (it's a supercomputing-only SKU), but it is the architectural foundation.
• MI300X — discrete OAM GPU. 192 GB HBM3 in a single package. No CPU. This is what shows up in Microsoft Azure ND-MI300X-v5, Oracle, Meta, and the hyperscaler procurements. The 192 GB is the killer feature: you can fit a 70B fp16 model on a single GPU with full KV cache. With 8 of them in a node (1.5 TB HBM3) you can host Llama-3.1-405B fp8 on a single box without tensor-parallel sharding.

CDNA 4 (MI350 series — MI350X, MI355X, 2025)

3 nm process. FP6/FP4 support — same direction Blackwell took. 288 GB HBM3e per GPU (more than B200's 192, less than B300's 288). 8 TB/s memory bandwidth (matches B200/B300). The MI355X is the liquid-cooled, full-power flagship; the MI350X is the air-cooled-friendly variant. Both are 8-way OAM platforms.

Infinity Fabric: AMD's NVLink

This is the one piece of architecture you have to internalize before anything else makes sense.

The most important architectural difference: AMD does not have an NVSwitch. On an 8-GPU MI300X OAM platform, the 8 GPUs are wired in a fully-connected point-to-point mesh (each GPU has 7 xGMI links to the other 7). On NVIDIA HGX H100, the 8 GPUs all hang off 4 NVSwitch chips that provide non-blocking all-to-all.

Operationally:

• For an 8-GPU AllReduce, both architectures effectively achieve full bandwidth. The mesh is enough.
• For uneven traffic patterns (tensor parallel + pipeline parallel mixing on the same box), NVSwitch can rebalance dynamically; the AMD mesh has fixed link allocations. Heavily-skewed workloads can leave some xGMI links idle while others bottleneck.
• For >8-GPU intra-node domains, NVIDIA scales via NVL72 (72 GPUs in one NVLink domain). AMD does not have an equivalent — beyond 8 GPUs, you go over the NIC fabric (RoCE/IB), same as NVIDIA pre-NVL72.

This affects model parallelism strategies and matters when you're sizing a fleet for a specific training job. For inference of a single model that fits in 8 GPUs, the difference is essentially irrelevant.

CDNA vs NVIDIA generations: side-by-side

For operators who think in NVIDIA terms, here is how to map generations to roughly comparable AMD silicon:

When AMD is the right choice

This is the question that gets you in trouble if you answer it ideologically. "Just buy NVIDIA" is wrong as often as "just buy AMD." Here are the cases where the operationally correct answer is AMD:

Inference of large open-weight models, single-node deployments

The MI300X with 192 GB lets you fit Llama-3.1-405B in fp8 (~410 GB) on a single 8-GPU node (1.5 TB HBM3). The same model on H100 80 GB needs careful tensor-parallel sharding plus offloading. The same model on H200 141 GB fits but with no headroom for batching. Cost-per-token-served on MI300X is meaningfully lower for models in the 70B–405B fp8 range. This is the biggest production AMD niche today.

Cost per FLOP / cost per HBM GB

When you put MI300X in a procurement against H100 SXM5 (not H200/B200, those are different generations), you typically see:

• ~30–40% lower list price per GPU.
• 2.4× the HBM capacity.
• ~1.5× the HBM bandwidth.

Whether that translates to lower cost-per-token depends entirely on your kernel performance (see below). For inference-heavy fleets on well-optimized models (vLLM, TGI, llama.cpp on ROCm), it often does.

HPC workloads that already use ROCm / HIP

If you are operating one of the DOE/EU exascale machines (Frontier, El Capitan, LUMI, Adastra, Setonix), you already have a ROCm software stack and your codes are already HIP-ified. AMD is the obvious choice for the next refresh.

Hyperscaler procurement

Microsoft Azure, Oracle, Meta, and several sovereign clouds have committed multi-billion-dollar AMD orders. If you are operating inside or alongside one of those clouds, MI300X / MI325X capacity is more available than H100/H200 capacity, and often cheaper per GPU-hour.

Memory-bound inference (long-context, large KV cache)

Anything where the bottleneck is HBM capacity, not FLOPS, favors AMD. Long-context serving (1M-token windows), KV-cache-heavy reasoning models, batch-large-N inference for MoE — these all map well to MI300X/MI325X/MI355X.

When AMD is the wrong choice

Be honest about this too:

Brand-new training research where every PyTorch nightly matters

NVIDIA's CUDA ecosystem is years ahead of ROCm in coverage. If your researchers want to use the latest FlashAttention-3 fork, the latest TransformerEngine kernel, the latest Megatron-Core feature — there's a good chance it ships CUDA-only first and HIP-ports show up weeks or months later. ROCm 6 closed a lot of this gap; ROCm 7 closes more; but at the bleeding edge, NVIDIA still wins.

Mixed multi-tenant inference with strict GPU partitioning needs

NVIDIA MIG (Multi-Instance GPU) lets you slice an H100 into up to 7 isolated instances. AMD has no equivalent on MI300X. On CDNA 3, the MI300X is a single GPU, period. CDNA 4 adds partitioning modes (CPX, SPX) that approximate MIG, but they are coarser-grained. For multi-tenant clouds offering "1/7 of a GPU," NVIDIA is still the operationally easier choice. See k8s-amd-gpu-operator.

Mature trillion-parameter training at NVL72-rack scale

Until MI400 ships in volume, AMD has no answer to GB200 NVL72 — 72 GPUs in one NVLink domain. If your training plan requires that, NVIDIA is the only option in 2026.

Mixed teams with little ROCm experience

If your operator team and your ML team have collectively zero ROCm experience, the ramp is real. Budget 4–8 weeks of teething: kernel-module debugging, NCCL → RCCL env-var translation, container-image rebuilds, hipify of internal kernels. Don't pretend it's free.

CDNA architecture vs RDNA, and why it matters

AMD GPUs split into two architecture families that share branding but very little silicon:

• RDNA (Radeon DNA) — consumer/gaming. Powers Radeon RX 7000 / RX 9000 series, Steam Deck, etc. Optimized for raster, ray tracing, FP32, gaming workloads. Uses GCN-derived compute units organized for graphics.
• CDNA (Compute DNA) — datacenter/HPC. Powers all Instinct MI100/MI200/MI300/MI350 silicon. Strips out raster pipelines, doubles down on Matrix Cores, FP64, FP16, BF16, FP8, FP6/FP4. Optimized for compute density and memory bandwidth.

You cannot run datacenter ROCm workloads on a consumer Radeon card in any production sense. ROCm officially supports a small set of consumer GPUs (RX 7900 XTX, certain Pro variants) for hobbyist development, but the supported-list is narrow and the performance characteristics differ wildly from the Instinct line. If someone offers you "AMD GPUs for AI" and the SKU is a consumer Radeon, that is not a serious offering for production HPC.

Equivalent NVIDIA split: GeForce / Quadro RTX (consumer, workstation) vs Tesla / A-series / H-series / B-series (datacenter). The H100 has very little in common architecturally with an RTX 4090 even though both are "Hopper-generation" silicon — and similarly, MI300X has very little in common with an RX 7900 XTX.

How to read AMD silicon names

The naming convention is internally consistent once you know the pattern:

   M I 3 5 5 X
   | | | | | +-- variant suffix:
   | | | | |     X = OAM/discrete top-of-stack
   | | | | |     A = APU (CPU + GPU on one package, MI300A only)
   | | | | |     (no suffix) = standard configuration
   | | | | +---- minor revision (5 = MI355X is 5th in MI3-series)
   | | | +------ generation marker (5 = CDNA 4 era)
   | | +-------- series (3 = MI300/MI350 family, 2 = MI200, 1 = MI100)
   | +---------- product line marker
   +------------ "M" for "Machine" / "Instinct"
   "I" for Instinct

Examples:

• MI50, MI60 — pre-CDNA Vega-derived, mostly historical.
• MI100 — CDNA 1.
• MI200, MI210, MI250, MI250X — CDNA 2.
• MI300A, MI300X, MI325X — CDNA 3.
• MI350X, MI355X — CDNA 4.
• MI400 series — announced for 2026, CDNA Next.

When operators say "MI300" without qualification, they almost always mean MI300X (the discrete inference accelerator). MI300A is an HPC/supercomputing-only SKU.

What an MI300X OAM platform actually looks like

The reference design is the AMD Universal Baseboard (UBB) — an 8-GPU OAM platform that hyperscalers and OEMs use as the building block. SuperMicro AS-8125GS-TNMR2, Dell XE9680, HPE Cray XD675, Lenovo SR685a — under the brand-skin they are all variations of the UBB reference.

      +-----------------------------------------------------------+
      |  AMD UBB (8x MI300X OAM)                                  |
      |                                                           |
      |  GPU0 ====xGMI mesh==== GPU1 ====xGMI mesh==== GPU2 ====  |
      |   ||                     ||                     ||        |
      |   ||                     ||                     ||        |
      |  GPU3 ====xGMI mesh==== GPU4 ====xGMI mesh==== GPU5 ====  |
      |   ||                     ||                     ||        |
      |   ||                     ||                     ||        |
      |  GPU6 ====xGMI mesh==== GPU7  (each GPU has 7 xGMI links) |
      |                                                           |
      |  PCIe Gen5 x16 to host CPU per GPU (8 lanes total)        |
      |  8x 400/800 Gb/s NICs (one per GPU rail)                  |
      +-----------------------------------------------------------+

The host CPU is typically dual-socket AMD EPYC 9004/9005 (Genoa/Turin) or Intel Xeon (Sapphire Rapids / Emerald Rapids / Granite Rapids). Each GPU has its own dedicated PCIe Gen5 x16 link to the host root complex; rail-binding works the same as on NVIDIA HGX platforms — GPU N pairs with NIC N on the same NUMA node.

For a typical 1024-GPU MI300X cluster, you have:

• 128 nodes of 8 GPUs each.
• 128 × 8 = 1024 GPUs, total HBM = 192 TB.
• Per-node aggregate xGMI bandwidth = 8 × 896 GB/s = 7.2 TB/s within each box.
• Per-node aggregate NIC bandwidth = 8 × 400 Gb/s = 3.2 Tb/s to the spine fabric.
• Total cluster HBM bandwidth (all GPUs simultaneously) = 1024 × 5.3 TB/s = 5.4 PB/s.

This is roughly equivalent to a 1024× H100 cluster on memory bandwidth, with 2.4× the aggregate HBM capacity.

Pre-procurement benchmark checklist

Before you sign for a multi-million-dollar AMD fleet, you want hands-on numbers, not vendor-supplied PDFs. The minimum viable benchmark set, in priority order:

1. rccl-tests AllReduce — single-node 8-GPU at sizes 8 KB, 1 MB, 64 MB, 1 GB, 8 GB. Compare busbw to AMD's published reference. Target: ≥ 90% of the reference number.
2. rccl-tests AllReduce — 4-node × 8-GPU, same sizes. Target: ≥ 85% of reference.
3. MLPerf Inference Llama-2 70B — both server and offline scenarios. Compare against published MI300X baselines. Use AMD's reference vLLM container.
4. A real production-shape inference workload — your actual model, your actual context length, your actual batch shape. P99 latency and throughput matter more than peak.
5. Sustained 6-hour training run — same recipe you'd run in production. Watch for thermal throttling, ECC scrubbing, NaN losses, fabric error growth.
6. Container-pull timing — pull rocm/pytorch from your intended registry to a fresh node. If it takes more than 5 minutes, plan a local mirror.

Skipping any of these is how you end up with a fleet that benchmarks beautifully on AllReduce but loses 30% of inference throughput to a kernel-arch mismatch you didn't catch.

"I have an MI300X — show me it works in 60 seconds"

For an operator who has just been handed access to a freshly-imaged MI300X box, the literal first three commands:

# 1. Module loaded and reporting devices
$ rocm-smi
# expect: 8 GPUs (or 1 / 2 / 4 / however many physically installed)

# 2. HIP runtime works, sees right gfx target
$ rocminfo | grep -E "Name:.*gfx"
# expect: gfx942 (MI300A/X), gfx942 (MI325X), gfx950 (MI350/MI355X)

# 3. Topology is the expected mesh
$ rocm-smi --showtopo
# expect: XGMI off-diagonal entries on multi-GPU boxes

If all three return what they should, you have a functional GPU stack. Anything else is debug territory — see troubleshooting.

Power and cooling: the real procurement constraint

This is where AMD vs NVIDIA conversations get operationally serious. The TDP numbers on paper are similar — H100 SXM5 at 700 W, MI300X at 750 W. But the density is what determines whether you need to retrofit your datacenter:

The operationally important pattern: AMD and NVIDIA are converging on the same per-node power envelope at each generation. If your datacenter can host 8× H200 boxes air-cooled at 14 kW per rack, it can host 8× MI300X air-cooled. If you need MI355X, you need direct-to-chip liquid cooling, same as B300.

What's not equivalent: NVIDIA's NVL72 packaging puts 72 GPUs in one rack at 120 kW. AMD has no equivalent rack-scale shrink today; an 8-GPU UBB stays in its own 4U or 8U chassis. For sites that have already retrofitted for 120 kW NVL72, they could in principle host more than one AMD UBB per rack, but few sites do.

Procurement-side notes that matter at scale

A few practical realities operators have learned the hard way:

Dual-die naming on MI200

The MI250 and MI250X are dual-die packages — one OAM module, two GPU dies (GCDs — Graphics Compute Dies). The OS sees them as two devices. So an "8-OAM MI250X node" actually has 16 logical GPUs in rocm-smi, in lsmod, in PyTorch. Inventory systems get this wrong constantly. When someone says "I have a 16-GPU MI250X box," they likely mean an 8-OAM node where each OAM has 2 dies.

MI300X is single-die per OAM, so 8 OAMs = 8 logical GPUs. MI355X is also single-die per OAM. The dual-die confusion is MI200-era only.

"MI300X" without specifying the platform

"MI300X" alone is ambiguous. There are at least three form factors in the wild:

1. MI300X OAM — what hyperscalers buy. 750 W, 8-way UBB.
2. MI300X PCIe — a card-format variant for non-OAM chassis. Lower-bandwidth xGMI bridge between cards via cable. Less common.
3. MI300A APU — different SKU entirely; CPU+GPU on one package, 128 GB unified memory. Used in El Capitan and a handful of HPC sites; not for general AI workloads.

When evaluating quotes or deployments, pin down the specific SKU and form factor. The performance and operational characteristics diverge significantly.

CDNA generation skipping

Operators sometimes ask "should I wait for MI355X / MI400 / next-gen?" The pattern that holds for both NVIDIA and AMD: the new generation is always 30% better and ships 9-12 months later than announced. Buying current-gen at a known discount usually beats waiting for next-gen. The exceptions are when a specific feature (FP4 on Blackwell/CDNA4, NVL72 scaling) is genuinely transformative for your workload — then the wait is justified.

The procurement signal

When a hyperscaler or sovereign cloud announces a multi-billion AMD order (Microsoft, Oracle, Meta, G42, BSC, Lawrence Livermore), it is not an ideological choice. It is two things:

1. Capacity — NVIDIA H100/H200 had multi-quarter lead times in 2024–2025. AMD had inventory.
2. TCO for inference — The 192 / 256 / 288 GB HBM configurations beat H100/H200 on $/HBM-GB and on $/HBM-TB-bandwidth, and inference is where most of the GPU spend goes (training is once, inference is forever).

If you are sizing a fleet at the >$100M scale, you should evaluate AMD seriously. If you are running <50 GPUs and doing primarily R&D, the answer is almost always still NVIDIA.

Software ecosystem: where AMD has caught up, where it hasn't

The single biggest operational change between 2023 (when MI300X first shipped) and 2026 (now) is that the ROCm software story stopped being a fixer-upper.

The takeaway: what was a research-grade software stack in 2023 is now a production-grade stack in 2026. For mainstream LLM training and inference, you are not blocked by missing tooling. For bleeding-edge research that publishes CUDA-first reference implementations, expect a 2–8 week delay before HIP ports land.

The remaining gaps that actually bite:

• Some vendor-specific kernels (e.g. cuDNN-fused attention variants) don't have direct ROCm equivalents. MIOpen covers most cases but not all niche ops.
• TensorRT (NVIDIA's inference compiler) has no exact AMD analog. MIGraphX is the closest thing but is less mature; vLLM/TGI are the more common deployment paths on AMD.
• NCCL has SHARP support; RCCL does not. For very-large-scale (>1024 GPU) AllReduce-bound training, this is the gap that matters most.

A typical 8-GPU MI300X box vs DGX H100, side by side

For operators sizing or comparing platforms, the practical specs of an 8-GPU UBB MI300X vs a DGX H100:

The headline operational difference: 2.4× the HBM, 1.6× the bandwidth, comparable compute on AMD. For inference of memory-bound models, this is decisive. For training where collective bandwidth dominates and SHARP/NVLS matter, NVIDIA still has the edge — but not by 2.4×.

A note on FP4/FP6 (CDNA 4)

Both CDNA 4 (MI355X) and Blackwell (B200/B300) introduced FP4 / FP6 support. These are the formats that drive the headline "20+ PFLOPS" numbers in the spec sheets. The operational impact:

• FP8 is now mature on both vendors. Production training in FP8 is real (DeepSeek, Llama-3.1, etc.) and works on H100/H200/B100/B200 and MI300X/MI325X/MI355X. The inference savings vs FP16 are 2× memory + ~1.7× compute.
• FP6 is mostly an inference format. Llama-3.1 405B fits in FP6 on a single 8× MI355X node (288 GB × 8 = 2.3 TB, model is ~310 GB in FP6 with KV cache).
• FP4 is still experimental for production training. Inference works (DeepSeek-V3 NVFP4 demos) but loss landscape stability for from-scratch training is an open question.

If you are buying for a 2026–2027 inference-heavy fleet, FP4/FP6 capability matters. Both NVIDIA Blackwell and AMD CDNA 4 give it to you; older silicon does not.

What this section covers

• ROCm stack — the AMD equivalent of the NVIDIA driver stack. Kernel modules (amdgpu, amdkfd), userspace (rocm-hip-runtime, rocm-libs), distro packaging, validation.
• RCCL vs NCCL — RCCL is a fork of NCCL with the same API but different transports. What changes, what doesn't.
• Troubleshooting — common AMD failures and their NVIDIA-side equivalents, so a CUDA-trained operator can debug ROCm by analogy.
• AMD GPU Operator on Kubernetes — the AMD analogue to the NVIDIA GPU Operator. DaemonSets, device-plugin, amd.com/gpu resource, multi-tenant patterns.

Buyer's questions to ask the vendor

If you're evaluating an MI300X / MI325X / MI355X procurement, these are the questions that surface real differences across vendors and platform variants:

1. What is the exact gfx target? (gfx942 for MI300X/MI325X, gfx950 for MI350X/MI355X.) Container compatibility hinges on this.
2. Are these dual-die or single-die OAMs? MI200-era is dual-die; MI300/MI350-era is single-die. Affects logical GPU count.
3. What is the per-GPU TDP, and is the chassis rated for sustained operation at that TDP? Many "MI300X" boxes are spec'd for transient peak but throttle under sustained load.
4. Is cooling air or liquid? Sustained MI355X requires direct-to-chip liquid; some vendors ship "air-cooled MI355X" that throttles after 10 minutes.
5. Which ROCm version is the platform certified for? MI300X needs ROCm 6.2+ for stability; MI355X requires 7.x.
6. What is the NIC count and rail-binding layout? 8× 400 Gb/s rail-bound is the baseline; some discount platforms ship 4× NICs with 2 GPUs sharing a rail (much worse for collective bandwidth).
7. Is amdgpu-dkms the only kernel driver path, or does the vendor ship a different OEM-customized fork? OEM forks lag upstream; avoid.
8. What's the RMA process for an MI300X module? Single-die failures mean replacing an OAM, which costs roughly the same as replacing an H100 SXM5.

A vendor who answers all 8 confidently is a vendor who has actually shipped the platform. A vendor who hedges on 3+ is one whose platform you'll be debugging in production.

What this section doesn't cover

A few topics that come up but are out of scope here:

• AMD's GPU TEE / Confidential Computing. Earlier-stage than NVIDIA's; covered briefly in k8s-amd-gpu-operator.
• AMD ROCm-DCN / FPGA / network-attached accelerators. These are different product lines (Pensando, Xilinx-derived); they don't share the ROCm stack.
• Consumer Radeon for development. Possible but narrow; if you're hobbyist-prototyping on a 7900 XTX, the official ROCm-on-Radeon documentation is the right starting point.
• Frontier / El Capitan / LUMI specific operations. Site-specific HPC operations have their own playbooks layered on top of ROCm.

See also

• NVIDIA GPU generations — the NVIDIA-side lineup this page maps against
• NVIDIA driver stack — what amdgpu vs amdkfd map to
• NVLink and NVSwitch — the NVIDIA fabric AMD's xGMI/Infinity Fabric is competing with
• NCCL multi-node tuning — RCCL inherits most of these env vars
• PCIe topology — same constraints apply on AMD platforms
• GPUDirect RDMA — works on AMD via in-amdgpu peermem, configured similarly