ROCm: the AMD GPU software stack from kernel module to userspace

ROCm (Radeon Open Compute) is AMD's answer to CUDA. It's the umbrella name for everything from the kernel module up through the math libraries — analogous to "the NVIDIA stack" meaning nvidia.ko + libcuda.so + cuBLAS/cuDNN/NCCL. If you have spent years debugging the NVIDIA driver stack, the conceptual layout will feel familiar; the package names and kernel module names are different, and there are a handful of operationally meaningful divergences worth knowing before you SSH into your first MI300X box.

This is the equivalent of the NVIDIA driver stack page, but for AMD.

The ROCm stack, top to bottom

The single most important architectural difference: AMD has effectively one kernel module (amdgpu) where NVIDIA has five. That means fewer load-order failure modes, but it also means a misbehaving display feature can affect compute and vice versa. There's no "compute-only" mode; you get the whole module.

amdgpu vs amdkfd: what they each do

This is the question that confuses CUDA operators on day 1.

amdgpu (the modern path, in-tree since Linux 4.7)

amdgpu is the upstream Linux kernel module that handles AMD discrete GPUs — including the Instinct datacenter cards. It does:

• DRM/KMS — display, framebuffer, mode-setting (irrelevant on a headless MI300X box but still loaded).
• Compute submission — graphics + compute share the same command processor. Compute work goes through the same kernel module.
• Power management — DPM (Dynamic Power Management) state control, exposed through /sys/class/drm/card*/device/.
• Memory management — VRAM allocation, page tables, IOMMU coordination.
• GPUDirect RDMA — since ROCm 5, the peer-memory hooks for InfiniBand/RoCE NICs are inside amdgpu itself. There is no separate amd-peermem module the way NVIDIA has nvidia_peermem. (See GPUDirect for the broader concept.)

When you apt install amdgpu-dkms, this is what gets compiled and loaded.

amdkfd (the legacy compute path, deprecated)

amdkfd (Kernel Fusion Driver) was a separate kernel module that handled compute submission on older AMD GPUs (Radeon VII, MI25, MI50, MI60, original CDNA1 silicon). On modern systems, its functionality has been absorbed into amdgpu. You will not see amdkfd.ko loaded as a separate module on an MI300X box.

If you still see references to amdkfd in old documentation, treat them as historical. On Linux 5.x and 6.x kernels with ROCm 6+, only amdgpu exists.

# What's actually loaded on a healthy MI300X box
$ lsmod | grep -E '^amdgpu|^amdkfd|^drm'
amdgpu               16646144  56
drm_buddy             20480     1 amdgpu
gpu_sched             65536     1 amdgpu
drm_display_helper    278528    1 amdgpu
drm_kms_helper        249856    1 amdgpu
drm                  831488    9 gpu_sched,drm_kms_helper,drm_display_helper,drm_buddy,amdgpu

No amdkfd. Just amdgpu plus its DRM helper modules.

ROCm 5 vs ROCm 6 vs ROCm 7: what changed

You will inherit a fleet running one of these. Knowing what changed between major versions tells you what to expect when you upgrade.

The version you pick is dictated by your GPU generation, not by personal preference:

• MI100 / MI210 / MI250X: ROCm 5 minimum, ROCm 6 recommended. ROCm 7 still supports them but the development focus has shifted.
• MI300X / MI300A / MI325X: ROCm 6.0 minimum, ROCm 6.2 strongly recommended (a lot of MI300X kernels were unstable in 6.0/6.1). ROCm 7 is current.
• MI350X / MI355X: ROCm 7.0 minimum. There is no backport.

Mixing GPU generations on a single host is permitted but means you must be on the ROCm version that supports the newest card.

Distro support matrix

ROCm has a well-defined supported-OS list. If you are not on one, things will appear to work and then break in subtle ways during long training runs. Don't fight it — pick a supported combo.

If you are deploying greenfield, Ubuntu 22.04 + ROCm 6.2/6.3 + MI300X, or Ubuntu 24.04 + ROCm 7.x + MI355X, are the well-trodden paths.

Installing ROCm via the apt repository

The canonical AMD-blessed install is amdgpu-install, a wrapper script that adds the AMD apt repo and lets you choose a "use case" (graphics, compute, both). For datacenter operators, the use case is always rocm.

Ubuntu 22.04 / 24.04

# Add the AMD repository (replace 7.2.3 with the version you want)
wget https://repo.radeon.com/amdgpu-install/7.2.3/ubuntu/jammy/amdgpu-install_7.2.3.70203-1_all.deb
sudo apt install ./amdgpu-install_7.2.3.70203-1_all.deb
sudo apt update

# Install kernel headers + module sources
sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)"

# Install the kernel module via DKMS
sudo apt install amdgpu-dkms

# Reboot to load the freshly compiled module
sudo reboot

After reboot, install ROCm userspace:

sudo apt install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME    # or your service account
sudo apt install rocm
sudo reboot

The rocm meta-package pulls in everything: HIP runtime, math libraries, RCCL, debugger, profiler, the works. If you want a smaller install (a container builder, for example), use one of the more granular packages:

RHEL 9.x

# Add the AMD repository
sudo dnf install https://repo.radeon.com/amdgpu-install/7.2.3/rhel/9.7/amdgpu-install-7.2.3.70203-1.el9.noarch.rpm
sudo dnf clean all

# Install kernel headers + devel
sudo dnf install "kernel-headers-$(uname -r)" "kernel-devel-$(uname -r)" "kernel-devel-matched-$(uname -r)"

# Kernel module
sudo dnf install amdgpu-dkms
sudo reboot

# EPEL + CRB (for python3-setuptools and friends)
sudo dnf install https://dl.fedoraproject.org/pub/epel/epel-release-latest-9.noarch.rpm
sudo dnf config-manager --enable codeready-builder-for-rhel-9-x86_64-rpms

# ROCm
sudo dnf install python3-setuptools python3-wheel
sudo usermod -a -G render,video $LOGNAME
sudo dnf install rocm
sudo reboot

DKMS or precompiled?

Same trade-off as on the NVIDIA side. AMD ships only DKMS for the datacenter packages (amdgpu-dkms). There is no precompiled .ko-shipping equivalent of nvidia-driver-XXX. This means:

• Pro: kernel upgrades on the host transparently rebuild the module. No "kernel upgraded, GPU vanished after reboot" surprise.
• Con: if linux-headers-$(uname -r) is missing at upgrade time, DKMS silently fails and you find out next reboot. Always install matching headers before upgrading the kernel.

# Verify DKMS health
$ dkms status
amdgpu/6.10.5.70203-1, 6.8.0-49-generic, x86_64: installed
amdgpu/6.10.5.70203-1, 6.8.0-50-generic, x86_64: installed

# If you see "WARNING: not built for kernel X" — rebuild before you reboot:
sudo dkms autoinstall

The minute-of-paranoia check before any reboot:

# 1. Is amdgpu currently loaded?
lsmod | grep amdgpu

# 2. Is it built for the kernel you'll boot into?
ls /var/lib/dkms/amdgpu/*/$(uname -r)/x86_64/module/

# 3. Are there headers for that kernel?
dpkg -l "linux-headers-$(uname -r)" 2>/dev/null || rpm -q "kernel-headers-$(uname -r)"

If any of those is no, do not reboot.

Persistence mode equivalent

NVIDIA has nvidia-smi -pm 1 (persistence mode), which keeps the driver loaded so the first CUDA call doesn't pay the ~3-second init cost.

AMD has no exact equivalent. The amdgpu module is loaded at boot and stays loaded — it doesn't unload between processes, so there's no equivalent latency to amortize. There's nothing to enable.

What you might want, similar in spirit:

• Power state pinning — keep the GPU in the highest performance state instead of letting it idle:

  # See current DPM state
  cat /sys/class/drm/card0/device/power_dpm_force_performance_level
  # auto
  
  # Pin to high
  echo high | sudo tee /sys/class/drm/card0/device/power_dpm_force_performance_level
  

  This is the rough equivalent of nvidia-smi -lgc (lock GPU clocks). For inference latency-sensitive deployments, pinning to high removes a ~2 ms warm-up on the first batch.

• Clock pinning — set fixed sclk (graphics clock) and mclk (memory clock):

  echo manual | sudo tee /sys/class/drm/card0/device/power_dpm_force_performance_level
  echo "s 7" | sudo tee /sys/class/drm/card0/device/pp_dpm_sclk
  echo "m 3" | sudo tee /sys/class/drm/card0/device/pp_dpm_mclk
  

  (The numeric levels come from cat /sys/class/drm/card0/device/pp_dpm_sclk which lists the available DPM levels.)

For most production deployments, the kernel default (auto) is fine. Pin only if you've measured a real latency win.

Validation: rocm-smi, rocminfo, amd-smi

The minute the install finishes and you've rebooted, run all three before declaring victory.

rocm-smi — the closest analog to nvidia-smi

$ rocm-smi


============================ ROCm System Management Interface ============================
====================================== Concise Info ======================================
Device  [Model : Revision]    Temp     Power  Partitions      SCLK    MCLK    Fan  Perf  PwrCap  VRAM%  GPU%
        Name (20 chars)       (Edge)   (Avg)  (Mem, Compute)
==========================================================================================
0       [0x74a1 : 0x00]       38.0°C   142.0W N/A, N/A        140Mhz  900Mhz  0%   auto  750.0W   0%    0%
        AMD Instinct MI300X
1       [0x74a1 : 0x00]       37.0°C   140.0W N/A, N/A        140Mhz  900Mhz  0%   auto  750.0W   0%    0%
        AMD Instinct MI300X
... (8 GPUs)
==========================================================================================
================================== End of ROCm SMI Log ===================================

What you check immediately:

• All 8 GPUs visible (or however many are physically installed).
• Power cap matches the SKU (750 W for MI300X, 1000 W for MI325X air-cooled, 1400 W for MI355X liquid).
• No GPUs in a lower DPM state unexpectedly — if one card sits at 700 MHz SCLK while others are at 1900 MHz under load, that's a thermal-throttling or hardware issue.
• VRAM column shows the right capacity per generation (192 GB MI300X, 256 GB MI325X, 288 GB MI355X).

rocminfo — the equivalent of nvidia-smi -q plus cuobjdump

$ rocminfo | head -50
ROCk module is loaded
=====================
HSA System Attributes
=====================
Runtime Version:         1.14
System Timestamp Freq.:  1000.000000MHz
Sigmask Base Pointer:    0x...
=====================
HSA Agents
==========
*******
Agent 1
*******
  Name:                    AMD EPYC 9534 64-Core Processor
  ...
*******
Agent 2
*******
  Name:                    gfx942
  Marketing Name:          AMD Instinct MI300X
  Vendor Name:             AMD
  Feature:                 KERNEL_DISPATCH
  ...
  Compute Unit:            304
  SIMDs per CU:            4
  Shader Engines:          32
  ...

The line you actually care about for compatibility: Name: gfx942. This is the GPU architecture string — what HIP / PyTorch / vLLM use to find the right precompiled kernels.

If you're running a container that was built with PYTORCH_ROCM_ARCH=gfx90a on an MI300X, PyTorch will JIT-recompile every kernel on first use — many minutes per kernel, sometimes failing on more exotic ops. Always make sure the container's PYTORCH_ROCM_ARCH env (or HCC_AMDGPU_TARGET, HSA_OVERRIDE_GFX_VERSION) matches the actual silicon. This is the AMD equivalent of TORCH_CUDA_ARCH_LIST.

amd-smi — the newer monitoring CLI

amd-smi is the future. AMD is migrating monitoring functionality from rocm-smi (which is GPU-only) to amd-smi (which covers CPU, GPU, NIC, BMC). For now, both ship and both work.

$ amd-smi list
GPU: 0
    BDF: 0000:18:00.0
    UUID: ...
GPU: 1
    BDF: 0000:38:00.0
    UUID: ...
...

$ amd-smi monitor -ptum
GPU  POWER  GPU_TEMP  MEM_TEMP  GFX_UTIL  GFX_CLOCK  MEM_UTIL  MEM_CLOCK  ENC_UTIL  DEC_UTIL  THROTTLE  VRAM_USED  VRAM_TOTAL  VRAM_PERCENT
0    142 W  38°C      35°C      0%        140 MHz    0%        900 MHz    -         -         -         512 MB     192 GB      0%
1    140 W  37°C      35°C      0%        140 MHz    0%        900 MHz    -         -         -         512 MB     192 GB      0%
...

For Prometheus scraping, AMD ships amd-device-metrics-exporter (analog of dcgm-exporter). See k8s-amd-gpu-operator.

hipify-perl — sanity-check CUDA porting

If your team is porting a CUDA codebase, hipify-perl is the regex-based translator that turns cudaMalloc into hipMalloc, cudaMemcpy into hipMemcpy, etc.

$ hipify-perl my_kernel.cu > my_kernel.cpp.hip
# review the diff carefully — regex-based, will miss user-defined wrappers
$ hipify-perl --inplace my_kernel.cu
$ hipify-perl --examine my_kernel.cu     # report what it would change without modifying

For more thorough porting, hipify-clang parses the AST instead of regex. It needs a working CUDA install and produces cleaner output, but takes more setup.

The HIP API has been getting closer to CUDA over time — ROCm 7 specifically aimed at "even closer alignment" — but a typical mid-sized CUDA codebase will still have 20–100 places that need manual cleanup after hipify-perl. Common gotchas: thrust → rocThrust isn't 1:1, cub → hipCUB has API differences, __shfl_sync semantics differ, warp size is 64 on AMD vs 32 on NVIDIA.

A typical first-day-of-fleet validation script

Here's the smoke test to run on every new node before you accept it into the pool:

#!/bin/bash
set -e

echo "=== amdgpu module loaded ==="
lsmod | grep -q '^amdgpu' || { echo "FAIL: amdgpu not loaded"; exit 1; }

echo "=== DKMS health ==="
dkms status amdgpu | grep -q installed || { echo "FAIL: DKMS broken"; exit 1; }

echo "=== rocminfo finds GPUs ==="
GPU_COUNT=$(rocminfo | grep -c "Name:.*gfx")
[ "$GPU_COUNT" -ge 1 ] || { echo "FAIL: no GPUs in rocminfo"; exit 1; }
echo "Found $GPU_COUNT GPUs"

echo "=== rocm-smi reports all GPUs ==="
rocm-smi --showid

echo "=== HIP runtime works ==="
cat > /tmp/hip-hello.cpp << 'EOF'
#include <hip/hip_runtime.h>
#include <cstdio>
int main() {
  int n; hipGetDeviceCount(&n);
  printf("HIP device count: %d\n", n);
  for (int i = 0; i < n; i++) {
    hipDeviceProp_t p; hipGetDeviceProperties(&p, i);
    printf("  GPU %d: %s, %.1f GB\n", i, p.name, p.totalGlobalMem/1e9);
  }
  return 0;
}
EOF
hipcc /tmp/hip-hello.cpp -o /tmp/hip-hello
/tmp/hip-hello

echo "=== xGMI links healthy ==="
rocm-smi --showtopo | grep -E "XGMI|GPU"

echo "=== ECC counters at zero ==="
rocm-smi --showrasinfo all | grep -iE "uncorrectable|fatal" || echo "OK"

echo "ALL CHECKS PASSED"

If any of these fail, do not put the node into production. The cost of debugging a single bad kernel module on a busy training run far exceeds the cost of holding the node out for 30 minutes.

See also

• AMD GPU stack overview — the hardware lineup this stack runs on
• RCCL vs NCCL — the collectives library that sits on top of ROCm
• AMD troubleshooting — when this stack misbehaves
• AMD GPU Operator on Kubernetes — packaging this stack for k8s
• NVIDIA driver stack — the analogous NVIDIA stack
• GPUDirect RDMA — works on AMD too, configured differently