Transparent Huge Pages: when to disable, when to madvise

Transparent Huge Pages (THP) is the kernel's attempt to give you the latency wins of 2 MiB pages without you having to ask. On a desktop or a generic workload, that's a fine default. On an HPC GPU node, THP defaults — transparent_hugepage=always — produce some of the most painful intermittent stalls in the catalog.

This page covers what THP is, why the default hurts compute jobs, when madvise is the sweet spot, and how to actually verify which mode is in effect.

What THP actually is

Linux can serve memory in 4 KiB pages, 2 MiB pages, or 1 GiB pages (the last only via explicit hugetlb pre-allocation, see hugepages). 2 MiB pages reduce TLB pressure and let the CPU translate fewer addresses per second of work.

THP is the "automatic" path: the kernel tries to allocate 2 MiB pages whenever possible, and if it can't (memory fragmented), it falls back to 4 KiB. There's a daemon, khugepaged, that periodically scans memory and tries to promote runs of 4 KiB pages into 2 MiB pages by moving them to be physically contiguous.

Three modes exist:

Read current mode:

cat /sys/kernel/mm/transparent_hugepage/enabled
# always [madvise] never                 <-- bracketed = active

Same for defrag (controls how aggressively to compact for THP):

cat /sys/kernel/mm/transparent_hugepage/defrag
# always defer defer+madvise [madvise] never

defrag modes:

Why always hurts HPC

Three failure modes, all subtle:

1. khugepaged stalls

khugepaged scans memory every ~10 seconds (scan_sleep_millisecs). When it finds a 4 KiB region it can promote, it has to:

1. Allocate a fresh 2 MiB hugepage (which often requires direct compaction — moving pages to make room)
2. Copy the 512 4 KiB pages into the new 2 MiB page
3. Update the page tables atomically (taking PTL locks across the whole range)
4. Free the old 4 KiB pages

Step 3 holds locks. If your training thread happens to fault into memory in that range during step 3, it stalls — milliseconds of microseconds in the worst cases. For NCCL, where every collective is a barrier across all ranks, a single 5 ms stall on one rank stalls the entire world.

The fix isn't to slow khugepaged; it's to prevent it from running on memory you care about. madvise mode does exactly that — only memory the app explicitly opted in gets touched.

2. Direct compaction in the page-fault path

Under defrag=always, the first touch of a fresh anon mapping that can be a hugepage triggers compaction synchronously. If memory is fragmented (which it is on a long-running node with mixed workload history), compaction can take tens to hundreds of milliseconds. The faulting process is blocked the whole time.

This shows up as periodic latency spikes that correlate with allocator activity, not real work.

3. Memory-bandwidth amplification on cross-NUMA

A 2 MiB page is allocated on a single NUMA node. If a process touches a 2 MiB region from CPUs on multiple NUMA nodes (because it spawned worker threads on the other socket), every access from the wrong socket pays the cross-UPI penalty for the entire 2 MiB. Under 4 KiB pages, the penalty is at most for one 4 KiB page at a time — the kernel's first-touch policy has finer granularity.

This is one of those "THP made it worse" cases that's almost impossible to spot without numastat -p showing the imbalance.

When madvise is the right answer

madvise lets the application opt in. Modern compute libraries (PyTorch, JAX, CUDA itself for some allocations) call madvise(MADV_HUGEPAGE) on their large arenas. Those get THP. Everything else (the random little allocations from Python imports, monitoring agent metrics buffers, kubelet inotify caches) stays on 4 KiB and doesn't pay the khugepaged tax.

Set it via two paths.

GRUB cmdline (persistent, takes effect at boot)

GRUB_CMDLINE_LINUX_DEFAULT="... transparent_hugepage=madvise ..."

Then update-grub && reboot.

Runtime (no reboot, lost on reboot)

echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
# also for defrag
echo defer+madvise | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

Most production HPC nodes use enabled=madvise and defrag=defer+madvise. The defer part means even un-madvised allocations will eventually get THP from khugepaged in the background, but no allocation blocks for it.

When never is correct

never = THP off, period. Use only when:

1. You measured a real-time / latency-critical workload (e.g., financial market data) where ANY khugepaged activity is unacceptable.
2. A specific buggy workload trips a kernel bug under THP. These exist; check the kernel changelog.
3. You're using huge explicit hugetlb pages already and don't want THP fighting with them.

never costs you ~5-10% of TLB-bound performance compared to madvise on HPC kernels. It's a real tax.

echo never | sudo tee /sys/kernel/mm/transparent_hugepage/enabled

Detecting THP-induced stalls

/proc/vmstat exposes the THP counters. The right ones to watch:

grep thp /proc/vmstat
# thp_fault_alloc 12345                    <-- THP allocated on fault
# thp_fault_fallback 6789                  <-- couldn't, fell back to 4 KiB
# thp_fault_fallback_charge 0              <-- couldn't due to memcg charge
# thp_collapse_alloc 567                   <-- khugepaged successfully promoted
# thp_collapse_alloc_failed 12             <-- khugepaged tried but couldn't
# thp_split_page 89                        <-- THP split back to 4 KiB
# thp_split_pmd 1234                       <-- pmd split
# thp_split_pud 0
# thp_zero_page_alloc 3
# thp_zero_page_alloc_failed 0
# thp_swpout 0
# thp_swpout_fallback 0

Rates (delta per second) tell the story. Watch them with:

# Sample every 5 seconds
while sleep 5; do
  grep thp /proc/vmstat | awk '{print $1, $2}'
  echo ---
done

If thp_collapse_alloc is running at thousands per second under steady-state load, khugepaged is busy promoting pages and you should expect latency jitter. Setting enabled=madvise will stop almost all of that.

thp_split_page rising means the kernel is splitting THP back to 4 KiB — happens when memory gets pinned (RDMA registration), when fork+CoW touches it, or when the kernel needs smaller pages for memory pressure relief. Splits are also expensive; if you have lots of them, your THP is doing harm not good.

khugepaged knobs

ls /sys/kernel/mm/transparent_hugepage/khugepaged/
# alloc_sleep_millisecs           defrag                pages_collapsed
# full_scans                       max_ptes_none         pages_to_scan
# pages_swap                       scan_sleep_millisecs

cat /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs
# 10000                                  <-- 10 s between scans

cat /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan
# 4096                                   <-- pages per scan

To reduce khugepaged aggressiveness without disabling it:

# Less frequent scans
echo 60000 > /sys/kernel/mm/transparent_hugepage/khugepaged/scan_sleep_millisecs

# Smaller scans
echo 256 > /sys/kernel/mm/transparent_hugepage/khugepaged/pages_to_scan

But if you've gone to madvise mode this rarely matters — khugepaged only touches MADV'd ranges anyway.

Per-process THP usage — smaps_rollup

Want to know whether a specific process is actually using THP?

cat /proc/<pid>/smaps_rollup
# 56400000-7ffe... [rollup]
# Rss:               123456 kB
# Pss:                98765 kB
# AnonHugePages:      45678 kB        <-- THP-backed anon, in KB
# ShmemPmdMapped:         0 kB
# FilePmdMapped:          0 kB
# Shared_Hugetlb:         0 kB
# Private_Hugetlb:        0 kB
# ...

AnonHugePages > 0 means the process is using THP. Big numbers (gigabytes) mean it's correctly opting in via madvise. Zero on a large allocator-heavy process running under enabled=madvise means it's not opting in (which is fine — it's just on 4 KiB pages).

For comparison, under enabled=always, every process will show some AnonHugePages. That's the THP-everywhere mode you usually don't want.

Workloads that benefit vs hurt under THP

Helps:

• Large in-memory workloads with predictable working sets (some KV-stores, some MD codes)
• Code that walks a big hash table sequentially (TLB-bound)
• HBM/GPU host-side staging buffers (large, contiguous, reused)
• DPDK / userspace networking (already pinned via hugetlb usually, but bonus)

Hurts:

• Latency-sensitive HPC where any jitter matters (NCCL collectives)
• RDMA workloads with frequent ibv_reg_mr calls (page splits during pinning)
• Mixed-NUMA processes that touch the same buffer from both sockets
• Memory-pressured systems where compaction stalls are bad

Kubelet and THP

If your K8s nodes ship with enabled=always, the kubelet itself can be slow on busy nodes. Worth setting via DaemonSet at boot time, or as part of the node bootstrap.

# A trivial DaemonSet that just enforces THP=madvise on every node
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: thp-set-madvise
spec:
  selector:
    matchLabels: { name: thp-set-madvise }
  template:
    metadata:
      labels: { name: thp-set-madvise }
    spec:
      hostPID: true
      containers:
      - name: thp
        image: alpine:3
        securityContext:
          privileged: true
        command: ["/bin/sh", "-c"]
        args:
        - |
          echo madvise > /host/sys/kernel/mm/transparent_hugepage/enabled
          echo defer+madvise > /host/sys/kernel/mm/transparent_hugepage/defrag
          # stay running so the DS isn't restarted forever
          sleep infinity
        volumeMounts:
        - name: host-sys
          mountPath: /host/sys
      volumes:
      - name: host-sys
        hostPath: { path: /sys }

Permanent fix is the GRUB cmdline; the DS is for emergency rollouts when you can't reboot.

Troubleshooting

Symptom: latency spikes with no other obvious cause

# Watch /proc/vmstat thp counters during the workload
while sleep 1; do
  date +%H:%M:%S
  grep -E 'thp_collapse|thp_split|compact_' /proc/vmstat
  echo ---
done | tee /tmp/thp.log

Spikes in thp_collapse_alloc or thp_split_* correlated with the latency events confirm THP. Switch to madvise.

Symptom: INFO: task ... blocked for ... on memory-touching syscalls

dmesg -T | grep -i 'blocked\|compact'
# [time] INFO: task python:12345 blocked for more than 120 seconds.
# [time] EXT4-fs (nvme0n1p1): ... ;
# [time] compaction stalled by ...

If compact_stall and compact_fail in /proc/vmstat are growing at hundreds per second, you're under heavy memory pressure trying to make hugepages out of fragmented memory.

grep compact /proc/vmstat
# compact_migrate_scanned   1234567
# compact_free_scanned      89012345
# compact_isolated          234567
# compact_stall              45678   <-- direct compaction blocking processes
# compact_fail               12345
# compact_success             6789
# compact_daemon_wake          234

Under defrag=defer+madvise (or just madvise), compact_stall should be near zero — direct compaction in fault path is what defer avoids.

Symptom: RDMA ibv_reg_mr slow or fails intermittently

dmesg -T | grep -i 'split\|mlx5\|hugepage'
# Could include: "mlx5_core: failed to register MR" sporadically

# Counter
grep thp_split /proc/vmstat
# thp_split_page 12345           <-- if this rises during the failures, suspect THP

RDMA pin (via get_user_pages) can require splitting THPs. Under heavy concurrent registrations, splits race with khugepaged collapses. The cleanest fix is enabled=madvise so the application's RDMA buffers (which often DON'T madvise) stay on 4 KiB pages, no splits needed.

Symptom: numastat -p <pid> shows huge imbalance

numastat -p $(pgrep python)
# Node 0 1500 MB  Node 1 8500 MB     <-- 5x asymmetry

If your process is bound to node 0 but has more memory on node 1, two suspects:

1. Worker threads spawned without NUMA binding (see NUMA)
2. THP allocated 2 MiB pages on the wrong node, and khugepaged hasn't migrated them

If enabled=always plus suspect 1, the imbalance is amplified by 512x because each wrong allocation is 2 MiB instead of 4 KiB. madvise mitigates by going back to fine-grained first-touch on most allocations.

Symptom: a specific kernel version regresses on THP

Search the kernel mailing list / linux-mm archives for THP regressions. Several were fixed in 5.10, 5.15, 6.1. If you're on a stale kernel, just upgrading to a current LTS often fixes the issue without needing to disable THP.

uname -r
# 5.4.0-150-generic            <-- old, several known THP issues

# Test by setting madvise and re-running. If problem persists across
# enabled=never and enabled=madvise, it's not really THP, it's something else.

Validate the change took effect

# After GRUB change + reboot
cat /proc/cmdline | grep -o transparent_hugepage=\\S*
# transparent_hugepage=madvise

cat /sys/kernel/mm/transparent_hugepage/enabled
# always [madvise] never

If the cmdline shows madvise but /sys shows always, something else (a tuned profile, a custom systemd unit) is overwriting at runtime. Find it:

grep -r transparent_hugepage /etc/ /usr/lib/ 2>/dev/null
# /etc/tuned/<active>/tuned.conf:transparent_hugepages=always

Edit or disable that.

See also

• Sysctl tuning — vm.compaction_proactiveness, related VM knobs
• Hugepages — explicit 1G hugetlb (the non-transparent kind)
• GRUB cmdline — transparent_hugepage=madvise boot parameter
• NUMA — THP amplifies cross-NUMA penalties
• Kernel troubleshooting playbook — broader memory diagnostics

External:

• Documentation/admin-guide/mm/transhuge.rst (Linux kernel)
• Documentation/admin-guide/mm/concepts.rst (Linux kernel)
• man madvise — MADV_HUGEPAGE flag
• LWN: "Transparent huge pages and lessons learned" (lwn.net/Articles/698329)