Sysctl tuning for GPU and HPC nodes

The kitchen-sink list of sysctls every GPU/HPC node should ship with — what each one does, sane values, and where to put them.

▶Try the commands on this page in the command emulator — type help for the full list, or solutions for copy-paste fix recipes.

The kernel ships with values that target a generic 4-core laptop. A GPU node has 192 cores, 1 TiB of RAM, four 400 Gbps NICs, and a few hundred thousand inotify watches from the agent stack. Roughly thirty sysctls move the defaults toward what HPC actually needs.

This page is the reference list, organized by subsystem. Drop the values into /etc/sysctl.d/ (one logical file per subsystem) and run sysctl --system to apply.

Where to put them

Order matters: files are loaded alphabetically across /etc/sysctl.d/, /run/sysctl.d/, /usr/lib/sysctl.d/, and /etc/sysctl.conf. Last write wins for any conflicting key. Convention:

/etc/sysctl.d/10-network.conf      # net.*
/etc/sysctl.d/20-vm.conf           # vm.*
/etc/sysctl.d/30-fs.conf           # fs.*
/etc/sysctl.d/40-kernel.conf       # kernel.*
/etc/sysctl.d/99-overrides.conf    # one-off final overrides

Apply without reboot:

sysctl --system
# * Applying /usr/lib/sysctl.d/00-system.conf ...
# * Applying /etc/sysctl.d/10-network.conf ...
# ...

# Verify a specific key actually took
sysctl net.core.rmem_max
# net.core.rmem_max = 268435456

sysctl --system is idempotent and is what you want from config-management. sysctl -p somefile.conf only reads one file and silently misses ordering.

Network — TCP buffers and queue depth

Big-pipe NICs (100/200/400 Gbps) need much larger socket buffers and a deeper qdisc backlog than the defaults give you.

# /etc/sysctl.d/10-network.conf

# Maximum socket buffer size apps can request via setsockopt
net.core.rmem_max          = 268435456   # 256 MiB
net.core.wmem_max          = 268435456
net.core.rmem_default      = 16777216    # 16 MiB
net.core.wmem_default      = 16777216

# Backlog at the device layer before the protocol stack picks up
net.core.netdev_max_backlog = 250000

# Optmem (per-socket auxiliary memory, e.g. SCM rights)
net.core.optmem_max         = 67108864

# TCP autotuning: min, default, max (in bytes)
net.ipv4.tcp_rmem           = 4096 87380 268435456
net.ipv4.tcp_wmem           = 4096 65536 268435456

# Global TCP memory pressure thresholds (in pages — 4 KiB each)
# low / pressure / max — let the stack use up to several GB before throttling
net.ipv4.tcp_mem            = 4096 87380 6291456

# BBR is the right default congestion control for fat-pipe long-RTT
net.ipv4.tcp_congestion_control = bbr
net.core.default_qdisc          = fq

# Disable slow-start after idle (don't reset cwnd between bursts)
net.ipv4.tcp_slow_start_after_idle = 0

# More open ports for many short-lived connections
net.ipv4.ip_local_port_range = 1024 65535

# TIME-WAIT and FIN-WAIT — keep modest, not aggressive
net.ipv4.tcp_fin_timeout    = 15
net.ipv4.tcp_tw_reuse       = 1

# SYN cookies stay on (DDoS protection); SYN backlog deeper
net.ipv4.tcp_syncookies     = 1
net.ipv4.tcp_max_syn_backlog = 8192

Validate the running stack:

sysctl net.ipv4.tcp_congestion_control net.core.default_qdisc
ss -tin | head           # tcp_info, look for cwnd, rtt, send-Q

# Confirm BBR is loaded
cat /proc/sys/net/ipv4/tcp_available_congestion_control
# reno cubic bbr

If bbr isn't listed, load the module: modprobe tcp_bbr and add tcp_bbr to /etc/modules-load.d/.

VM (memory management)

# /etc/sysctl.d/20-vm.conf

# Critical for big-process workloads (PyTorch, training jobs with many threads).
# Default 65530 is way too low.
vm.max_map_count          = 1048576

# Don't swap unless we genuinely need to — HPC nodes should have swap-off
# behavior even if a small swap exists for emergency.
vm.swappiness             = 1

# Dirty page thresholds — lower than default, write back sooner.
# On 1 TiB nodes, default 20% means 200 GiB dirty before forcing writeback.
vm.dirty_ratio            = 10
vm.dirty_background_ratio = 5
vm.dirty_expire_centisecs = 3000

# Disable per-NUMA-node reclaim on first allocation (lets allocations spill
# to remote NUMA rather than reclaiming local pages — almost always what you want).
vm.zone_reclaim_mode      = 0

# Overcommit: 1 = always allow (typical for HPC, lots of fork+exec).
# 2 = strict, refuses overcommit (use only if you've sized swap carefully).
vm.overcommit_memory      = 1

# OOM panic — controversial. Off (default) means OOM-killer picks a victim.
# Some HPC sites prefer panic so the job gets requeued; we leave it default.
vm.panic_on_oom           = 0

# Min free memory in KB — keep enough for the kernel under pressure
vm.min_free_kbytes        = 4194304   # 4 GiB on a 1 TiB node

vm.max_map_count = 1048576 is the one that bites silently. PyTorch can blow past 65k mappings during a multi-process DataLoader run; you'll see OSError: [Errno 12] Cannot allocate memory from a perfectly healthy node with 800 GiB free. The error is a lie — it's the mapping count.

Filesystem and inotify

# /etc/sysctl.d/30-fs.conf

# System-wide max open file descriptors
fs.file-max               = 16777216

# Per-user inotify watches — kubelet, containerd, prometheus, fluentbit all
# burn through these. 8192 default = explosion.
fs.inotify.max_user_watches    = 1048576
fs.inotify.max_user_instances  = 8192
fs.inotify.max_queued_events   = 32768

# AIO (used by some training stacks for async disk I/O)
fs.aio-max-nr             = 1048576

# Allow non-root to use BPF (for bpftrace, etc) — set only if you trust users
# kernel.unprivileged_bpf_disabled = 0

# nr_open = max FDs per process (paired with ulimit -n)
fs.nr_open                = 16777216

The inotify defaults are the cause of most "Kubernetes pod won't restart, exec-into is laggy" issues on busy nodes. Bump them once and forget.

Kernel scheduling and core behavior

# /etc/sysctl.d/40-kernel.conf

# AutoNUMA balancing — moves pages between NUMA nodes based on access patterns.
# Great for generic workloads, BAD for HPC (causes mysterious page faults
# during steady-state). Disable for compute clusters.
kernel.numa_balancing     = 0

# Allow non-root processes to use real-time priorities (only if you need them)
# kernel.sched_rt_runtime_us = -1   # uncomment for -1 = unlimited RT slice

# pid_max: default 32768 is fine for laptops, way too low for big nodes.
kernel.pid_max            = 4194304

# Threads max — same reasoning
kernel.threads-max        = 4194304

# Kernel panic behavior — useful for unattended fleet, panics reboot in 30s
kernel.panic              = 30
kernel.panic_on_oops      = 1

# Core dumps go to systemd-coredump (off-disk by default)
kernel.core_pattern       = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h

# perf_event_paranoid: 2 default blocks unprivileged perf. -1 lets all users
# profile (set this on dev nodes, leave restrictive on shared clusters).
kernel.perf_event_paranoid = 2

# Lower the cost of context switches on many-core boxes
kernel.sched_min_granularity_ns       = 10000000   # 10 ms
kernel.sched_wakeup_granularity_ns    = 15000000   # 15 ms
kernel.sched_migration_cost_ns        = 5000000    # 5 ms

kernel.numa_balancing = 0 is the single biggest improvement you'll get on a static-job HPC cluster. The migration daemon's THP-splitting and page-moving cause unpredictable latency spikes during NCCL collectives.

RDMA / IB-specific (if applicable)

These show up in modules-conf or sysctl on top of OFED installations:

# /etc/sysctl.d/50-rdma.conf

# Increase max user verbs context (RDMA verbs queue depth budget)
# Most distros set this in the OFED package; only override if needed.

# IPoIB MTU and fragmentation
net.ipv4.ip_forward       = 1                    # enable forwarding for VxLAN/IPoIB bridging
net.ipv6.conf.all.disable_ipv6 = 0               # leave v6 on (or off if your fabric is v4)

Most RDMA tuning is in modprobe options and /etc/modules-load.d/, not sysctl. See the RDMA modules section for that.

Putting it all together

A reasonable ship-it-by-default file structure for a fresh GPU node:

/etc/sysctl.d/
├── 10-network.conf      # ~20 lines, the buffers + BBR + qdisc block above
├── 20-vm.conf           # ~10 lines, max_map_count + dirty + numa-zone
├── 30-fs.conf           # ~5 lines, inotify + file-max
├── 40-kernel.conf       # ~10 lines, numa_balancing off + pid_max
└── 99-site-overrides.conf  # whatever your specific cluster needs

Every value here is observable. sysctl <key> reads it back; cat /proc/sys/<dotted/path> reads it raw.

Validation cheatsheet

# Network
sysctl net.core.rmem_max net.ipv4.tcp_congestion_control net.core.default_qdisc

# VM
sysctl vm.max_map_count vm.swappiness vm.zone_reclaim_mode

# FS
sysctl fs.file-max fs.inotify.max_user_watches

# Kernel
sysctl kernel.numa_balancing kernel.pid_max

# Confirm /etc/sysctl.d/* actually loaded
sysctl --system 2>&1 | grep -E "Applying|kernel\.|net\.|vm\.|fs\." | head -50

When the value won't stick

sysctl --system says Applying, but sysctl <key> shows the old value. Three causes:

Read-only key under your kernel config. Some keys (e.g., kernel.unprivileged_userns_clone on some distros) are compiled out. sysctl will silently fail. Check dmesg | grep -i sysctl.
Module not loaded. net.ipv4.tcp_bbr_* doesn't exist until tcp_bbr is loaded. Add it to /etc/modules-load.d/.
Conflicting later file. A later-alphabetical file under /etc/sysctl.d/ is overriding yours. grep -r '<key>' /etc/sysctl.d/ /usr/lib/sysctl.d/ /run/sysctl.d/.

TCP fine-grain — knobs the buffers block didn't cover

A handful of TCP keys that aren't about buffer sizing but still matter for HPC traffic patterns (storage RPC, NCCL out-of-band sockets, control plane).

# /etc/sysctl.d/11-tcp-tuning.conf

# Window scaling — REQUIRED for any window > 64 KiB (so always, on modern fabrics)
net.ipv4.tcp_window_scaling      = 1

# Don't save metrics across connections. Caching RTT/cwnd from one connection
# poisons the next, especially on jobs that come and go on the same peer.
net.ipv4.tcp_no_metrics_save     = 1

# Selective ACK — keep on. tcp_dsack adds duplicate-SACK for better loss recovery.
net.ipv4.tcp_sack                = 1
net.ipv4.tcp_dsack               = 1
net.ipv4.tcp_fack                = 1

# tcp_low_latency: deprecated post-4.14, but still readable. Modern stack
# always uses the same path. Leave at default 0.
# net.ipv4.tcp_low_latency       = 0

# Keepalive — much shorter than default 7200s. Critical for long-running jobs
# behind NAT or stateful firewalls that drop idle flows.
net.ipv4.tcp_keepalive_time      = 600
net.ipv4.tcp_keepalive_intvl     = 30
net.ipv4.tcp_keepalive_probes    = 5

# MTU probing — let TCP discover path MTU instead of relying on PMTUD ICMP
# (which black-holed firewalls often eat). 1 = on, 2 = always probe.
net.ipv4.tcp_mtu_probing         = 1

# Don't suppress timestamps — needed for PAWS on long flows
net.ipv4.tcp_timestamps          = 1

# Increase orphan FDs (sockets in CLOSE-WAIT after the proc has gone away)
net.ipv4.tcp_max_orphans         = 65536

# RFC 6928 — initial congestion window 10 segments (modern default, but pin it)
# net.ipv4.tcp_init_cwnd is implicit in the kernel; nothing to set.

# Reuse TIME-WAIT sockets aggressively for outbound (already in main block)
# net.ipv4.tcp_tw_reuse          = 1

# Allow more TW slots — busy clients hit this on fan-out
net.ipv4.tcp_max_tw_buckets      = 2000000

# Enable Early Retransmit / Tail Loss Probe (default on for years, but explicit)
net.ipv4.tcp_early_retrans       = 3

Why tcp_no_metrics_save=1 matters on HPC: if a previous job had a flaky cross-rack flow and the cache stuck a small cwnd ceiling on that peer, the next job inherits it. The symptom is "the same all_reduce that was 380 GB/s yesterday is 90 GB/s today, restart fixes it".

Picking the congestion control

bbr is the right default for fat-pipe long-RTT, but it isn't always best:

CC algorithm	Use when
`bbr`	Fat-pipe (>= 25 Gbps), long-RTT (cross-DC), some packet loss tolerated
`bbr2`	Same as bbr, more friendly to non-bbr competing flows. Newer kernels.
`cubic`	Low-RTT, low-loss LAN. Good default for in-rack traffic on commodity.
`htcp`	Niche — academic HPC, very-long-RTT and zero loss. Rarely picked.
`dctcp`	Datacenter with ECN-marking switches. Almost nobody runs this.

# What's loaded vs available
cat /proc/sys/net/ipv4/tcp_allowed_congestion_control
# reno cubic bbr

cat /proc/sys/net/ipv4/tcp_available_congestion_control
# reno cubic bbr bbr2 htcp dctcp

# Switch live without reboot (only takes effect on new sockets)
sysctl -w net.ipv4.tcp_congestion_control=bbr

To check which CC a specific socket actually uses:

ss -tin | grep -A1 cwnd
# ESTAB    0   0   10.x.x.x:35421   10.x.x.y:443
#  bbr wscale:7,7 rto:204 rtt:1.234/0.456 ato:40 ...

The first word of the second line is the CC algorithm.

VM — the rest of the memory tunables

# /etc/sysctl.d/21-vm-extra.conf

# vfs_cache_pressure: how aggressively to reclaim dentry/inode cache.
# Default 100 = balance with pagecache. HPC nodes that touch billions of files
# (MD trajectories, datasets in many small chunks) want LOWER (50) to keep
# the dentry cache hot. Nodes that read few large files want HIGHER (200) so
# pagecache wins.
vm.vfs_cache_pressure            = 50

# Watermark scale factor — how aggressively kswapd reclaims relative to total.
# Higher = kswapd wakes earlier, less direct reclaim by the calling process.
vm.watermark_scale_factor        = 100   # default 10, units of 10000ths

# Compaction: let kcompactd run, but throttle proactive compaction
# (it can stall RDMA pinning if it tries to move pages we just registered).
vm.compaction_proactiveness      = 0     # default 20

# Prevent oom-killer from picking critical processes (set per-process oom_score_adj)
# Setting via sysctl only useful for vm.oom_dump_tasks (verbose OOM logs)
vm.oom_dump_tasks                = 1

# Stat interval — default 1s, fine. Raise to 10 if you have many CPUs and
# vmstat-collection overhead shows up.
# vm.stat_interval               = 1

# Disable memory hot-add (we never use it; reduces a small attack surface)
# vm.memory_failure_recovery     = 1

# Page-cluster — readahead for swap. With swap-off / swappiness=1, irrelevant.
vm.page-cluster                  = 0

vm.compaction_proactiveness=0 is the one that bites RDMA workloads. Default 20 makes kcompactd wake up periodically and migrate pages around — which fights with ibv_reg_mr()-pinned regions and can produce intermittent cannot allocate memory from RDMA registrations even with terabytes free.

Kernel scheduler — fine knobs

The scheduler section above sets the basics. A few more for compute nodes:

# /etc/sysctl.d/41-sched.conf

# Migration cost: how long after migrating a task before the scheduler will
# migrate it again. Higher = stickier (good for compute), lower = more
# aggressive load-balancing (good for many short tasks).
kernel.sched_migration_cost_ns         = 5000000     # 5 ms

# Min scheduling granularity. The CFS won't preempt a task earlier than this.
# Higher = fewer context switches, longer per-task runtime, better for HPC.
kernel.sched_min_granularity_ns        = 10000000    # 10 ms

# Wakeup granularity — bigger = woken tasks have to wait longer to preempt
# the running one. Reduces wakeup-driven context switches.
kernel.sched_wakeup_granularity_ns     = 15000000    # 15 ms

# Latency: the targeted period over which all runnable tasks should run once.
# Default 6 ms (NR_LATENCY_NS / 1M). Raise for compute nodes.
# kernel.sched_latency_ns              = 24000000    # 24 ms (autocomputed)

# Schedstats — per-runqueue statistics. Off by default in some kernels;
# turn on if you want to read /proc/<pid>/sched details.
kernel.sched_schedstats                = 1

# Disable autogroup (groups every TTY session into a separate sched group).
# Useful on desktops, useless on headless compute. Off = simpler reasoning.
kernel.sched_autogroup_enabled         = 0

The trio migration_cost_ns / min_granularity_ns / wakeup_granularity_ns is what tuned-adm calls sched_migration_cost in the latency-performance and throughput-performance profiles. Setting them ourselves means we don't depend on tuned running.

Under load, watch:

# Context switches per second (system-wide)
vmstat 1 5
# procs ----------memory--------- ---swap--  -----io----  -system-- ------cpu------
#  r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
#  3  0      0  890G    12K  120G     0    0     0     0  3.5k  4.2k 88  3  9  0  0

# The 'cs' column is context-switches/sec. >100k on a busy compute node is fine;
# >1M is a sign of scheduler thrash.

# Per-process schedstats
cat /proc/<pid>/sched
# se.exec_start                         :  12345.6
# se.vruntime                           :   2398.3
# nr_switches                           :    45678   <- look at growth rate
# nr_voluntary_switches                 :    12345
# nr_involuntary_switches               :    33333   <- preempted

A high nr_involuntary_switches rate means another task keeps preempting yours. On an isolated HPC node, that should be near zero — if it isn't, you have a stray service stealing the core (kworker, irqbalance, monitoring agent).

IPC — shared memory for MPI

MPI implementations (OpenMPI, MPICH, MVAPICH) use SysV or POSIX shared memory for intra-node messaging. The defaults are too small.

# /etc/sysctl.d/45-ipc.conf

# SysV shmmax: max single shared-memory segment, in bytes.
# Default on modern kernels is already huge (ULONG_MAX), but some distros
# still ship 4 GiB. Set explicitly. Half of RAM is a sane upper bound.
kernel.shmmax              = 549755813888    # 512 GiB

# SysV shmall: total shared memory pages (across all segments), in 4 KiB units.
# 134217728 pages = 512 GiB
kernel.shmall              = 134217728

# Max segments per user. Default 4096 is fine but grow if MPI complains.
kernel.shmmni              = 4096

# Semaphore params: SEMMSL SEMMNS SEMOPM SEMMNI
# (max sems per array / total sems / ops per semop / max sem arrays)
kernel.sem                 = 32000 1024000000 500 32000

# Message queues
kernel.msgmnb              = 65536
kernel.msgmax              = 65536
kernel.msgmni              = 32768

Verify SysV state:

ipcs -lm   # shared memory limits
# ------ Shared Memory Limits --------
# max number of segments = 4096
# max seg size (kbytes) = 536870912
# max total shared memory (kbytes) = 536870912

ipcs -ls   # semaphore limits
ipcs -lq   # message queue limits

# Actual usage
ipcs -m | head    # active SysV shm segments

If MPI startup is slow or MPI_Init fails with "shmget: No space left on device", kernel.shmmax or shmall is the cause 90% of the time.

Transparent Huge Pages (THP) — the sysctl-adjacent bits

THP isn't strictly a sysctl; it's /sys/kernel/mm/transparent_hugepage/*. But it's controlled at the same layer and people look here first.

# Read current state
cat /sys/kernel/mm/transparent_hugepage/enabled
# always [madvise] never            <- bracketed = active

cat /sys/kernel/mm/transparent_hugepage/defrag
# always defer defer+madvise [madvise] never

# Set runtime (lost on reboot — also set transparent_hugepage=madvise on GRUB cmdline)
echo madvise > /sys/kernel/mm/transparent_hugepage/enabled
echo madvise > /sys/kernel/mm/transparent_hugepage/defrag

# khugepaged metrics — the daemon that promotes 4 KiB pages into 2 MiB
grep thp /proc/vmstat
# thp_fault_alloc 12345
# thp_fault_fallback 6789
# thp_collapse_alloc 567
# thp_collapse_alloc_failed 12
# thp_split_page 89
# thp_split_pmd 1234
# thp_zero_page_alloc 3

Full discussion: Transparent Huge Pages.

Connection tracking — when conntrack table fills

Even on compute nodes you may inherit a kube-proxy / iptables stack that conntracks every flow. On a busy node with hundreds of pods doing fanout, nf_conntrack: table full, dropping packet is a real failure mode.

# /etc/sysctl.d/12-conntrack.conf

# Bump conntrack table size — default ~262144 entries
net.netfilter.nf_conntrack_max          = 2097152
net.netfilter.nf_conntrack_buckets      = 524288

# TCP-established timeout — default 5 days. Lower it if you see leakage.
net.netfilter.nf_conntrack_tcp_timeout_established = 86400      # 1 day

# Don't conntrack outbound from host network namespace (if your CNI lets you)
# (this is per-rule via -j NOTRACK in raw table, not a sysctl)

Read current usage:

sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max
# net.netfilter.nf_conntrack_count = 145678
# net.netfilter.nf_conntrack_max = 2097152

# Top conntrack entries by something
conntrack -L | wc -l
conntrack -L -p tcp --state ESTABLISHED | wc -l

# When the table fills you'll see this in dmesg:
dmesg | grep -i conntrack
# nf_conntrack: table full, dropping packet

Drop-in file — the everything-at-once

Save as /etc/sysctl.d/99-gpu-hpc.conf. This is the file you copy to a fresh node. Each section can also be split into separate files following the 10-network, 20-vm, etc. convention; the single file is for "I want to bootstrap this node in 30 seconds".

# /etc/sysctl.d/99-gpu-hpc.conf
# Single-file production tuning for an HPC GPU node.
# Apply with: sysctl --system

# ============================================================================
# NETWORK — TCP buffers, BBR, fast sockets
# ============================================================================
net.core.rmem_max                = 268435456
net.core.wmem_max                = 268435456
net.core.rmem_default            = 16777216
net.core.wmem_default            = 16777216
net.core.netdev_max_backlog      = 250000
net.core.optmem_max              = 67108864
net.core.somaxconn               = 65535
net.core.default_qdisc           = fq

net.ipv4.tcp_rmem                = 4096 87380 268435456
net.ipv4.tcp_wmem                = 4096 65536 268435456
net.ipv4.tcp_mem                 = 4096 87380 6291456
net.ipv4.tcp_congestion_control  = bbr
net.ipv4.tcp_slow_start_after_idle = 0
net.ipv4.tcp_window_scaling      = 1
net.ipv4.tcp_no_metrics_save     = 1
net.ipv4.tcp_sack                = 1
net.ipv4.tcp_fack                = 1
net.ipv4.tcp_dsack               = 1
net.ipv4.tcp_timestamps          = 1
net.ipv4.tcp_mtu_probing         = 1
net.ipv4.tcp_fin_timeout         = 15
net.ipv4.tcp_keepalive_time      = 600
net.ipv4.tcp_keepalive_intvl     = 30
net.ipv4.tcp_keepalive_probes    = 5
net.ipv4.tcp_tw_reuse            = 1
net.ipv4.tcp_max_tw_buckets      = 2000000
net.ipv4.tcp_max_syn_backlog     = 8192
net.ipv4.tcp_max_orphans         = 65536
net.ipv4.tcp_syncookies          = 1
net.ipv4.ip_local_port_range     = 1024 65535

# Conntrack (only relevant if netfilter is loaded; harmless otherwise)
net.netfilter.nf_conntrack_max   = 2097152
net.netfilter.nf_conntrack_tcp_timeout_established = 86400

# ============================================================================
# VM — memory management
# ============================================================================
vm.max_map_count                 = 1048576
vm.swappiness                    = 1
vm.dirty_ratio                   = 10
vm.dirty_background_ratio        = 5
vm.dirty_expire_centisecs        = 3000
vm.zone_reclaim_mode             = 0
vm.overcommit_memory             = 1
vm.panic_on_oom                  = 0
vm.min_free_kbytes               = 4194304
vm.vfs_cache_pressure            = 50
vm.watermark_scale_factor        = 100
vm.compaction_proactiveness      = 0
vm.oom_dump_tasks                = 1
vm.page-cluster                  = 0

# ============================================================================
# FS — file descriptors and inotify
# ============================================================================
fs.file-max                      = 16777216
fs.nr_open                       = 16777216
fs.aio-max-nr                    = 1048576
fs.inotify.max_user_watches      = 1048576
fs.inotify.max_user_instances    = 8192
fs.inotify.max_queued_events     = 32768

# ============================================================================
# KERNEL — scheduler, NUMA, panic, perf
# ============================================================================
kernel.numa_balancing            = 0
kernel.pid_max                   = 4194304
kernel.threads-max               = 4194304
kernel.panic                     = 30
kernel.panic_on_oops             = 1
kernel.perf_event_paranoid       = 2
kernel.sched_migration_cost_ns   = 5000000
kernel.sched_min_granularity_ns  = 10000000
kernel.sched_wakeup_granularity_ns = 15000000
kernel.sched_schedstats          = 1
kernel.sched_autogroup_enabled   = 0

# ============================================================================
# IPC — shared memory and semaphores for MPI
# ============================================================================
kernel.shmmax                    = 549755813888
kernel.shmall                    = 134217728
kernel.shmmni                    = 4096
kernel.sem                       = 32000 1024000000 500 32000
kernel.msgmnb                    = 65536
kernel.msgmax                    = 65536
kernel.msgmni                    = 32768

Apply:

sudo cp 99-gpu-hpc.conf /etc/sysctl.d/
sudo sysctl --system | grep -E '^\* Applying' | tail
sudo sysctl --system 2>&1 | grep -E '^kernel\.|^vm\.|^net\.|^fs\.' | head -50

Troubleshooting — finding which sysctl is misbehaving

Most "the kernel feels off" symptoms map to a small handful of sysctls. Walk this list before opening a kernel ticket.

Symptom: "Cannot allocate memory" with 800 GiB free

# Almost always vm.max_map_count
sysctl vm.max_map_count
# vm.max_map_count = 65530          <-- too low

# Confirm by counting the offending process's mappings
cat /proc/<pid>/maps | wc -l
# 65521                              <-- right at the edge

# Also check vm.overcommit_memory (2 = strict, can refuse otherwise-fine allocs)
sysctl vm.overcommit_memory

Bump vm.max_map_count = 1048576, no restart needed for new processes (existing ones already passed the check).

Symptom: dmesg flood of `nf_conntrack: table full`

dmesg -T | grep -i conntrack | tail
# [Mon May 5 14:23:11 2026] nf_conntrack: table full, dropping packet

sysctl net.netfilter.nf_conntrack_count net.netfilter.nf_conntrack_max
# count is at or near max

Bump net.netfilter.nf_conntrack_max. If it keeps refilling, you have a flood — find the source with conntrack -L | sort | uniq -c | sort -rn | head.

Symptom: "Too many open files" but ulimit says 1M

# Check the process-level limit
cat /proc/<pid>/limits | grep -i 'open files'
# Max open files            1048576              1048576              files

# Check global ceiling
sysctl fs.nr_open fs.file-max
# fs.nr_open = 1048576       <-- per-process kernel ceiling
# fs.file-max = 16777216     <-- system-wide

# Count actual open FDs across the system
cat /proc/sys/fs/file-nr
# 12345  0  16777216         <-- allocated / unused / max

If the kernel ceiling (fs.nr_open) is below the systemd LimitNOFILE you set, the LimitNOFILE is silently capped. Bump fs.nr_open to match.

Symptom: TCP throughput is half of what `ib_send_bw` shows

# Buffer sizes — are they actually applied?
sysctl net.core.rmem_max net.core.wmem_max net.ipv4.tcp_rmem net.ipv4.tcp_wmem

# Is the app actually requesting big buffers?
ss -tin dst <peer> | head -3
# look at "rcv_space" and "snd_wnd"; if they're stuck at 87380, the app
# isn't calling setsockopt SO_RCVBUF and the autotuner won't grow past
# the default. The only way to push it higher is to raise tcp_rmem max.

# Window-scaling negotiated?
ss -tin dst <peer> | head -3
# wscale:7,7  <-- both directions; 0,0 means scaling didn't get negotiated

If wscale is 0, a middlebox stripped the option from SYN. Force tcp_window_scaling=1 on both sides (already default) and complain to whoever runs the middlebox.

Symptom: under load, OOM-killer fires even with free memory

dmesg -T | grep -i 'killed process'
# [time] Out of memory: Killed process 12345 (python) total-vm:128GB

# Was it a NUMA imbalance?
cat /proc/buddyinfo
# Node 0, zone DMA32  ... hundreds of zero-order pages
# Node 1, zone DMA32  ... <-- maybe one node is starved

# Was vm.zone_reclaim_mode stuck at 1?
sysctl vm.zone_reclaim_mode
# vm.zone_reclaim_mode = 1   <-- WRONG, should be 0

vm.zone_reclaim_mode = 1 makes the kernel reclaim aggressively from the local NUMA node before spilling to the other. On a 2-socket box with one heavy process, this OOM-kills the process even though the other node is half-empty.

Symptom: high `iowait`, kernel logs `task ... blocked for more than 120 seconds`

dmesg -T | grep -i blocked
# [time] INFO: task python:12345 blocked for more than 120 seconds.

# This is hung_task_timeout firing. Two common causes on HPC:
# 1. A storage backend stalled (Weka/Lustre client wait)
# 2. dirty_ratio is high and writeback can't keep up

sysctl vm.dirty_ratio vm.dirty_background_ratio
# vm.dirty_ratio = 40        <-- too high for big-RAM nodes

Bump dirty_ratio = 10 and dirty_background_ratio = 5, watch the messages stop. If they don't, the storage is the issue, not the VM.

Symptom: containerd / Kubernetes pod restart is laggy

# inotify watches exhausted
sysctl fs.inotify.max_user_watches fs.inotify.max_user_instances

# Per-user usage
for u in $(awk -F: '{print $1}' /etc/passwd); do
  count=$(find /proc/*/fd -lname "anon_inode:inotify" 2>/dev/null \
    | xargs -I{} ls -la {} 2>/dev/null \
    | grep -c "inotify")
  echo "$u $count"
done | sort -k2 -n | tail

Bump fs.inotify.max_user_watches = 1048576.

Symptom: BBR doesn't seem to engage

# Is the module loaded?
lsmod | grep bbr

# Is BBR in the available list?
cat /proc/sys/net/ipv4/tcp_available_congestion_control

# What CC does the socket actually use?
ss -tin | grep -B1 cwnd | head
# bbr wscale:... <-- look for the algo word

If cubic shows up despite tcp_congestion_control = bbr, the module didn't load. Add tcp_bbr to /etc/modules-load.d/bbr.conf and reboot (or modprobe tcp_bbr).

Symptom: NUMA-balancing is stealing time even with sysctl=0

# Confirm the runtime value
cat /proc/sys/kernel/numa_balancing
# 1                <-- not 0!

sysctl kernel.numa_balancing
# kernel.numa_balancing = 1

Either the sysctl file didn't load (check /etc/sysctl.d/ ordering, run sysctl --system), or the kernel cmdline is overriding it: cat /proc/cmdline | grep -o numa_balancing=enable. Boot cmdline wins over sysctl on this one. Edit GRUB.

Symptom: NCCL out-of-band hangs after a network blip

NCCL's bootstrap socket has no app-level keepalive. If keepalive is at default 7200s and a switch flaps, the socket sits in a half-open state for 2 hours.

sysctl net.ipv4.tcp_keepalive_time
# net.ipv4.tcp_keepalive_time = 7200      <-- default, two hours

# Lower to 600 (10 min) for HPC; bootstrap retries kick in much faster

General diff method

Take a snapshot before a problem and after:

# Before
sysctl -a 2>/dev/null | sort > /tmp/sysctl-before.txt

# ... time passes, something drifts ...

sysctl -a 2>/dev/null | sort > /tmp/sysctl-after.txt
diff /tmp/sysctl-before.txt /tmp/sysctl-after.txt

Anything that changed without your hand on it is suspicious. The cloud-init services on Ubuntu have been known to overwrite sysctl files on every boot — if your /etc/sysctl.d/99-*.conf keeps mysteriously reverting, search /etc/cloud/cloud.cfg.d/ for a write_files block.