MIG: GPU partitioning for multi-tenancy

A single H100 has 80 GB of HBM3 and 132 SMs. If your workload needs 8 GB and a fraction of those SMs — say a small inference service serving a 7B model — giving it the whole GPU is a 90% waste. MIG (Multi-Instance GPU) carves the silicon into hardware-isolated slices that look to the OS like independent GPUs, with their own SMs, L2 cache slice, and HBM partition. This page is what MIG actually does, why the slice geometry is fiddly, and how it lands on Kubernetes.

What MIG is — and isn't

MIG partitions a single physical GPU into up to 7 independent GPU Instances. Each instance has:

• A dedicated slice of SMs (compute units).
• A dedicated slice of L2 cache.
• A dedicated slice of HBM memory.
• Its own PCIe BAR address space (so VFIO/SR-IOV passthrough is clean).
• Its own fault domain: a fault in one instance does not crash the others.

What MIG is not:

• NVLink across instances: when MIG is enabled, NVLink between instances is disabled. A MIG-partitioned GPU is essentially 7 separate GPUs with no high-bandwidth path between them. This rules MIG out for any workload that needs intra-GPU NVLink (multi-instance tensor-parallel, large NCCL ops within a node).
• Compute multiplexing: MIG is spatial partition, not time-slicing. If an instance has 14 SMs, that's all it gets, even if the rest of the GPU is idle.

For time-slicing, NVIDIA has MPS (Multi-Process Service) and the Kubernetes time-slicing feature. They're different tools — see the comparison below.

Hardware support

MIG was introduced on Ampere (A100). Currently supported:

V100, T4, A30 and similar do not support MIG.

The naming convention: Ng.MMgb

Profiles are named <compute-slice>g.<memory>gb, where Ng is the number of "compute slices" and MMgb is the memory partition size.

The compute slice is the atomic unit of MIG partitioning. On A100, the GPU has 7 compute slices (108 SMs ÷ 7 = ~15 SMs per slice, with one slice slightly larger). On H100 it's similar with 132 SMs over 7 slices.

The memory slice is more flexible: a profile reserves contiguous memory partitions. On A100-80GB each memory slice is ~10 GB.

So 2g.10gb = 2 compute slices + 1 memory slice (= 10 GB). 3g.40gb = 3 compute slices + 4 memory slices. 7g.80gb = the whole GPU as a single MIG instance (fully homogeneous to a non-MIG GPU but isolated under the MIG fault model).

Profiles per GPU

A100 80 GB

H100 80 GB SXM5

H200

H200 has 141 GB of HBM3e — same 7 compute slices, but memory partitions scale: 1g.18gb, 2g.35gb, 3g.71gb, 7g.141gb (numbers approximate; check current docs).

B100 / B200 (Blackwell)

192 GB → memory slices are larger: 1g.23gb, 2g.45gb, 3g.90gb, 7g.180gb. Compute is also rebalanced (148 SMs per die, 7 slices).

Authoritative table: NVIDIA MIG User Guide, "Supported Profiles" section. Profile names sometimes drift across driver versions — always cross-check nvidia-smi mig -lgip.

Geometry constraints

Not every combination of profiles fits on a GPU. The slices live on a fixed grid; allocations must be left-aligned in the order they're created and the totals can't exceed 7 compute + 8 memory slices. For example on A100-80GB:

valid layout:  [1g.10gb][1g.10gb][1g.10gb][1g.10gb][1g.10gb][1g.10gb][1g.10gb]    (7×1g)
valid layout:  [3g.40gb               ][3g.40gb               ]                   (2×3g, leaves 1 slice unused)
valid layout:  [2g.20gb       ][2g.20gb       ][2g.20gb       ]                   (3×2g, 1 unused)
valid layout:  [4g.40gb               ][3g.40gb               ]                    (1×4g + 1×3g)
INVALID:       [3g.40gb][3g.40gb][1g.10gb]                                         (2×3g consumes ALL memory; no room for 1g)

The takeaway: you choose a layout, not arbitrary slice sets. NVIDIA publishes the valid placements table per GPU; in practice you pick one of a handful of useful patterns:

• All-1g (7 instances) — max tenant count, smallest workloads.
• Mixed (1g + 1g + 1g + 1g + 3g) — handful of small tenants + one bigger.
• 3g + 3g — two equal halves.
• 2g + 2g + 3g — three workloads with one larger.
• 7g — full GPU (under MIG, useful for fault isolation).

Enabling MIG

MIG mode is persistent across reboots but configuring partitions is not — partitions are recreated on every driver reload.

# enable MIG on GPU 0
$ sudo nvidia-smi -i 0 -mig 1
Enabled MIG Mode for GPU 00000000:1B:00.0
All done.

# verify
$ nvidia-smi
+---------------------------------------------------------------------------------------+
| MIG Devices:                                                                           |
| ...                                                                                    |
+---------------------------------------------------------------------------------------+

# list available GPU Instance Profile IDs
$ nvidia-smi mig -lgip
+-----------------------------------------------------------------------------+
| GPU instance profiles:                                                       |
| GPU   Name             ID    Instances   Memory  ...                         |
|       1g.10gb          19    7           1/8     ...                         |
|       2g.20gb           14    3           2/8     ...                         |
|       3g.40gb            9    2           4/8     ...                         |
|       4g.40gb            5    1           4/8     ...                         |
|       7g.80gb            0    1           8/8     ...                         |
+-----------------------------------------------------------------------------+

# create one of each (example layout 3g + 2g + 2g)
$ sudo nvidia-smi mig -cgi 9,14,14 -C
Successfully created GPU instance ID  2 on GPU  0 using profile  3g.40gb (ID  9)
Successfully created compute instance ID  0 on GPU  0 GPU instance ID  2 using profile  3g.40gb (ID  2)
...

# tear them all down
$ sudo nvidia-smi mig -dci -i 0; sudo nvidia-smi mig -dgi -i 0

The -C flag also creates a Compute Instance per GPU Instance — without it the GPU Instance exists but no process can run on it.

Two-level partition: GPU Instance + Compute Instance

MIG actually has two partition levels:

1. GPU Instance (GI) — the SM/memory/L2 carve-out. This is what counts toward "max 7".
2. Compute Instance (CI) — within a GI, you can further partition the SMs into smaller compute groups. A 3g GI can host 1×3c CI or 1×1c + 1×2c CIs.

For most operators, 1:1 GI:CI is correct — you don't need this complexity. Where it's used: time-slicing within a GI for cooperating processes (rare in inference; common in HPC niches).

MIG on Kubernetes — the GPU Operator path

Doing MIG by hand on every node every reboot doesn't scale. The GPU Operator ships mig-manager, a controller that:

1. Watches a node label like nvidia.com/mig.config=all-3g.40gb (or a custom key).
2. Renders the requested profile by calling nvidia-smi mig under the hood.
3. Restarts the device-plugin so it advertises the new resource names.
4. Reports status back via labels (nvidia.com/mig.config.state=success).

# mig-manager configmap snippet
apiVersion: v1
kind: ConfigMap
metadata:
  name: default-mig-parted-config
data:
  config.yaml: |
    version: v1
    mig-configs:
      all-disabled:
        - devices: all
          mig-enabled: false
      all-1g.10gb:
        - devices: all
          mig-enabled: true
          mig-devices:
            "1g.10gb": 7
      mixed-3g-2g-2g:
        - devices: all
          mig-enabled: true
          mig-devices:
            "3g.40gb": 1
            "2g.20gb": 2

Apply by labelling the node:

$ kubectl label node gpu-host-1 nvidia.com/mig.config=all-1g.10gb --overwrite
node/gpu-host-1 labeled

Mig-manager:

1. Cordons the node.
2. Drains GPU pods.
3. Reconfigures partitions.
4. Uncordons.

The device plugin then advertises nvidia.com/mig-1g.10gb: 7 instead of nvidia.com/gpu: 1.

Pod requesting a MIG slice

apiVersion: v1
kind: Pod
spec:
  containers:
  - name: triton
    image: nvcr.io/nvidia/tritonserver:24.10-py3
    resources:
      limits:
        nvidia.com/mig-1g.10gb: 1

Important: there is no automatic "give me any 1g slice on whatever GPU has room" — you request a specific MIG profile resource. If your cluster has heterogeneous MIG layouts (some nodes 1g, some 3g), pods need to pick which.

Strategies: single vs mixed

The device plugin supports two MIG strategies, set per node:

• single — all GPUs on the node have the same MIG profile; resource name is generic (nvidia.com/gpu: 1 per slice). Fewer resource names but no heterogeneity.
• mixed — explicit profile names (nvidia.com/mig-3g.40gb). Heterogeneous layouts allowed; pod manifests must name the profile.

Production deployments overwhelmingly use mixed.

When MIG is the right answer

MIG is good when:

• You have many small workloads that fit comfortably in a fraction of a GPU (< 40 GB, < 50% of SMs for the workload's working set).
• You need hard isolation for multi-tenancy (one customer's bug must not crash another's job).
• You serve many lightweight inference replicas of a small model — MIG gives you 7× the instance count per GPU at the cost of total throughput.

MIG is wrong when:

• You need NVLink between slices (training large models).
• Your workload's working set exceeds the slice memory size (no spilling to other slices).
• Your workload is bursty — MIG gives a fixed compute budget; idle slices don't help busy ones. Use MPS or time-slicing instead.

MIG vs MPS vs time-slicing vs full GPUs

Operational gotchas

• Driver and fabric-manager must support the GPU + MIG: H100 needs driver ≥ 525, B-series 550+. Older drivers may pretend MIG works and partially fail.
• Persistence mode + MIG: must enable persistence (nvidia-persistenced) before configuring partitions, or you lose layouts on first GPU close.
• DCGM under MIG: you scrape per-MIG-instance metrics. Make sure dcgm-exporter has the MIG label config; otherwise dashboards are blank.
• CUDA + MIG: an app must call cudaSetDevice with the MIG instance UUID, not the parent GPU. Most frameworks discover via CUDA_VISIBLE_DEVICES set by the runtime, but custom code can break.
• Container runtime + MIG: MIG_VISIBLE_DEVICES is the env var the NVIDIA Container Toolkit honours. The K8s device plugin sets it for you; bare-Docker users must set it manually.
• vGPU vs MIG: vGPU (the licensed product) can sit on top of MIG on supported SKUs (e.g. L40S — wait, no, L40S has no MIG; H100 vCS does). Don't confuse the two: vGPU is a virtualization product for VDI; MIG is a hardware feature.

See also

• GPU generations — which SKUs support MIG
• DCGM under MIG
• Kubernetes GPU operator + mig-manager
• NIM / Triton on MIG slices
• NVIDIA MIG User Guide: https://docs.nvidia.com/datacenter/tesla/mig-user-guide/