GPU sharing: MIG vs MPS vs time-slicing vs vGPU (and where DRA fits)

A modern GPU is too big for most pods. An H100 has 80 GB of HBM3 and 132 SMs; a small inference replica serving a 7B model needs maybe 8 GB and a sliver of compute. Renting the whole card per pod burns 90% of the silicon. Four mechanisms exist to put more than one workload on the same GPU, and they make very different trade-offs. This page is a side-by-side: what each one does, where it breaks, the Kubernetes wiring with the GPU Operator, and why Dynamic Resource Allocation (DRA) is becoming the answer to "I want to express what I actually need."

The four mechanisms in one paragraph each:

• MIG (Multi-Instance GPU) — hardware partitioning. The GPU is carved into up to 7 isolated slices with their own SMs, L2, and HBM. Hard isolation, no NVLink across slices, fixed compute budget.
• MPS (Multi-Process Service) — CUDA-level multiplexing. A daemon proxies all GPU contexts so multiple processes share SMs concurrently rather than serially. Higher utilization, no fault isolation.
• Time-slicing — the device plugin advertises N replicas of one GPU; the kernel scheduler round-robins. No isolation, no quotas, no quality-of-service. Cheap and cheerful, dev/test only.
• vGPU — NVIDIA's licensed virtualization product. SR-IOV-style virtual functions for VDI and per-VM GPUs. Different problem domain (mostly VMs, not Kubernetes-native pods).

And the newer one:

• DRA — not a sharing mechanism itself; it's the Kubernetes API that lets you describe "I want a GPU with at least 40 GB free" or "I want a 2g.20gb MIG slice on a node that also has IB" and let a driver provision it. DRA is stable in Kubernetes 1.35; the NVIDIA k8s-dra-driver-gpu is shipping ComputeDomain support today and experimental on-demand MIG.

Decision matrix

This is the table to print and tape to your monitor.

The "wrong" choices fail in characteristic ways:

• Time-slicing for multi-tenant inference — one tenant's runaway kernel starves the others; OOM in one container takes the whole GPU offline until reset.
• MIG for training — slices have no NVLink between them. Tensor-parallel collapses to PCIe bandwidth, training time multiplies.
• MPS in a hostile multi-tenant cluster — no memory protection between clients; a bug in one process can read/clobber another's tensors.
• Full GPU for cheap inference — wastes 80%+ of HBM and SMs per replica.

MIG

Detailed coverage lives in MIG: GPU partitioning for multi-tenancy. The short version for sharing decisions:

• Spatial partition, hardware-enforced. Each instance gets its own SM slice, L2 slice, HBM partition, PCIe BAR.
• Up to 7 instances per GPU (Ampere and later: A100, H100, H200, B100/B200/B300, GB200 per die).
• No NVLink across instances — this is the deal-breaker for any tensor-parallel or NCCL-heavy workload.
• Fixed compute — a 1g.10gb slice has a fixed budget; idle siblings don't help it burst.
• Kubernetes wiring: mig-manager (part of the GPU Operator) renders profiles based on a node label.

# Label a node to apply a 7×1g layout
$ kubectl label node gpu-node-a nvidia.com/mig.config=all-1g.10gb --overwrite

# Watch the state machine
$ kubectl get nodes -l nvidia.com/mig.config.state \
    -o custom-columns=NAME:.metadata.name,STATE:.metadata.labels."nvidia\.com/mig\.config\.state"
NAME         STATE
gpu-node-a   pending
gpu-node-a   rebooting
gpu-node-a   success

After success, the device plugin advertises nvidia.com/mig-1g.10gb: 7 instead of nvidia.com/gpu: 1. Pods request it the obvious way:

resources:
  limits:
    nvidia.com/mig-1g.10gb: 1

The strategy is per-node (single for homogeneous nodes, mixed for heterogeneous). Production overwhelmingly uses mixed.

MPS

MPS is a binary-compatible CUDA shim. Without MPS, two processes on the same GPU time-slice at the context-switch boundary — process A runs for a quantum, process B runs for a quantum, etc. With MPS, both processes' kernels are submitted to the same CUDA context via a per-GPU control daemon, and the GPU's hardware schedulers (Hyper-Q on Volta+) interleave their warps on the SMs. Net effect: real concurrency, higher utilization, lower per-request latency when neither workload alone saturates the GPU.

Architecture

[client A] ──┐
[client B] ──┼──► nvidia-cuda-mps-control (daemon, per node)
[client C] ──┘                │
                              ▼
                     nvidia-cuda-mps-server (per GPU, per user)
                              │
                              ▼
                          GPU SMs

• The control daemon owns a Unix socket at $CUDA_MPS_PIPE_DIRECTORY (default /tmp/nvidia-mps).
• Clients connect by setting CUDA_MPS_PIPE_DIRECTORY and CUDA_MPS_LOG_DIRECTORY; the CUDA driver then routes through MPS instead of opening a primary context.
• One server process per GPU per user. MPS is single-user by default — only the same UID can connect to a given server. Multi-user MPS exists but is fiddly to set up.

Launch on bare metal

# Set GPU to EXCLUSIVE_PROCESS so only the MPS server holds a context
$ sudo nvidia-smi -i 0 -c EXCLUSIVE_PROCESS

# Start the control daemon
$ export CUDA_VISIBLE_DEVICES=0
$ export CUDA_MPS_PIPE_DIRECTORY=/var/run/nvidia-mps
$ export CUDA_MPS_LOG_DIRECTORY=/var/log/nvidia-mps
$ sudo -E nvidia-cuda-mps-control -d

# Verify it's listening
$ echo get_default_active_thread_percentage | nvidia-cuda-mps-control
100.0

# Cap each client to 50% of the SMs (Volta+)
$ echo set_default_active_thread_percentage 50 | nvidia-cuda-mps-control

Two important throttles:

• set_default_active_thread_percentage (Volta+, formerly EXCLUSIVE_PROCESS only) — caps the SM fraction a client can grab. Useful to keep one rogue process from monopolizing the GPU.
• set_default_device_pinned_mem_limit (Volta+) — caps pinned (page-locked) memory per client.

These are advisory, not hard isolation. The MPS server will not OOM-kill a client that exceeds its quota; it just denies further allocations.

Limits — the ones that bite

• No fault isolation. A segfault in one client process can kill the MPS server, which kills all clients on that GPU. This is why MPS is not safe for hostile multi-tenancy.
• No memory protection between clients. All clients share one CUDA context. A buggy kernel can stomp another client's allocations.
• Single user (default). Different Linux users cannot share an MPS server.
• Compute mode requirement. GPU should be EXCLUSIVE_PROCESS for clean MPS operation; DEFAULT works but you can have non-MPS contexts colliding.
• No oversubscription protection. If 16 clients each request 30% of SMs, MPS doesn't queue them — they all run, fighting for warps. Throughput collapses.
• CUDA Graph and stream synchronization edge cases — some workloads behave subtly differently under MPS, especially with peer-to-peer memory or streams that cross context boundaries.

Performance: rough numbers

The rule of thumb operators quote: MPS adds ~1-3% per-request overhead (extra IPC hop), but yields 30-100% higher aggregate throughput when individual workloads under-utilize the GPU. Two workloads each using 40% of the SMs can both run at near-full speed under MPS; without MPS they time-slice and each gets ~50% throughput.

It's a utilization play, not a latency play. If you have one workload that already saturates the GPU, MPS adds overhead with no benefit.

MPS on Kubernetes via the GPU Operator

The GPU Operator's device plugin gained MPS support in late 2023. Configuration is similar in shape to time-slicing:

apiVersion: v1
kind: ConfigMap
metadata:
  name: mps-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    sharing:
      mps:
        renameByDefault: false
        resources:
          - name: nvidia.com/gpu
            replicas: 4

$ kubectl create -n gpu-operator -f mps-config.yaml

$ kubectl patch clusterpolicies.nvidia.com/cluster-policy \
    -n gpu-operator --type merge \
    -p '{"spec":{"devicePlugin":{"config":{"name":"mps-config","default":"any"}}}}'

$ kubectl rollout restart -n gpu-operator daemonset/nvidia-device-plugin-daemonset

The plugin spawns one MPS control daemon per GPU as a sidecar, and pods that request nvidia.com/gpu get routed through it transparently. With replicas: 4 the GPU advertises four units; with renameByDefault: true it advertises nvidia.com/gpu.shared.

Time-slicing

The simplest mechanism. The device plugin lies to the kubelet: "this GPU has N replicas." Four pods land on it, each thinks it has a GPU, and the NVIDIA driver's kernel scheduler round-robins CUDA contexts at the hardware time-slice boundary — same mechanism the OS uses for any pre-MPS multi-context GPU.

Configuration

apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    flags:
      migStrategy: none
    sharing:
      timeSlicing:
        renameByDefault: false
        failRequestsGreaterThanOne: false
        resources:
          - name: nvidia.com/gpu
            replicas: 4

$ kubectl create -n gpu-operator -f time-slicing-config.yaml

$ kubectl patch clusterpolicies.nvidia.com/cluster-policy \
    -n gpu-operator --type merge \
    -p '{"spec":{"devicePlugin":{"config":{"name":"time-slicing-config","default":"any"}}}}'

$ kubectl rollout restart -n gpu-operator daemonset/nvidia-device-plugin-daemonset

After the rollout, the node advertises nvidia.com/gpu: 4 (with one physical card). Four pods can request nvidia.com/gpu: 1 and all schedule.

Per-node overrides via labels:

$ kubectl label node gpu-node-b nvidia.com/device-plugin.config=high-replicas --overwrite

Knobs

What it actually does

• Round-robin at the CUDA context level. No quotas, no priorities, no resource guarantees.
• No memory partitioning. All pods share the GPU's full HBM. One pod's OutOfMemoryError can come from another pod allocating tensors.
• No fault isolation. A driver-killing kernel from one container takes the whole node's GPU offline.
• DCGM-exporter cannot attribute metrics to containers under time-slicing — the GPU shows aggregate utilization, no per-pod breakdown. Capacity planning under time-slicing is hard for this reason.

Where it's correct

Dev clusters, JupyterHub-style notebook farms, tutorial environments, anywhere the cost of a fight is "Slack the user." It is not correct for any production multi-tenant inference path.

vGPU

vGPU is NVIDIA's licensed virtualization product. It uses SR-IOV (or older driver-mediated approaches) to expose virtual GPUs to virtual machines. Architecturally:

• Hypervisor (VMware ESXi, KVM, Hyper-V, etc.) loads the vGPU host driver.
• Each VM gets a virtual GPU (vGPU) with a fixed memory carve-out and a scheduling policy.
• Inside the VM, the vGPU guest driver behaves like a normal NVIDIA driver.

vGPU profiles look superficially like MIG (e.g. A100-10C is a 10 GB profile), but the underlying mechanism is different — it's hypervisor scheduling, not hardware partitioning, except on MIG-backed vGPU SKUs where vGPU and MIG combine.

Operationally: vGPU is for VDI (Citrix, Horizon), per-VM data science workstations, and some cloud-provider IaaS GPU offerings. It is not the right tool for Kubernetes pod-level sharing — there's no clean Kubernetes integration story, and you're paying license fees for a feature you can get from MIG/MPS for free.

If your workload is "give 100 analysts a desktop with a GPU," vGPU is correct. Otherwise it's almost never the right answer in the K8s sharing decision.

DRA: the new shape of the question

The device plugin model — nvidia.com/gpu: 1, nvidia.com/mig-3g.40gb: 1 — is a flat namespace of opaque resources. It cannot express:

• "I want any GPU with ≥ 40 GB free HBM."
• "I want a 2g.20gb MIG slice, but if there isn't one available, create one on a node where 4 SMs are free."
• "I want two GPUs that share the same NVLink fabric."
• "I want one GPU and one BlueField DPU on the same PCIe root."

Dynamic Resource Allocation (DRA) is the Kubernetes API that does this. Stable in Kubernetes 1.35. The driver (e.g. k8s-dra-driver-gpu from NVIDIA) advertises devices with attributes in ResourceSlice objects; workloads request via ResourceClaim or ResourceClaimTemplate with CEL expressions over those attributes.

The four objects

# 1. DeviceClass — defined by the driver / cluster admin
apiVersion: resource.k8s.io/v1
kind: DeviceClass
metadata:
  name: gpu-large
spec:
  selectors:
  - cel:
      expression: "device.driver == 'gpu.nvidia.com' && device.attributes['memory'].quantity >= quantity('40Gi')"

# 2. ResourceClaimTemplate — the workload's wishlist
apiVersion: resource.k8s.io/v1
kind: ResourceClaimTemplate
metadata:
  name: one-h100-template
spec:
  spec:
    devices:
      requests:
      - name: gpu
        deviceClassName: gpu-large
        selectors:
        - cel:
            expression: "device.attributes['productName'].string.matches('NVIDIA H100.*')"

# 3. Pod uses the template
apiVersion: v1
kind: Pod
metadata:
  name: inference-server
spec:
  resourceClaims:
  - name: gpu
    resourceClaimTemplateName: one-h100-template
  containers:
  - name: server
    image: vllm/vllm-openai:latest
    resources:
      claims:
      - name: gpu

# 4. ResourceSlice — created by the driver, advertises devices
apiVersion: resource.k8s.io/v1
kind: ResourceSlice
metadata:
  name: gpu-node-a-slice
spec:
  nodeName: gpu-node-a
  driver: gpu.nvidia.com
  devices:
  - name: gpu-0
    attributes:
      productName: { string: "NVIDIA H100 80GB HBM3" }
      memory:      { quantity: "80Gi" }
      uuid:        { string: "GPU-abc..." }

Install the NVIDIA DRA driver

$ helm repo add nvidia https://helm.ngc.nvidia.com/nvidia
$ helm install --create-namespace -n nvidia-dra-driver-gpu \
    nvidia-dra-driver-gpu nvidia/nvidia-dra-driver-gpu \
    --set image.tag=v25.x.x \
    --set gpuKubeletPlugin.enabled=true \
    --set computeDomainKubeletPlugin.enabled=true

Today the driver ships:

• ComputeDomains — supported. The abstraction for multi-node NVLink (GB200 NVL72): pods in the same ComputeDomain are guaranteed MNNVL-reachable, others are isolated. The driver orchestrates the IMEX daemons / channels under the hood.
• GPU allocation by attribute — experimental, off by default. Includes the path to on-demand MIG creation, where the driver creates a MIG instance to satisfy the claim instead of you pre-partitioning the GPU.

Why DRA matters operationally

• Heterogeneous fleets. Mixed A100 / H100 / H200 / L40S without 40 different resource names.
• Bin-packing by attribute. "Schedule on the first node that has any GPU with ≥ 60 GB" is a one-liner.
• Sharing semantics across pods. A ResourceClaim with allocationMode: All can be referenced by multiple pods — actual GPU sharing expressed cleanly.
• No more pre-partitioning rituals. Want a 1g slice? Ask for one. The driver creates it on a node that has spare slices.

The transition is not free: device plugins still ship and most operators will use both for the next several releases. Expect to migrate piecemeal — start with ComputeDomain for GB200 deployments where DRA is the only reasonable path.

Putting it together: which one when

A short, opinionated decision flow:

Is this multi-tenant with hostile / untrusted workloads?
├── Yes → MIG. Hard isolation is the only option.
│
└── No (single tenant, cooperating workloads)
    │
    Does each workload saturate the GPU on its own?
    ├── Yes → No sharing. Full GPU per pod.
    │
    └── No (workloads under-utilize)
        │
        Is this prod inference with QoS expectations?
        ├── Yes → MPS. Better latency than time-slicing under load.
        │
        └── No (dev/test, notebooks, tolerant workloads)
            └── Time-slicing. Cheapest; users tolerate jitter.

Cross-cutting:
- VDI / per-VM       → vGPU (separate problem space).
- GB200 NVL72        → DRA ComputeDomain (the only clean path).
- Heterogeneous fleet→ DRA, regardless of mechanism above.

Operational gotchas across all mechanisms

• DCGM and per-tenant attribution. MIG: works (per-instance metrics). MPS / time-slicing: aggregate only — you cannot tell which tenant burned the SMs.
• Driver / fabric-manager versions. Each mechanism has minimum driver versions. Mixing modes during a driver rollout is asking for trouble; drain GPU pods first.
• Persistence mode. nvidia-persistenced should be on, otherwise the first GPU close tears down state on some configurations.
• The mechanisms compose, mostly badly. MPS over MIG instances is supported and useful (per-slice MPS); time-slicing over MIG is rarely what you want; vGPU over MIG is a licensed combination on specific SKUs.
• Container runtime. The NVIDIA Container Toolkit honours NVIDIA_VISIBLE_DEVICES, MIG_VISIBLE_DEVICES, and CUDA_VISIBLE_DEVICES. The K8s device plugin sets these for you. Bare-Docker users must set them by hand, and confused environment variables are the #1 cause of "GPU not visible inside container" tickets.
• GPU sharing breaks some assumptions. Frameworks that introspect nvidia-smi output or assume one cudaSetDevice(0) covers the whole GPU need to be checked under MIG/MPS. vLLM, Triton, TensorRT-LLM all work; some hand-rolled training scripts do not.

See also

• MIG: GPU partitioning for multi-tenancy
• GPU Operator — installs the device plugin, mig-manager, MPS sidecar
• GPU generations — which SKUs support MIG, NVLink, etc.
• DCGM under sharing modes — what metrics you do and don't get
• Inference vs training infrastructure — sharing decisions are different on each side
• Inference stack — frameworks that benefit most from sharing
• NVIDIA MIG User Guide: https://docs.nvidia.com/datacenter/tesla/mig-user-guide/
• NVIDIA MPS docs: https://docs.nvidia.com/deploy/mps/index.html
• GPU Operator GPU sharing docs: https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/gpu-sharing.html
• Kubernetes DRA: https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic-resource-allocation/
• NVIDIA k8s-dra-driver-gpu: https://github.com/NVIDIA/k8s-dra-driver-gpu