kubernetes-containers • 6 min read (1053 words)

Kubernetes GPU Partitioning Guide: NVIDIA MIG vs. Time-Slicing vs. MPS & DRA

Architectural comparison of GPU sharing mechanisms in Kubernetes: hardware-isolated NVIDIA Multi-Instance GPU (MIG), CUDA Time-Slicing, MPS, and Dynamic Resource Allocation (DRA).

Diagram comparing NVIDIA Multi-Instance GPU hardware partitioning vs CUDA time-slicing
Methodology: Technical guide based on official NVIDIA GPU Operator documentation, NVIDIA Multi-Instance GPU (MIG) architecture manuals, and Kubernetes Dynamic Resource Allocation (DRA) specifications.

1. The Kubernetes Whole-GPU Allocation Bottleneck

By default, the standard Kubernetes device plugin framework (nvidia.com/gpu) treats a GPU as an indivisible integer resource (1, 2, 4, 8). If a pod requests nvidia.com/gpu: 1 on a node equipped with 80 GB NVIDIA A100 or H100 GPUs—even to run a lightweight 1.5 GB text-embedding model (BAAI/bge-m3), an audio transcription microservice (Whisper-large-v3), or a developer Jupyter notebook—the entire 80 GB GPU is locked exclusively to that single container.

To solve low GPU utilization across AI inference and development clusters, Kubernetes and the NVIDIA GPU Operator support three distinct GPU sharing architectures, plus Kubernetes Dynamic Resource Allocation (DRA).

2. Comparing Time-Slicing, CUDA MPS, and Hardware MIG

Before configuring your cluster, compare how each mechanism handles VRAM isolation, compute SMs, fault domains, and tail latency (P99):

| Technical Dimension | CUDA Time-Slicing | Multi-Process Service (MPS) | Multi-Instance GPU (MIG) |
| :--- | :--- | :--- | :--- |
| Supported GPU Architectures | All NVIDIA Datacenter GPUs (T4, L4, L40S, A100, H100, H200, B200) | Volta, Turing, Ampere, Ada, Hopper, Blackwell | A30, A100, H100, H200, B200 (Datacenter HBM GPUs) |
| Compute Execution Model | Temporal context-switching across processes (one workload runs at a time) | Spatial concurrent kernel execution on shared SMs | Dedicated physical SMs per MIG instance |
| VRAM Memory Isolation | None (an OOM in one pod can crash or starve sharing pods) | Optional per-client pinned memory limit (shared address controller) | Strict Hardware HBM Partition with dedicated memory controllers & L2 cache |
| Error & Fault Isolation | None (a fatal CUDA fault resets the shared context) | None (a fatal fault in one client terminates the MPS server group) | Hardware Fault Isolation (errors stay contained inside the MIG slice) |
| Max Partitions per GPU | Configurable software replicas (e.g., 2 to 16 replicas) | Up to 48 client processes | Up to 7 hardware instances on A100 / H100 / H200 |
| Recommended Use Case | Dev/test notebooks, CI/CD GPU tests, low-duty-cycle internal tools | Single-tenant cooperative inference pipelines | Production multi-tenant inference with strict P99 latency SLAs |

3. Hardware Partitioning with NVIDIA MIG (1g.10gb, 2g.20gb, 3g.40gb)

On an 80 GB NVIDIA H100 or A100 (or a 141 GB H200, where corresponding slices are 1g.18gb, 2g.35gb, 3g.71gb), Multi-Instance GPU (MIG) physically partitions the GPU silicon into isolated instances, each with its own Streaming Multiprocessors, L2 cache banks, memory crossbars, and HBM stacks.

Step 1: Enable the mixed or single MIG Strategy on Your Kubernetes Node

When running the NVIDIA GPU Operator with mig.strategy=mixed, you apply a node label (nvidia.com/mig.config) that instructs the nvidia-mig-manager DaemonSet to partition the GPUs automatically: 
# Partition all GPUs on node gpu-worker-01 into two 3g.40gb slices and one 1g.10gb slice
# Wait: on an 80GB GPU (7 compute slices / 8 memory slices), valid mixed combinations include:
# 1x 4g.40gb + 1x 2g.20gb + 1x 1g.10gb, OR 2x 3g.40gb, OR 7x 1g.10gb
kubectl label nodes gpu-worker-01 nvidia.com/mig.config=all-balanced --overwrite

Under the built-in all-balanced profile on an 80 GB GPU, the GPU Operator creates:
- 2x nvidia.com/mig-1g.10gb slices (for lightweight embedding / reranker models)
- 1x nvidia.com/mig-2g.20gb slice (for 7B–8B quantized LLMs or vision models)
- 1x nvidia.com/mig-3g.40gb slice (for 14B–32B models)

Step 2: Request a Specific MIG Slice in Your Pod Manifest

apiVersion: apps/v1
kind: Deployment
metadata:
  name: bge-reranker-inference
  namespace: ai-production
spec:
  replicas: 2
  selector:
    matchLabels:
      app: bge-reranker
  template:
    metadata:
      labels:
        app: bge-reranker
    spec:
      containers:
        - name: tei-server
          image: ghcr.io/huggingface/text-embeddings-inference:1.5
          args: ["--model-id", "BAAI/bge-reranker-v2-m3", "--port", "8080"]
          resources:
            limits:
              nvidia.com/mig-1g.10gb: 1

4. Configuring CUDA Time-Slicing for Non-MIG GPUs (L4, L40S, RTX 4090)

PCIe GPUs based on the Ada Lovelace architecture (NVIDIA L4 24GB and NVIDIA L40S 48GB) do not support hardware MIG. To allow four development pods to share a single L40S GPU via time-slicing, create a ConfigMap in the gpu-operator namespace:

apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    sharing:
      timeSlicing:
        renameByDefault: false
        failRequestsGreaterThanOne: true
        resources:
          - name: nvidia.com/gpu
            replicas: 4

Setting failRequestsGreaterThanOne: true is a critical safeguard: it prevents a single pod requesting nvidia.com/gpu: 2 from accidentally receiving two virtual time-sliced replicas located on the *same* physical GPU instead of two distinct physical GPUs!

5. Next-Generation GPU Allocation: Kubernetes Dynamic Resource Allocation (DRA)

While traditional Kubernetes Device Plugins only expose static integer counters (nvidia.com/gpu: 1), Dynamic Resource Allocation (DRA, resource.k8s.io/v1beta1) in Kubernetes 1.31+ / 1.32+ replaces rigid integer counting with structured ResourceClaim and ResourceClaimTemplate objects.

How DRA Improves Multi-GPU & Multi-Tenant Scheduling

1. Topology-Aware Co-Allocation: A single ResourceClaim can request multiple GPUs guaranteed to reside on the same NVSwitch domain or PCIe root complex alongside a paired RDMA NIC (mlx5_0). 2. Delayed Binding & Parameterized Sharing: Workloads can specify minimum VRAM requirements (memory >= 20Gi) and sharing mode (TimeSlicing vs. MPS vs. MIG) inside a DeviceClass without hardcoding node-specific MIG profile names in every Helm chart. 
# Verify active GPU MIG instances and compute slice utilization across your Kubernetes node
nvidia-smi mig -lgi
nvidia-smi dmon -s u -c 10

6. Architectural Selection Matrix: Whole GPU vs. MIG vs. Time-Slicing vs. MPS

| Sharing Mechanism | Hardware Fault Isolation | Memory Protection (OOM Isolation) | Best Target Workload |
| :--- | :--- | :--- | :--- |
| Whole GPU (nvidia.com/gpu: 1) | 100% Dedicated | 100% Dedicated HBM | Production 70B+ LLM serving (vLLM) and distributed training |
| Hardware MIG (1g.10gb, 2g.20gb, 3g.40gb) | Hardware SMs & Crossbar Isolated | Dedicated HBM Partition | Multi-tenant production inference (7B/8B models, Whisper, Embeddings) on A100/H100/H200/B200 |
| CUDA Multi-Process Service (MPS) | Shared SMs (Concurrent Kernels) | Shared Address Space (No OOM wall) | Trusted single-tenant microservices needing concurrent kernel execution |
| CUDA Time-Slicing (replicas: 4) | Context-Switched SMs | Shared VRAM (OOM affects peers) | CI/CD test runners, Jupyter development notebooks, and L4/L40S GPUs without MIG |

Model your GPU cluster sizing and hourly costs in the GPU Cluster Cost Calculator.

Official References