NVIDIA Multi-Instance GPU (MIG) Partitioning with Docker

In high-density AI infrastructure (servers equipped with 80 GB NVIDIA A100 or H100 GPUs), running light workloads—such as microservice inference, small model fine-tuning, or MLOps dev environments—on an entire GPU is inefficient and expensive.

Software time-slicing introduces memory contention and VRAM latency risks. To solve this, NVIDIA introduced Multi-Instance GPU (MIG) architecture on Ampere and Hopper GPUs.

MIG enables a single physical GPU to be partitioned into up to 7 independent GPU Instances, each with its own dedicated Tensor Cores, media decoders, memory crossbar, and hardware-isolated VRAM.

This guide explains how to configure MIG on Linux hosts and bind isolated GPU instances to Docker containers.

---

1. Understanding MIG Geometry and Profiles

Each MIG instance is defined by the proportion of compute resources and VRAM memory allocated to it.

On an NVIDIA A100 80GB GPU, the card provides 7 GPCs (Graphics Processing Clusters) and 80 GB VRAM.

Standard MIG Profile Examples

Profile Name	Compute Cores (GPC)	Dedicated VRAM	Max Instances per A100	Ideal Use Case
1g.10gb	1/7	10 GB	7 instances	Light inference (BERT, Whisper, embeddings)
2g.20gb	2/7	20 GB	3 instances	Quantized 7B/13B LLM inference
3g.40gb	3/7	40 GB	2 instances	Llama-3 / Mistral LoRA fine-tuning
7g.80gb	7/7 (Full)	80 GB	1 instance	Full training / 70B LLM inference

[ NVIDIA A100 80GB (Physical GPU) ]
 ├─────── MIG Instance 1 (1g.10gb) ──► Docker Container 1 (Embedding Service)
 ├─────── MIG Instance 2 (1g.10gb) ──► Docker Container 2 (Whisper STT)
 ├─────── MIG Instance 3 (2g.20gb) ──► Docker Container 3 (vLLM Mistral-7B)
 └─────── MIG Instance 4 (3g.40gb) ──► Docker Container 4 (LoRA Training)

---

2. Enabling MIG Mode on the Host Linux Server

Step 1: Verify Host GPU

Run nvidia-smi to verify GPU support and note the GPU Index (e.g., GPU 0).

nvidia-smi -L

Step 2: Enable MIG Mode

sudo nvidia-smi -i 0 -mig 1

Note: If CUDA processes are currently running, restart the NVIDIA persistence daemon or host server:

sudo systemctl restart nvidia-persistenced

Verify MIG status:

nvidia-smi

The output table will display MIG Devices: Enabled.

---

3. Creating GPU Instances (GI) and Compute Instances (CI)

For a container to consume a MIG partition, create a GPU Instance (GI) and pair it with a Compute Instance (CI).

Step 1: List Available Profiles

nvidia-smi mig -lgip

Step 2: Partition GPU into Instances (Example: 2x 2g.20gb and 3x 1g.10gb)

Create two 20 GB instances (Profile ID 19) and three 10 GB instances (Profile ID 14) on GPU 0:

# Create two 2g.20gb MIG instances with Compute Instances (-C)
sudo nvidia-smi mig -cgi 19,19 -C

# Create three 1g.10gb MIG instances with Compute Instances (-C)
sudo nvidia-smi mig -cgi 14,14,14 -C

Step 3: Retrieve MIG Instance UUIDs

Run nvidia-smi -L to inspect unique MIG UUIDs:

GPU 0: NVIDIA A100-SXM4-80GB (UUID: GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0)
  MIG 2g.20gb Device 0: (UUID: MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/1/0)
  MIG 2g.20gb Device 1: (UUID: MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/2/0)
  MIG 1g.10gb Device 2: (UUID: MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/3/0)
  MIG 1g.10gb Device 3: (UUID: MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/4/0)
  MIG 1g.10gb Device 4: (UUID: MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/5/0)

---

4. Binding MIG Instances to Docker Containers

Method 1: Docker CLI (NVIDIA_VISIBLE_DEVICES)

Pass the exact MIG UUID using the NVIDIA_VISIBLE_DEVICES environment variable:

docker run --rm \
  --gpus '"device=MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/1/0"' \
  ubuntu:22.04 nvidia-smi

Inside the container, nvidia-smi will see only the designated 20 GB hardware slice.

---

Method 2: Declarative Setup with Docker Compose v2

version: "3.8"

services:
  llm-inference:
    image: vllm/vllm-openai:latest
    environment:
      - NVIDIA_VISIBLE_DEVICES=MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/1/0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  embedding-service:
    image: ghcr.io/huggingface/text-embeddings-inference:latest
    environment:
      - NVIDIA_VISIBLE_DEVICES=MIG-GPU-a1b2c3d4-e5f6-7890-1234-56789abcdef0/3/0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

---

Method 3: Container Device Interface (CDI)

With Docker Engine 25+ and CDI:

1. Generate CDI device files:

sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml

2. Launch using CDI MIG device names:

docker run --rm --device nvidia.com/mig=0:0 ubuntu:22.04 nvidia-smi

---

Production Best Practices

1. Persist MIG Geometry Across Reboots: MIG configurations reset on host reboot. Use nvidia-mig-manager or a systemd startup script to re-apply partitions.
2. Avoid Dynamic Destruction on Running Containers: Do not destroy MIG instances (nvidia-smi mig -dgi) while active containers are attached.
3. Monitor Individual MIG Instances: Configure nvidia-dcgm-exporter with DCGM_FI_DEV_MIG_ACTIVE for per-instance Prometheus metrics.