Fix: vLLM CUDA Out of Memory (OOM) Errors: Complete Guide

Quick Fix for vLLM CUDA OOM

vLLM pre-allocates 90% of total available GPU VRAM on startup by default for its Key-Value (KV) cache. If your GPU has background processes, display servers, or your model context is too large, the startup initialization will fail.

Apply this immediate remediation to restrict memory pre-allocation and limit the context window:

python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Meta-Llama-3-8B-Instruct \
  --gpu-memory-utilization 0.75 \
  --max-model-len 4096 \
  --enforce-eager

---

Symptoms of vLLM CUDA OOM

1. Crash during KV Cache Profiling:

   ValueError: The model's weights and KV cache are too large for the GPU memory.
   Total GPU memory: 23.69 GiB, Model weights: 15.02 GiB, KV cache: 8.67 GiB

2. PyTorch Runtime Error:

   torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 512.00 MiB

3. Zombie VRAM Lock: nvidia-smi displays high VRAM utilization (e.g., 20GB/24GB), but no active PID is visible in standard process managers.

---

Root Causes of Memory Allocation Failures

Total GPU VRAM (e.g. 24GB RTX 4090)
┌─────────────────────────────────────────────────────────────┐
│ [1. Model Weights (~15GB)]                                  │
│ [2. PyTorch Activation Buffers (~1.5GB)]                    │
│ [3. KV Cache for Concurrent Users (~6.5GB)]                 │
│ [4. Display / System Overhead (~1.0GB)] ──▶ CRASH TRIGGER!  │
└─────────────────────────────────────────────────────────────┘

1. Default 0.90 Memory Utilization: Assumes 100% of the GPU is exclusively dedicated to vLLM with zero OS/driver overhead.
2. Unconstrained Context Length (--max-model-len): Modern models support 32k to 128k context windows. Each extra token requires proportional memory inside the KV cache blocks.
3. Ghost / Zombie CUDA Processes: Previous crashed Python or Docker instances holding onto GPU contexts without releasing CUDA handles.

---

Step-by-Step Fixes

1. Kill Zombie CUDA Processes & Free Trapped VRAM

Run this diagnostic command to find and terminate processes holding NVIDIA file descriptors:

# Check running processes attached to GPU 0
sudo fuser -v /dev/nvidia0

# Kill all dangling processes holding GPU handles
sudo fuser -k -9 /dev/nvidia*

---

2. Context Window & VRAM Sizing Reference Matrix

Before choosing your --max-model-len, check the required memory per user stream:

Model Architecture	Precision	Minimum Weights VRAM	Recommended Max Context (24GB GPU)	Recommended Max Context (80GB GPU)
Llama 3 8B	FP16 (16-bit)	~16 GB	8192	32768
Llama 3 8B	AWQ (4-bit)	~5.5 GB	16384	65536
Llama 3 70B	AWQ (4-bit)	~38 GB	Requires 2x 24GB or 1x 48GB	32768
Mistral 7B	FP16 (16-bit)	~14.5 GB	8192	32768

---

3. Tune Memory Utilization & CPU Swap Space

Configure vLLM to reserve CPU system RAM for KV cache offloading during traffic spikes:

python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Meta-Llama-3-8B-Instruct \
  --gpu-memory-utilization 0.80 \
  --max-model-len 8192 \
  --swap-space 8 \
  --disable-log-stats

• --gpu-memory-utilization 0.80: Restricts vLLM to use max 80% of the GPU’s memory.
• --swap-space 8: Allocates 8GB of system RAM as a fallback swap space for evicted KV blocks.

---

4. Enable AWQ / GPTQ 4-Bit Quantization

Switching from unquantized FP16 to 4-bit AWQ reduces base model weight memory by ~70% without sacrificing inference quality:

python -m vllm.entrypoints.openai.api_server \
  --model casperhansen/llama-3-8b-instruct-awq \
  --quantization awq \
  --dtype half

---

5. Multi-GPU Tensor Parallelism

If your model exceeds the capacity of a single physical card, split layers across multiple GPUs using tensor parallelism:

# Split across 2x GPUs
python -m vllm.entrypoints.openai.api_server \
  --model meta-llama/Meta-Llama-3-70B-Instruct \
  --tensor-parallel-size 2

---

Prevention & Production Best Practices

1. Deploy in Headless Linux Environments: Avoid running inference on desktop operating systems (Ubuntu Desktop, Windows WSL) where Xorg/Wayland consumes 1–2GB of baseline VRAM.
2. Set Docker ipc: host: Avoid PyTorch shared-memory bus errors when sharing tensors between container workers.
3. Monitor with DCGM: Use NVIDIA DCGM Exporter and Prometheus to monitor real-time KV cache usage.

---

Related Guides & Blueprints

• Deployment Setup: How to Deploy vLLM on a VPS with Docker
• Performance Benchmarks: vLLM vs Ollama: Inference Latency & Concurrency Benchmark
• GPU Cloud Automation: RunPod Terraform Provider: Complete IaC Guide