RunPod Terraform Provider: Complete IaC GPU Provisioning Guide

Quick Answer

To deploy GPU instances on RunPod using Terraform, configure the official runpod-infra/runpod provider in your HCL files. This allows automated provisioning of NVIDIA RTX 4090, A100, and H100 pods with attached network storage, custom Docker images, and port forwarding for Ollama, vLLM, or training pipelines:

terraform {
  required_providers {
    runpod = {
      source  = "runpod-infra/runpod"
      version = "~> 0.1.0"
    }
  }
}

provider "runpod" {
  api_key = var.runpod_api_key
}

resource "runpod_pod" "inference_node" {
  name         = "vllm-inference-node"
  image_name   = "vllm/vllm-openai:latest"
  gpu_type_id  = "NVIDIA GeForce RTX 4090"
  cloud_type   = "SECURE"
  gpu_count    = 1
  ports        = "8000/http"
}

Why Use Terraform for RunPod GPU Provisioning?

While the RunPod web console is fast for one-off experimentation, production AI infrastructure requires Infrastructure as Code (IaC):

1. Automated Cost Control: Spin up expensive high-VRAM instances ($1.50–$3.50/hr for A100/H100) dynamically during batch inference or fine-tuning, then cleanly run terraform destroy.
2. Deterministic Model Environments: Pin exact container tags, environment variables, CUDA driver requirements, and disk mount paths across staging and production.
3. Multi-Region Redundancy: Programmatically deploy across alternative regions if primary GPU pools face spot capacity constraints.

Step-by-Step Implementation: The Production Terraform Stack

1. Project Directory Layout

runpod-terraform/
├── main.tf           # Pod and volume definitions
├── variables.tf      # API keys, GPU IDs, and container params
├── outputs.tf        # Public IP, SSH command, and HTTP endpoints
└── terraform.tfvars  # Environment variables (keep in .gitignore)

2. Provider Configuration (providers.tf)

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    runpod = {
      source  = "runpod-infra/runpod"
      version = "~> 0.1"
    }
  }
}

provider "runpod" {
  api_key = var.runpod_api_key
}

3. Variable Schema (variables.tf)

variable "runpod_api_key" {
  description = "RunPod API Key generated from user settings"
  type        = string
  sensitive   = true
}

variable "gpu_type_id" {
  description = "Hardware GPU Model identifier"
  type        = string
  default     = "NVIDIA GeForce RTX 4090" # Options: "NVIDIA A100-SXM4-80GB", "NVIDIA H100 80GB HBM3"
}

variable "gpu_count" {
  description = "Number of GPUs allocated to the pod"
  type        = number
  default     = 1
}

variable "container_image" {
  description = "Docker image with pre-packaged CUDA & inference engine"
  type        = string
  default     = "ollama/ollama:latest"
}

variable "volume_size_gb" {
  description = "Persistent Network Volume size in GB"
  type        = number
  default     = 100
}

variable "data_center_id" {
  description = "Target datacenter region (e.g., US-CA-1, EU-RO-1)"
  type        = string
  default     = "US-CA-1"
}

4. Complete Resource Definition (main.tf)

# 1. Persistent Network Volume for Model Weights
resource "runpod_network_volume" "model_cache" {
  name             = "llama-model-cache"
  size             = var.volume_size_gb
  data_center_id   = var.data_center_id
}

# 2. Compute Pod with Volume Attachment
resource "runpod_pod" "ai_worker" {
  name               = "opsnexus-ai-pod"
  image_name         = var.container_image
  gpu_type_id        = var.gpu_type_id
  gpu_count          = var.gpu_count
  cloud_type         = "SECURE" # SECURE = ISO/SOC2 datacenter; COMMUNITY = P2P
  data_center_id     = var.data_center_id

  # Storage configuration
  container_disk_in_gb = 50
  volume_in_gb         = var.volume_size_gb
  volume_mount_path    = "/root/.ollama"

  # Network exposure
  ports              = "11434/http,22/tcp"

  # Environment variables for the inference runtime
  env = [
    { key = "OLLAMA_HOST", value = "0.0.0.0" },
    { key = "OLLAMA_KEEP_ALIVE", value = "24h" },
    { key = "OLLAMA_NUM_PARALLEL", value = "4" }
  ]
}

5. Outputs for Continuous Integration (outputs.tf)

output "pod_id" {
  description = "The unique RunPod instance ID"
  value       = runpod_pod.ai_worker.id
}

output "endpoint_url" {
  description = "HTTP API ingress point for the inference engine"
  value       = "https://${runpod_pod.ai_worker.id}-11434.proxy.runpod.net"
}

output "machine_ip" {
  description = "Public IP for direct SSH access"
  value       = runpod_pod.ai_worker.machine_id
}

Deployment & Lifecycle Execution

# 1. Initialize the provider plugin
terraform init

# 2. Preview the hardware allocation & billing
terraform plan -var="runpod_api_key=rpa_xxxxxxxxxxxxxxxx"

# 3. Provision the GPU Pod
terraform apply -auto-approve -var="runpod_api_key=rpa_xxxxxxxxxxxxxxxx"

# 4. Verify API connectivity
curl https://<POD_ID>-11434.proxy.runpod.net/api/tags

# 5. Clean teardown when workload completes (prevents accidental charges)
terraform destroy -auto-approve -var="runpod_api_key=rpa_xxxxxxxxxxxxxxxx"

Architecture Comparison: Secure Cloud vs. Community Cloud

Failure Modes & Troubleshooting

1. Error: GPU Type Unavailable in Selected Region

• Root Cause: The selected datacenter (data_center_id) has exhausted on-demand stock for that specific GPU identifier.
• Fix: Omit data_center_id to allow RunPod’s scheduler to allocate across all secure clusters, or set up Terraform dynamic locals to fall back to an equivalent GPU tier (e.g., RTX 4090 -> RTX 3090).

2. Timeout during Docker Pull

• Root Cause: Large weights baked into Docker images (20GB+) can cause container initialization timeouts.
• Fix: Mount a runpod_network_volume to /models and download weights at startup using an init script rather than baking them into the container image layers.

Related Guides & Blueprints

• Cloud Evaluation: RunPod vs AWS for AI Workloads (Cost & VRAM)
• Decentralized Clouds: RunPod vs Vast.ai: Managed vs Decentralized GPU Hosting
• Production Serving: How to Deploy vLLM on a GPU VPS with Docker