Terraform Module for RunPod GPU Provisioning

Quick Answer

This Terraform blueprint automates the deployment of NVIDIA GPU pods on RunPod Secure Cloud. It is designed for ephemeral workloads—provisioning a pod, executing a task (like model quantization or training), and destroying the resource immediately to optimize costs.

Overview

RunPod is a leader in the “GPU Cloud” space, offering high-end hardware (A100, H100) at competitive hourly rates. While the RunPod UI is intuitive, production engineering teams should use Infrastructure as Code (IaC) to ensure reproducible environments and automated cleanup.

Implementation: The Terraform Stack

1. Provider Configuration

Add the following to your providers.tf.

terraform {
  required_providers {
    runpod = {
      source  = "runpod-infra/runpod"
      version = "~> 0.1"
    }
  }
}

provider "runpod" {
  api_key = var.runpod_api_key
}

2. Variables & Schema

Define your hardware constraints in variables.tf.

variable "runpod_api_key" {
  type        = string
  sensitive   = true
}

variable "gpu_type" {
  description = "Hardware ID (e.g., 'NVIDIA RTX 4090' or 'NVIDIA A100 80GB PCIe')"
  type        = string
  default     = "NVIDIA RTX 4090"
}

variable "container_image" {
  type        = string
  default     = "ollama/ollama:latest"
}

3. Pod Definition

The runpod_pod resource allows you to define GPU count, storage, and networking in a single block.

resource "runpod_pod" "ai_node" {
  name          = "opsnexus-worker"
  image_name    = var.container_image
  gpu_type_id   = var.gpu_type
  cloud_type    = "SECURE" # Critical for privacy
  gpu_count     = 1

  container_disk_in_gb = 50
  volume_in_gb         = 100
  volume_mount_path    = "/workspace"

  ports = "11434/http"

  env = [
    { key = "OLLAMA_HOST", value = "0.0.0.0" },
    { key = "OLLAMA_KEEP_ALIVE", value = "1h" }
  ]
}

Architecture Notes: Secure vs. Community Cloud

RunPod offers two distinct cloud types:

• Secure Cloud: Tiers of high-security data centers with strict access controls. Recommended for all enterprise AI workloads.
• Community Cloud: Peer-to-peer hardware hosted by individual operators. Use only for testing with non-sensitive data.

Security Notes

• API Key Hygiene: Use environment variables (TF_VAR_runpod_api_key) or a secrets manager. Never hardcode keys in .tf files.
• Data Persistence: If your pod is ephemeral, ensure that results (weights/logs) are synced to an S3 bucket or external volume before the pod is destroyed.

Common Failure Modes

• “GPU Unavailable”: The requested GPU type is out of stock in the selected region. Fix: Implement a variable-based fallback or use the RunPod API to check availability before running terraform apply.
• Image Pull Failures: Large AI images (10GB+) can timeout during pull. Fix: Increase the container disk size to accommodate both the image layers and the working model.

FAQ

Q: Can I use this for multi-GPU training? A: Yes. Set gpu_count = 2 (or more). Note that RunPod supports NVLink on specific A100/H100 configurations, which is necessary for high-bandwidth training.

Q: How do I access the pod after it’s deployed? A: Use the runpod_pod.ai_node.network_addr output to get the public IP and the ports mapping to connect to your service.

Next Steps

Need to compare RunPod with other GPU providers? See our Best VPS for AI Review.

Disclosure: We recommend RunPod for their technical documentation and developer-friendly API.