← Back to Labs / OPNsense

All-in-One AI Server Architecture: OPNsense, Ollama & Docker on One Host

OpsNexusAI Engineering •
12 min read

All-in-One AI Server Architecture: OPNsense, Ollama & Docker on One Host

Consolidating network routing, zero-trust security, and GPU-accelerated AI inference onto a single physical machine is one of the most efficient ways to build a private homelab or small-office sovereign AI stack. However, co-locating a perimeter firewall (OPNsense) alongside resource-intensive workloads (Ollama, PostgreSQL, Docker web apps, and CCTV/NVR) introduces significant architectural challenges:

  1. Hardware Virtualization & PCIe Isolation: How to safely pass dedicated PCIe GPUs to an AI VM while keeping NIC interfaces assigned to the firewall.
  2. Network Micro-Segmentation: Ensuring that local Docker containers (e.g., Laravel, Postgres, Ollama) cannot bypass the OPNsense packet filter.
  3. Resource Contention: Preventing LLM VRAM/CPU spikes from crashing core routing and DNS services.

This guide details the complete hardware configuration, Proxmox VE hypervisor setup, bridge topology, and firewall rules for a secure single-machine AI deployment.

Architectural Topology: Type-1 Hypervisor on Single Machine

Physical Hardware (Bare Metal: Ryzen 9 / Intel Core i9 / Xeon + RTX 4090 + Dual NICs)
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│  Proxmox VE (Type-1 Hypervisor Host)                                        │
│                                                                             │
│  ┌─────────────────────────┐         ┌───────────────────────────────────┐  │
│  │ VM 100: OPNsense Router │         │ VM 200: AI & Application Worker   │  │
│  │ - WAN: Physical NIC 1   │         │ - OS: Ubuntu 24.04 Server         │  │
│  │ - LAN: vmbr0 (Internal) │◄───────▶│ - GPU: PCIe Passthrough (RTX 4090)│  │
│  │ - VLAN 50: AI Subnet    │         │ - Docker Engine:                  │  │
│  │ - Unbound DNS & mTLS    │         │   ├─ Ollama (GPU Port 11434)      │  │
│  │ - Default-Deny Egress   │         │   ├─ PostgreSQL (Port 5432)       │  │
│  └─────────────────────────┘         │   ├─ App Services (Laravel/Node)  │  │
│                                      │   └─ NVR / Vector DB (Qdrant)     │  │
│                                      └───────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────────────────┘

Hardware Sizing & PCI Topology Requirements

To run this stack reliably on a single chassis without performance degradation:

  • CPU: 8+ Physical Cores (AMD Ryzen 9 7900X / 9900X or Intel Core i7/i9 14th Gen) with IOMMU / VT-d support.
  • RAM: 64GB DDR5 ECC / Non-ECC (8GB allocated to OPNsense; 48GB allocated to AI Worker VM; 8GB reserved for Proxmox Host).
  • Physical NICs: At least 2 physical Gigabit / 2.5GbE Ethernet ports (one dedicated for WAN ingress, one for LAN switch uplink).
  • GPU: 1x NVIDIA RTX 3090 / 4090 (24GB VRAM) for Ollama inference.
  • Storage: Dual NVMe SSDs in ZFS Mirror (RAID 1) for boot integrity and high IOPS database operations.

Step 1: Proxmox Host Configuration & IOMMU Setup

Enable hardware virtualization and IOMMU groups in /etc/default/grub:

# For AMD Processors:
GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

# For Intel Processors:
# GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"

Update grub and configure kernel VFIO modules in /etc/modules:

sudo update-grub

# Append VFIO drivers for GPU passthrough
cat <<EOF | sudo tee -a /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
EOF

Reboot the host machine:

sudo reboot

Verify that IOMMU groups are isolated:

dmesg | grep -E "DMAR|IOMMU"

Step 2: Network Bridge Configuration in Proxmox (/etc/network/interfaces)

Configure two Linux Bridges:

  • vmbr0: Connected to the physical LAN interface.
  • vmbr1: Connected to the physical WAN interface (Internet modem).
# Physical WAN Interface (Attached to Modem)
auto eth0
iface eth0 inet manual

# Physical LAN Interface (Attached to Local Switch)
auto eth1
iface eth1 inet manual

# LAN Bridge (Internal Virtual Switch)
auto vmbr0
iface vmbr0 inet static
    address 192.168.1.2/24
    gateway 192.168.1.1
    bridge-ports eth1
    bridge-stp off
    bridge-fd 0
    bridge-vlan-aware yes

# WAN Bridge (Dedicated to OPNsense WAN Interface)
auto vmbr1
iface vmbr1 inet manual
    bridge-ports eth0
    bridge-stp off
    bridge-fd 0

Step 3: Configuring the AI Worker VM with GPU Passthrough

  1. Create a Linux VM (ID 200) in Proxmox with Machine Type: q35 and BIOS: OVMF (UEFI).
  2. Under Hardware > Add > PCI Device, select your NVIDIA GPU.
    • Check All Functions (to pass both GPU compute and audio controller).
    • Check Primary GPU (allows display output driver initialization).
    • Check PCI-Express.
  3. Under Hardware > Network Device, attach to vmbr0 with VLAN Tag: 50 (isolates the AI worker on a dedicated firewall interface).

Step 4: Docker Compose Manifest for Consolidated Services

Inside the AI Worker VM (/opt/services/docker-compose.yml), orchestrate Ollama, PostgreSQL, and application containers:

version: '3.8'

services:
  # 1. Ollama LLM Inference Engine
  ollama:
    image: ollama/ollama:latest
    container_name: local-ai-ollama
    restart: unless-stopped
    ports:
      - "11434:11434"
    volumes:
      - ollama_weights:/root/.ollama
    environment:
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_KEEP_ALIVE=24h
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    networks:
      - internal_stack

  # 2. Production PostgreSQL Database (Used by Laravel / Local Apps)
  postgres:
    image: postgres:16-alpine
    container_name: local-postgres
    restart: unless-stopped
    environment:
      POSTGRES_USER: app_admin
      POSTGRES_PASSWORD: super_secure_db_password_123
      POSTGRES_DB: production_db
    volumes:
      - postgres_data:/var/lib/postgresql/data
    ports:
      - "127.0.0.1:5432:5432" # Bind only to localhost on the VM
    networks:
      - internal_stack

  # 3. Application Ingress / Web Layer (e.g., Laravel / Node.js)
  app:
    image: mycompany/core-app:latest
    container_name: app-service
    restart: unless-stopped
    ports:
      - "8080:80"
    environment:
      - DB_HOST=postgres
      - OLLAMA_API_URL=http://ollama:11434
    depends_on:
      - postgres
      - ollama
    networks:
      - internal_stack

volumes:
  ollama_weights:
  postgres_data:

networks:
  internal_stack:
    driver: bridge

Step 5: OPNsense Micro-Segmentation Firewall Rules

To protect your internal network if a web service or AI container is compromised, enforce these rules in OPNsense (Firewall > Rules > AI_VLAN50):

OPNsense AI_VLAN50 Rules Table:
┌────────┬──────────────────────┬─────────────┬───────────────────────────┬────────┐
│ Action │ Source               │ Port        │ Destination               │ Port   │
├────────┼──────────────────────┼─────────────┼───────────────────────────┼────────┤
│ PASS   │ AI_VLAN50 net        │ *           │ AI_VLAN50 Address (DNS)   │ 53     │
│ PASS   │ Trusted_LAN net      │ *           │ AI_Worker_IP              │ 11434  │
│ PASS   │ Trusted_LAN net      │ *           │ AI_Worker_IP              │ 8080   │
│ BLOCK  │ AI_VLAN50 net        │ *           │ Trusted_LAN net (LAN)     │ *      │
│ BLOCK  │ AI_VLAN50 net        │ *           │ WAN / Internet (No Egress)│ *      │
└────────┴──────────────────────┴─────────────┴───────────────────────────┴────────┘
  • Zero Outbound Telemetry: The AI server cannot initiate outbound connections to the internet, preventing model weights or prompt leakage.
  • Controlled Management Access: Only trusted workstations on the primary LAN can reach the Ollama API port (11434) and web portal (8080).

Frequently Asked Questions

Q: Why not run Docker directly on bare metal without Proxmox?

A: Running OPNsense requires a FreeBSD kernel. Installing Docker directly alongside OPNsense on FreeBSD is unsupported and creates serious security vulnerabilities on a firewall router. A Type-1 hypervisor like Proxmox provides strict hardware separation and instant snapshot rollbacks.

Q: Will GPU passthrough add latency to Ollama inference?

A: No. PCIe passthrough via VFIO gives the guest VM direct raw access to the physical GPU hardware with near 0% virtualization overhead.

Q: How do I handle CCTV/NVR streams without choking the CPU?

A: Assign 4 dedicated vCPUs to your NVR container (e.g. Frigate) and configure Intel QuickSync (iGPU) or NVIDIA Tensor Cores for hardware-accelerated video decoding.

  1. How to Secure a Private AI Server with OPNsense
    • Anchor text: How to Secure a Private AI Server with OPNsense
    • Why it is relevant: In-depth walkthrough on creating strict floating firewall rules and VLAN aliases.
  2. Deploying Ollama with Docker Compose: Production Guide
    • Anchor text: Deploying Ollama with Docker Compose: Production Guide
    • Why it is relevant: Step-by-step instructions for containerizing Ollama with multi-model caching.
  3. Best VPS Providers for Self-Hosted AI Workloads
    • Anchor text: Best VPS Providers for Self-Hosted AI Workloads
    • Why it is relevant: Compares when to keep workloads on-prem vs offloading to high-VRAM cloud GPU instances.

Authoritative External Sources

  1. Proxmox VE PCIe Passthrough Documentation (https://pve.proxmox.com/wiki/PCI_Passthrough): Official configuration guide for VFIO IOMMU grouping.
  2. OPNsense VLAN Architecture Guide (https://docs.opnsense.org/manual/how-tos/vlan.html): First-party reference for 802.1Q tagged interface setup.
  3. NVIDIA Container Toolkit Official Setup (https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html): Official guide for registering NVIDIA runtime in Docker.

🏷️ SEO Metadata

  • SEO Title: All-in-One AI Server Architecture: OPNsense & Docker on One Host
  • Meta Description: Architecture guide for configuring a single-host sovereign AI server running virtualized OPNsense, Docker containers, PostgreSQL, and GPU passthrough.
  • URL Slug: all-in-one-ai-server-opnsense-docker
  • Primary Keyword: all in one ai server opnsense docker
  • Secondary Keywords: server configuration for ai cctv opnsense, ai enable opnsense, proxmox gpu passthrough ollama, docker vlan opnsense
  • Search Intent: Practical Informational / Advanced Architecture Design Guide

OpsNexusAI Engineering

Verified Lab Publication

OpsNexusAI is a technical laboratory dedicated to sovereign AI infrastructure. Every implementation guide and architectural blueprint published here is tested on physical hardware and isolated networks. Our team specializes in the deployment of private LLMs, network hardening with OPNsense, and enterprise-grade automation patterns.

Editorial Principles Infrastructure Consulting Inquiry & Feedback

Join the OpsNexus Brief

Get technical teardowns on sovereign AI architectures delivered to your inbox.