Deploying Autonomous AI Agents with Docker Compose: Ollama, LiteLLM & n8n

Autonomous AI agents—systems that execute multi-step reasoning, query vector databases, browse the web, and call external APIs—require fundamentally different infrastructure than traditional web applications. Running agents locally or on a private VPS demands a resilient orchestration layer that handles continuous token streams, rate-limiting, local model fallbacks, and persistent workflow execution.

This production guide walks through deploying a self-hosted, sovereign multi-agent architecture on Linux using Docker Compose, integrating:

1. Ollama for local, private open-weights inference (e.g., Llama 3.2, Mistral).
2. LiteLLM Proxy as an OpenAI-compatible gateway with load-balancing and budget controls.
3. n8n (Self-Hosted) with PostgreSQL for visual agent workflow design and tool execution.

---

High-Level Multi-Agent Architecture

External Webhook / User Ingress
               │
               ▼
┌─────────────────────────────────────────────────────────────┐
│  n8n Agent Workflow Orchestrator                           │
│  - Multi-Step Reasoning Loops                               │
│  - Tool Calling (HTTP Requests, Python Code, Web Scrapers)  │
│  - PostgreSQL Execution State & Chat Memory                 │
└──────────────────────────────┬──────────────────────────────┘
                               │ (OpenAI-Compatible API Calls)
                               ▼
┌─────────────────────────────────────────────────────────────┐
│  LiteLLM Proxy Gateway (Port 4000)                          │
│  - Model Fallback Routing (Local Ollama ──▶ Cloud Claude/GPT)│
│  - Request Logging, API Key Auth & Token Budgeting          │
└──────────────┬──────────────────────────────┬───────────────┘
               │                              │
       (Local GPU Inference)          (External HTTPS API)
               ▼                              ▼
┌──────────────────────────────┐    ┌───────────────────────────┐
│ Ollama LLM Runner (GPU/CPU)  │    │ Commercial APIs           │
│ - Llama 3.2 (Reasoning)      │    │ - Anthropic Claude 3.5    │
│ - Nomic-Embed (Vectors)      │    │ - OpenAI GPT-4o           │
└──────────────────────────────┘    └───────────────────────────┘

---

Hardware & System Prerequisites

• Operating System: Ubuntu 22.04 / 24.04 LTS or Debian 12.
• CPU / RAM: Minimum 4 Dedicated vCPUs and 16GB System RAM (8GB minimum if running only cloud models; 16GB+ if running local 8B models).
• Storage: 50GB+ NVMe SSD storage for Docker volumes and model checkpoints.
• GPU (Optional): NVIDIA GPU with 8GB+ VRAM for accelerated local inference via NVIDIA Container Toolkit.

---

Step 1: Directory Setup & Configuration Files

Create a dedicated project directory on your server:

mkdir -p /opt/ai-agent-stack/{n8n_data,ollama_data,litellm_config,postgres_data}
cd /opt/ai-agent-stack

---

Step 2: Configure LiteLLM Routing (litellm_config/config.yaml)

LiteLLM acts as a central switchboard. It maps multiple model aliases so that your n8n agents can request agent-brain and LiteLLM automatically routes to your local Ollama instance, falling back to external cloud APIs if local VRAM is exhausted.

Create /opt/ai-agent-stack/litellm_config/config.yaml:

model_list:
  # Primary: Local Ollama Model (Fast, Zero Cost)
  - model_name: agent-brain
    litellm_params:
      model: ollama/llama3.2:3b
      api_base: http://ollama:11434

  # Fallback: Commercial Cloud API (Complex Multi-Step Coding/Logic)
  - model_name: agent-brain-heavy
    litellm_params:
      model: anthropic/claude-3-5-sonnet-20241022
      api_key: "os.environ/ANTHROPIC_API_KEY"

  # Local Embeddings for Vector Memory & RAG
  - model_name: text-embedding
    litellm_params:
      model: ollama/nomic-embed-text
      api_base: http://ollama:11434

litellm_settings:
  drop_params: true
  set_verbose: false

---

Step 3: Complete Production docker-compose.yml

Create /opt/ai-agent-stack/docker-compose.yml:

version: '3.8'

services:
  # 1. PostgreSQL Database for n8n Workflow State & Execution History
  postgres:
    image: postgres:16-alpine
    container_name: agent-postgres
    restart: unless-stopped
    environment:
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: ${POSTGRES_DB}
    volumes:
      - ./postgres_data:/var/lib/postgresql/data
    networks:
      - agent_network
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER} -d ${POSTGRES_DB}"]
      interval: 5s
      timeout: 5s
      retries: 5

  # 2. Ollama Local LLM Inference Engine
  ollama:
    image: ollama/ollama:latest
    container_name: agent-ollama
    restart: unless-stopped
    environment:
      - OLLAMA_HOST=0.0.0.0
      - OLLAMA_KEEP_ALIVE=24h
      - OLLAMA_NUM_PARALLEL=4
    volumes:
      - ./ollama_data:/root/.ollama
    networks:
      - agent_network
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  # 3. LiteLLM Gateway / OpenAI Proxy
  litellm:
    image: ghcr.io/berriai/litellm:main-latest
    container_name: agent-litellm
    restart: unless-stopped
    ports:
      - "4000:4000"
    environment:
      - LITELLM_MASTER_KEY=${LITELLM_MASTER_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    volumes:
      - ./litellm_config/config.yaml:/app/config.yaml
    command: ["--config", "/app/config.yaml", "--port", "4000"]
    networks:
      - agent_network
    depends_on:
      - ollama

  # 4. n8n Autonomous Workflow & Agent Runtime
  n8n:
    image: docker.n8n.io/n8nio/n8n:latest
    container_name: agent-n8n
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      - DB_TYPE=postgresdb
      - DB_POSTGRESDB_HOST=postgres
      - DB_POSTGRESDB_PORT=5432
      - DB_POSTGRESDB_DATABASE=${POSTGRES_DB}
      - DB_POSTGRESDB_USER=${POSTGRES_USER}
      - DB_POSTGRESDB_PASSWORD=${POSTGRES_PASSWORD}
      - N8N_ENFORCE_SETTINGS_FILE_PERMISSIONS=true
      - N8N_DIAGNOSTICS_ENABLED=false
      - N8N_HOST=${N8N_HOST}
      - WEBHOOK_URL=https://${N8N_HOST}/
    volumes:
      - ./n8n_data:/home/node/.n8n
    networks:
      - agent_network
    depends_on:
      postgres:
        condition: service_healthy
      litellm:
        condition: service_started

networks:
  agent_network:
    driver: bridge

---

Step 4: Environment Variables (.env)

Create /opt/ai-agent-stack/.env:

# PostgreSQL Credentials
POSTGRES_USER=n8n_agent_admin
POSTGRES_PASSWORD=generate_a_strong_database_password_here
POSTGRES_DB=n8n_agent_db

# LiteLLM Proxy Master API Key (Passed to n8n)
LITELLM_MASTER_KEY=sk-litellm-master-agent-key-9988

# Cloud LLM Fallback Keys (Optional)
ANTHROPIC_API_KEY=sk-ant-api03-xxxx
OPENAI_API_KEY=sk-proj-xxxx

# n8n Domain Configuration
N8N_HOST=agents.yourdomain.com

---

Step 5: Launching & Initializing Models

Start the full stack in detached mode:

cd /opt/ai-agent-stack
docker compose up -d

Pull the required local reasoning and embedding models into Ollama:

# 1. Pull lightweight Llama 3.2 3B for fast agent tool calling
docker exec -it agent-ollama ollama pull llama3.2:3b

# 2. Pull high-performance vector embedding model
docker exec -it agent-ollama ollama pull nomic-embed-text

# 3. Verify models are loaded in Ollama
docker exec -it agent-ollama ollama list

---

Step 6: Connecting n8n to the LiteLLM Proxy

1. Open http://YOUR_SERVER_IP:5678 (or your reverse-proxied HTTPS domain).
2. Complete the initial admin account creation.
3. In the n8n canvas, add an AI Agent Node (or OpenAI Chat Model Node).
4. Create new OpenAI Credentials:
   • API Key: sk-litellm-master-agent-key-9988 (from your .env)
   • Base URL: http://litellm:4000/v1
5. Set Model Name to: agent-brain.

Test the connection. n8n will dispatch requests directly to LiteLLM, which proxies them seamlessly to local Ollama with zero external telemetry.

---

Production Security & Hardening Best Practices

• Isolate PostgreSQL & Ollama Ports: Notice that in the docker-compose.yml, neither postgres nor ollama exposes public ports to the host (ports: is omitted). They communicate exclusively over the internal bridge network agent_network.
• Put n8n Behind SSL Reverse Proxy: Use Caddy or Nginx with Cloudflare Access or Tailscale VPN to prevent public unauthorized access to your workflow execution canvas.
• Set Container Memory Limits: Unconstrained Python scripts executed by n8n agents can cause Out-Of-Memory host crashes. Add mem_limit: 4g to the n8n service block if agents process massive web scraping tasks.

---

Frequently Asked Questions

Q: Can I run this stack without an NVIDIA GPU?

A: Yes. If no GPU is available, remove the deploy.resources.reservations block from the ollama service in docker-compose.yml. Ollama will run in CPU mode using AVX-512 instructions, which is fast enough for smaller 3B parameter models.

Q: Why use LiteLLM between n8n and Ollama instead of connecting directly?

A: LiteLLM provides standard OpenAI API formatting, automatic model fallbacks (e.g., routing to Claude 3.5 if a complex tool-call fails locally), rate limiting, and exact token consumption tracking across multiple workflows.

Q: How do I persist chat memory across multi-agent loops?

A: n8n includes native Window Buffer Memory and Postgres Chat Memory nodes. Attach the Postgres Chat Memory node directly to your Agent node using the internal connection string postgres:5432.

---

Related Guides & Infrastructure Blueprints

• Hardware Sizing: Best VPS Providers for Autonomous AI Agents & Workloads
• High-Throughput Serving: How to Deploy vLLM on a VPS with Docker
• Firewall Hardening: How to Secure a Private AI Server with OPNsense
• GPU Automation: RunPod Terraform Provider: Complete IaC Guide