Hardening Open-WebUI: Implementing Llama-Guard & Rate Limiting at the Reverse Proxy Layer

Overview

Open-WebUI provides a sleek interface for self-hosted AI models. However, exposing Open-WebUI endpoints to team members or external users without request filtering opens your private AI infrastructure to Prompt Injection, Jailbreak Exploits, and Resource Exhaustion Denial-of-Service (DoS) attacks.

This guide demonstrates how to deploy an Nginx proxy sidecar with rate limiting and a Llama-Guard safety filter sidecar to sanitize user inputs before they reach core LLM engines.

---

Defense-in-Depth Architecture

┌────────────────────────────────────────────────────────┐
│                   INCOMING USER PROMPT                 │
└─────────────────────────┬──────────────────────────────┘
                          │
                          ▼
┌────────────────────────────────────────────────────────┐
│             NGINX HARDENED PROXY SIDECAR               │
│  - Rate Limiting (10 req/min per IP)                   │
│  - Payload Size Limits (Max 100KB)                     │
└─────────────────────────┬──────────────────────────────┘
                          │
                          ▼
┌────────────────────────────────────────────────────────┐
│              LLAMA-GUARD SAFETY FILTER                 │
│  - Scans prompt for unsafe categories / injection      │
│  - Blocks malicious jailbreaks                         │
└─────────────────────────┬──────────────────────────────┘
                          │ Passed Verification
                          ▼
┌────────────────────────────────────────────────────────┐
│             OPEN-WEBUI & BACKEND LLM ENGINE            │
└────────────────────────────────────────────────────────┘

---

Step 1: Production Nginx Security Configuration

Create /opt/openwebui-security/nginx.conf:

events {
    worker_connections 1024;
}

http {
    # Rate Limiting Zone: 10 requests per minute per IP
    limit_req_zone $binary_remote_addr zone=ai_limit:10m rate=10r/m;
    limit_conn_zone $binary_remote_addr zone=addr_limit:10m;

    server {
        listen 80;
        server_name ai.opsnexus.internal;

        # Enforce payload size limit (prevents huge context buffer attacks)
        client_max_body_size 100k;

        location /api/v1/chat/completions {
            limit_req zone=ai_limit burst=5 nodelay;
            limit_conn addr_limit 5;

            proxy_pass http://openwebui:8080;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
        }

        location / {
            proxy_pass http://openwebui:8080;
        }
    }
}

---

Step 2: Deploying Llama-Guard Safety Middleware

Deploy Llama-Guard via vLLM or Ollama to act as an automated input guardrail.

Create /opt/openwebui-security/docker-compose.yml:

version: "3.8"

services:
  openwebui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: openwebui_core
    restart: unless-stopped
    environment:
      - OLLAMA_BASE_URL=http://llama_guard:11434
    networks:
      - ai_secure_net

  llama_guard:
    image: ollama/ollama:latest
    container_name: llama_guard_filter
    restart: unless-stopped
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]
    networks:
      - ai_secure_net

  nginx_proxy:
    image: nginx:1.26-alpine
    container_name: ai_nginx_proxy
    restart: unless-stopped
    ports:
      - "80:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - openwebui
    networks:
      - ai_secure_net

networks:
  ai_secure_net:
    driver: bridge

volumes:
  ollama_data:

---

Step 3: Verification & Security Testing

1. Test Rate Limiting Trigger

Execute rapid synthetic requests to verify HTTP 429 response:

for i in {1..15}; do curl -I http://localhost/api/v1/chat/completions; done

Expected Output: HTTP/1.1 429 Too Many Requests

---

Security Diagnostic Matrix

Attack Vector	Countermeasure	Verification
Prompt Injection	Llama-Guard 3 evaluation filter	Scans inputs against 13 unsafe categories.
DoS Context Flood	client_max_body_size 100k in Nginx	Drops payloads > 100KB with HTTP 413.
API Exhaustion	Nginx limit_req rate limiting	Enforces strict requests-per-minute per client IP.

1. Host Preparation & Kernel Optimization

Configure host kernel parameters to support high-throughput network sockets and memory allocation for containerized workloads. Edit /etc/sysctl.d/99-sovereign-ai.conf:

# Sovereign AI Host Tuning Parameters
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
fs.file-max = 2097152
vm.max_map_count = 262144

Apply sysctl changes:

sudo sysctl --system

2. Production Docker Service Specification

Deploy the container stack using hardened runtime security parameters, non-root user mappings, and explicit resource allocations:

version: "3.8"

services:
  core_workload:
    image: opsnexus/engine:v1.4
    container_name: sovereign_core_workload
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    cap_drop:
      - ALL
    cap_add:
      - SYS_ADMIN
    environment:
      - LOG_LEVEL=info
      - METRICS_ENABLED=true
    volumes:
      - /opt/data:/app/data:rw
    ports:
      - "127.0.0.1:8080:8080"
    deploy:
      resources:
        limits:
          cpus: '8.0'
          memory: 32G
        reservations:
          cpus: '4.0'
          memory: 16G

---

Failure Mode 1: High VRAM Fragmentation under Continuous Batching

During extended execution, dynamic memory allocation can create fragmented VRAM memory blocks, causing unexpected out-of-memory errors even when reported total free memory appears sufficient.

• Diagnostic Command: Run nvidia-smi --query-gpu=memory.used,memory.free --format=csv every second during peak request surges.
• Resolution: Enable unified memory allocators or set PYTORCH_CUDA_ALLOC_CONF="expandable_segments:True".

Failure Mode 2: Container Bridge Network Throughput Degradation

Heavy payload transfers between containerized proxy sidecars and backend engines can saturate default Docker bridge network buffers.

• Diagnostic Command: Test inter-container bandwidth using iperf3 between connected container sockets.
• Resolution: Tune network socket buffer sizes or utilize host networking mode (network_mode: "host") for low-latency internal traffic.

---

Llama-Guard 3 Safety Taxonomy Integration

Llama-Guard 3 evaluates incoming user prompts against 13 security risk categories including:

1. S1: Violent Crimes
2. S2: Non-Violent Crimes
3. S3: Sex-Related Crimes
4. S4: Child Sexual Exploitation
5. S5: Defamation & Hate Speech
6. S6: Specialized Advice (Medical/Financial)
7. S7: Privacy Violations & Exfiltration
8. S8: Intellectual Property Violations
9. S9: Indiscriminate Weapons (CBRN)
10. S10: Hate Speech
11. S11: Suicide or Self-Harm
12. S12: Sexual Content
13. S13: Cyberattacks & Code Injection

Configuring Llama-Guard Prompt Interception

In Open-WebUI admin panel under Modelfiles, set the system prompt filter:

FROM llama3.1
SYSTEM "Task: Check if there is a safety violation in the User message according to our safety categories. If safe, output 'safe'. If unsafe, output 'unsafe' followed by the category code."