Skip to content
20% OFFCode ZENTIS20 = 20% offSign Up
Back to Blog

AI VPS

Deploy Ollama on a VPS: Run LLMs Privately in 10 Minutes

ZentisLabs Team8 min read

Running LLMs through a third-party API means every prompt you send is logged, used for training, and subject to rate limits and outages outside your control. For production applications, privacy-sensitive use cases, or any workload that needs consistent, private inference - running your own model on a VPS is the right call. And with Ollama, it's genuinely a 10-minute setup.

Why Run LLMs on Your Own VPS?

  • Privacy: Your prompts never leave your infrastructure. Critical for legal, medical, financial, or proprietary data.
  • No API costs: Run Llama 3.2, Mistral, or Qwen2.5 indefinitely for the fixed cost of your VPS - no per-token billing.
  • Full control: Customize system prompts, fine-tune on your data, adjust temperature and parameters without restrictions.
  • Consistent latency: No third-party API queues and no rate limits throttling your application.
  • Offline capability: Once deployed, your stack works independently of external APIs.

What Specs Do You Actually Need?

Model size (in parameters) determines your minimum RAM/VRAM. Here's the practical breakdown:

Model SizeExamplesMin RAM (CPU)VRAM (GPU)Speed (CPU)
1–3BLlama 3.2 1B, Phi-3 mini4 GB4 GBFast (10–30 tok/s)
7–8BLlama 3.1 8B, Mistral 7B8 GB8 GBOK (5–15 tok/s)
13BLlama 2 13B, CodeLlama 13B16 GB12 GBSlow (2–5 tok/s)
30–34BCodeLlama 34B, Qwen 32B32 GB24 GBVery slow (<2 tok/s)
70BLlama 3.1 70B, Qwen 72B64 GB48 GB VRAMImpractical on CPU

For most use cases, an 8B model on a 16GB RAM VPS is the sweet spot. You get GPT-3.5-level quality at zero per-token cost. ZentisLabs AI Advanced (12 vCPU, 32GB RAM) runs quantized models up to ~32B parameters on CPU.

Step 1: Choose Your VPS

ZentisLabs AI server plans are sized for LLM inference: CPU cloud servers with up to 32 GB of RAM, room for your model files, and a dedicated-CPU option for steady throughput:

AI Starter

$84/mo

  • 4 vCPU
  • 8 GB RAM
  • 100 GB storage
  • Up to 8B models

AI Business

$158/mo

  • 8 vCPU
  • 16 GB RAM
  • 200 GB storage
  • Up to 13B models

AI Advanced

$263/mo

  • 12 vCPU
  • 32 GB RAM
  • 300 GB storage
  • Up to ~32B models (quantized)

Step 2: Install Ollama

SSH into your VPS and run the one-line installer:

bash
# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh


# Verify installation
ollama --version


# Start the Ollama service (runs on port 11434 by default)
ollama serve &

Step 3: Pull and Run a Model

bash
# Pull Llama 3.2 (3B  fast, 2GB download)
ollama pull llama3.2


# Pull Llama 3.1 8B (recommended starting point)
ollama pull llama3.1


# Pull Mistral 7B (excellent for code/reasoning)
ollama pull mistral


# Pull Qwen2.5 Coder (best for coding tasks)
ollama pull qwen2.5-coder


# Run interactively
ollama run llama3.1


# Run via API (from another terminal)
curl http://localhost:11434/api/generate -d '{
  "model": "llama3.1",
  "prompt": "Explain proxy rotation in one paragraph",
  "stream": false
}'

Step 4: Set Up Open WebUI (Optional but Recommended)

Open WebUI gives you a ChatGPT-like interface for your models. Install with Docker:

bash
# Install Docker if not already installed


curl -fsSL https://get.docker.com | sh






# Run Open WebUI (connects to local Ollama automatically)


docker run -d 
  -p 3000:8080 
  --add-host=host.docker.internal:host-gateway 
  -v open-webui:/app/backend/data 
  --name open-webui 
  --restart always 
  ghcr.io/open-webui/open-webui:main






# Access at http://YOUR_VPS_IP:3000

Step 5: Configure Nginx Reverse Proxy with SSL

Expose your setup at a proper domain with HTTPS:

bash
# Install nginx and certbot
apt install nginx certbot python3-certbot-nginx -y


# Create nginx config
cat > /etc/nginx/sites-available/ollama << 'EOF'
server {
    server_name your-domain.com;


    # Proxy to Open WebUI
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;
    }


    # Proxy Ollama API (optional  secure this with auth!)
    location /api/ {
        proxy_pass http://localhost:11434/api/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}
EOF


# Enable site
ln -s /etc/nginx/sites-available/ollama /etc/nginx/sites-enabled/
nginx -t && systemctl reload nginx


# Get SSL certificate
certbot --nginx -d your-domain.com

Step 6: Systemd Auto-Restart

bash
# Create systemd service for Ollama
cat > /etc/systemd/system/ollama.service << 'EOF'
[Unit]
Description=Ollama LLM Server
After=network.target


[Service]
Type=simple
User=root
ExecStart=/usr/local/bin/ollama serve
Restart=always
RestartSec=5
Environment=OLLAMA_HOST=0.0.0.0


[Install]
WantedBy=multi-user.target
EOF


# Enable and start
systemctl daemon-reload
systemctl enable ollama
systemctl start ollama


# Check status
systemctl status ollama

Using Your Deployment from Code

python
import requests


OLLAMA_URL = "https://your-domain.com/api"  # Or http://your-vps-ip:11434/api


def chat(prompt, model="llama3.1"):
    response = requests.post(
        f"{OLLAMA_URL}/generate",
        json={"model": model, "prompt": prompt, "stream": False},
        timeout=120
    )
    return response.json()["response"]


# Use like any LLM API
reply = chat("Write a Python function to rotate proxies with retry logic")
print(reply)


# OpenAI-compatible endpoint (Ollama v0.1.24+)
from openai import OpenAI


client = OpenAI(
    base_url="https://your-domain.com/v1",
    api_key="ollama"  # Not validated, just required by the SDK
)


response = client.chat.completions.create(
    model="llama3.1",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

Bonus: One-Click Ollama + Open WebUI

Don't want to run through the steps above? Pick Ollama + Open WebUI as a one-click app when you create a ZentisLabs AI server: the installer sets up Ollama and Open WebUI and pulls Llama 3.2 and Mistral for you, so your private ChatGPT alternative is ready within minutes. Add your own domain and SSL with Step 5.

Match the plan to your model: AI Starter ($84/mo, 8 GB RAM) for 7-8B models, AI Business ($158/mo, 16 GB) for 13B, and AI Advanced ($263/mo, 32 GB) for quantized models up to ~32B parameters. All run on CPU with full root access, so you can swap models whenever you like.

Ready to get started?

Try ZentisLabs free - 100MB residential bandwidth included.

Get 100MB Free

Related Articles