Proxy Rotation Best Practices to Avoid Bans

Rotating proxies is not hard. Rotating proxies well is surprisingly nuanced. The difference between a scraper that gets 95% success rates and one that gets banned after 200 requests often comes down to rotation strategy - when to rotate, how often, how to handle sessions, and how to detect when an IP has been flagged. This guide covers everything.

When to Rotate IPs

The most common mistake is rotating too aggressively or not enough. Here's the mental model:

Rotate per request when: you're making high-volume requests to the same domain and each request is independent (e.g., checking prices across thousands of SKUs).
Use sticky sessions when: you have multi-step workflows that require session continuity (login → navigate → extract). Switching IPs mid-session is an immediate red flag to most anti-bot systems.
Rotate on error when: you receive a 403, 429, or detect a CAPTCHA. Don't keep hammering a target from an IP that's already been flagged.
Rotate on time threshold when: using sticky sessions, set a maximum duration (10–30 minutes) and rotate even without errors - long-lived sessions from a single IP look suspicious.

Understanding Sticky Sessions

ZentisLabs (and most quality providers) support sticky sessions - a mechanism that pins your traffic to the same exit IP for a defined duration, even as the rotation pool changes around you.

python

import requests

# Sticky sessions: generate sticky lines in the dashboard or via
# POST /api/v1/proxies/generate (e.g. Basic with a 10-minute lifetime) and use
# them exactly as generated. Each line is pinned to one gateway node, so the
# session keeps its IP.
def sticky_proxy_from_line(line: str) -> dict:
    """Turn a generated 'host:port:user:pass' sticky line into requests proxies."""
    host, port, user, password = line.split(":")
    proxy_url = f"http://{user}:{password}@{host}:{port}"
    return {"http": proxy_url, "https": proxy_url}

# Independent tasks → rotating (new IP per request, no session ID)
rotating_proxy = {"http": "http://USER:PASS@basic.resi.zentislabs.com:12321",
                  "https": "http://USER:PASS@basic.resi.zentislabs.com:12321"}

# Multi-step workflow → sticky session (same IP for entire flow)
sticky_proxy = sticky_proxy_from_line("NODE_IP:12321:USER:PASS_country-us_session-a3f8c91b_lifetime-10m")

# Now do your multi-step flow:
session = requests.Session()
session.proxies = sticky_proxy

# Step 1: Load homepage (establishes cookies)
session.get("https://example.com")

# Step 2: Log in (same IP as step 1 - critical)
session.post("https://example.com/login", data={"user": "x", "pass": "y"})

# Step 3: Navigate to target page (still same IP)
result = session.get("https://example.com/protected-data")

# Same IP used throughout - session stays valid

Rotation Strategies

1. Round-Robin (Basic)

Cycle through a fixed list of proxies sequentially. Simple but predictable - if the pattern is detected, all IPs in your list get flagged simultaneously.

python

proxy_list = ["http://user:pass@ip1:port", "http://user:pass@ip2:port", ...]
proxy_index = 0

def get_next_proxy():
    global proxy_index
    proxy = proxy_list[proxy_index % len(proxy_list)]
    proxy_index += 1
    return proxy

2. Random Selection

Select randomly from your pool. Less predictable than round-robin, but you may hit the same IP multiple times in a row by chance.

python

import random

def get_random_proxy(proxy_list):
    return random.choice(proxy_list)

3. Gateway-Based Rotation (Recommended)

Use a rotation gateway like ZentisLabs - each request is automatically assigned a different IP from a pool of 100M+. No list management, no recycling: just point your client at the gateway.

python

# No list needed - the gateway handles rotation automatically
proxy = {"http": "http://USER:PASS@basic.resi.zentislabs.com:12321",
         "https": "http://USER:PASS@basic.resi.zentislabs.com:12321"}

# Every request automatically gets a different residential IP
for url in urls_to_scrape:
    r = requests.get(url, proxies=proxy, timeout=20)
    # Different IP used each time

4. Weighted Rotation

Track success rates per IP and weight selection toward IPs that have been working. Avoid IPs that recently triggered errors.

python

from collections import defaultdict
import random

class ProxyRotator:
    def __init__(self, proxies):
        self.proxies = proxies
        self.success_counts = defaultdict(int)
        self.failure_counts = defaultdict(int)
        self.cooldown = {}  # IPs cooling down after failures

    def get_proxy(self):
        available = [p for p in self.proxies if p not in self.cooldown]
        if not available:
            self.cooldown.clear()  # Reset cooldowns if all proxies cooling
            available = self.proxies
        
        # Weight by success rate
        weights = []
        for p in available:
            total = self.success_counts[p] + self.failure_counts[p]
            success_rate = self.success_counts[p] / total if total > 0 else 0.5
            weights.append(max(0.1, success_rate))
        
        return random.choices(available, weights=weights)[0]

    def record_success(self, proxy):
        self.success_counts[proxy] += 1

    def record_failure(self, proxy, cooldown_seconds=60):
        self.failure_counts[proxy] += 1
        import time
        self.cooldown[proxy] = time.time() + cooldown_seconds

Detecting When an IP Gets Banned

Don't just check HTTP status codes. Sophisticated sites return 200 with a CAPTCHA or honeypot response. Build a proper ban detector:

python

def is_blocked(response) -> bool:
    """Detect ban/block responses, including soft blocks that return 200."""
    
    # Hard blocks
    if response.status_code in (403, 429, 503):
        return True
    
    # Redirect to challenge page
    if "challenge" in response.url or "verify" in response.url:
        return True
    
    content = response.text.lower()
    
    # CAPTCHA indicators
    captcha_markers = [
        "g-recaptcha", "h-captcha", "cf-challenge",
        "please complete the security check",
        "verify you are human",
        "access denied",
        "unusual traffic",
    ]
    if any(marker in content for marker in captcha_markers):
        return True
    
    # Content anomaly: expected data missing
    # (customize for your target)
    if "expected-element-class" not in content and response.status_code == 200:
        return True
    
    return False


def scrape_with_rotation(url, max_retries=5):
    for attempt in range(max_retries):
        proxy = get_next_proxy()  # Your rotation method
        try:
            r = requests.get(url, proxies={"https": proxy}, timeout=20)
            if not is_blocked(r):
                return r
            print(f"Blocked on attempt {attempt+1}, rotating IP...")
        except Exception as e:
            print(f"Error: {e}, rotating IP...")
    
    raise Exception(f"Failed after {max_retries} attempts with IP rotation")

Rotation Patterns That Trigger Detection

Even with good proxies, these patterns will get you flagged:

Same request headers, different IP every time: A constant User-Agent across thousands of IPs is a fingerprint. Rotate your User-Agent alongside your IP.
Machine-speed timing: Requests at exactly 500ms intervals, every time, from thousands of IPs screams automation. Add random jitter: time.sleep(random.uniform(0.5, 3.0)).
IP switching mid-session: If you've started a session (loaded a page, set cookies) and then switch IPs, the session–IP mismatch is an immediate red flag.
Subnet clustering: Rotating between IPs in the same /24 subnet (256 IPs) means when one gets flagged, the whole subnet often does. Good rotation spreads across diverse subnets.
Missing common headers: Real browsers send 15–20 headers. If you send 3 headers from thousands of IPs, that's a fingerprint pattern that doesn't correlate with any real browser.

Header Rotation

python

import random

USER_AGENTS = [
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:122.0) Gecko/20100101 Firefox/122.0",
    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.2 Safari/605.1.15",
]

def get_headers(accept_language: str = "en-US,en;q=0.9") -> dict:
    return {
        "User-Agent": random.choice(USER_AGENTS),
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8",
        "Accept-Language": accept_language,
        "Accept-Encoding": "gzip, deflate, br",
        "DNT": "1",
        "Connection": "keep-alive",
        "Upgrade-Insecure-Requests": "1",
        "Sec-Fetch-Dest": "document",
        "Sec-Fetch-Mode": "navigate",
        "Sec-Fetch-Site": "none",
    }

# Rotate both proxy AND headers on each request
r = requests.get(url, proxies=get_proxy(), headers=get_headers(), timeout=20)

Programmatic Rotation Control

Rotating proxies change IP on every request; for a new sticky IP, generate a new sticky line. Location targeting is part of the proxy password, so you can build it in code:

python

def get_geo_specific_proxy(country: str, state: str = None, city: str = None) -> dict:
    """Get a proxy pinned to a specific geographic location (a city needs its state)."""
    target = f"country-{country}"
    if state:
        target += f"_state-{state.replace(' ', '').lower()}"
        if city:
            target += f"-city-{city.replace(' ', '').lower()}"
    proxy_url = f"http://USER:PASS_{target}@basic.resi.zentislabs.com:12321"
    return {"http": proxy_url, "https": proxy_url}

Key rotation principle: Match your rotation strategy to your session structure. Independent requests → rotate freely. Multi-step flows → sticky sessions. Error → immediately rotate and back off. Time threshold → rotate proactively before detection occurs.

Proxy Rotation Best Practices: Avoid Bans and Maximize Success Rates

When to Rotate IPs

Understanding Sticky Sessions

Rotation Strategies

1. Round-Robin (Basic)

2. Random Selection

3. Gateway-Based Rotation (Recommended)

4. Weighted Rotation

Detecting When an IP Gets Banned

Rotation Patterns That Trigger Detection

Header Rotation

Programmatic Rotation Control

Ready to get started?

How to Set Up a Rotating Proxy in Python, Node.js, and Bash (2025 Guide)

How to Size a VPS for Web Scraping (2025 Guide)

Deploy Ollama on a VPS: Run LLMs Privately in 10 Minutes

When to Rotate IPs

Understanding Sticky Sessions

Rotation Strategies

1. Round-Robin (Basic)

2. Random Selection

3. Gateway-Based Rotation (Recommended)

4. Weighted Rotation

Detecting When an IP Gets Banned

Rotation Patterns That Trigger Detection

Header Rotation

Programmatic Rotation Control

Ready to get started?

Related Articles

How to Set Up a Rotating Proxy in Python, Node.js, and Bash (2025 Guide)

How to Size a VPS for Web Scraping (2025 Guide)

Deploy Ollama on a VPS: Run LLMs Privately in 10 Minutes