Load Balancing

Every proxy component in rust-web-server — ReverseProxy, H2ReverseProxy, GrpcProxy, TcpProxy, UdpProxy, WsProxy, and DynamicProxy (config-driven mode) — uses the same underlying selection mechanism: a lock-free atomic round-robin counter.

How round-robin works#

Each proxy struct holds an AtomicUsize counter initialised to 0. On every request the counter is incremented with Ordering::Relaxed and the backend index is computed as:

let idx = counter.fetch_add(1, Ordering::Relaxed) % backends.len();

No mutex is acquired; the operation is a single CPU instruction on every major architecture. Under concurrent load requests are distributed evenly across all live backends in cyclic order.

Library API#

use rust_web_server::proxy::{LoadBalancing, ReverseProxy};

// Explicit — same as the default.
let proxy = ReverseProxy::new(["http://a:8080", "http://b:8080", "http://c:8080"])
    .strategy(LoadBalancing::RoundRobin);

The .strategy() method exists to allow future strategies to be added without a breaking API change.

Config-driven mode#

In config-driven mode, [[upstream]]'s strategy field selects between four strategies:

[[upstream]]
name     = "api"
backends = ["api-1:3000", "api-2:3000", "api-3:3000"]
strategy = "least_connections"   # "round_robin" (default) | "random" | "ip_hash" | "least_connections"

An unrecognized or omitted strategy value falls back to round_robin.

weights = [3, 1, 1] (one per backend) makes round_robin, random and least_connections proportional — smooth weighted round-robin, so 3:1:1 goes a b a c a rather than a a a b c — and [upstream.sticky] pins a client to a backend with a cookie. See Weights, stickiness, retries and the circuit breaker.

DynamicProxy and live backend lists#

When [upstream.health_check] is configured, the health-checker thread maintains a shared Arc<RwLock<Vec<String>>> containing only the currently healthy backends. DynamicProxy (the config-driven proxy adapter) reads from this live list on every request:

All backends:  [api-1, api-2, api-3]
                          ↓  health checker removes api-2
Live backends: [api-1, api-3]
                          ↓  round-robin counter % 2
Request N+0 → api-1
Request N+1 → api-3
Request N+2 → api-1

The live list is protected by RwLock — reads are concurrent; the health-checker thread takes a brief write lock once per probe round to publish the list.

Failover behaviour#

ReverseProxy tries backends in order starting from the round-robin cursor position. If a backend fails to connect, the next backend in the list is tried. This continues until either a backend succeeds or all backends have been tried (returning 502).

backends = [A, B, C]
counter = 7 → idx 7 % 3 = 1 → start at B
B fails → try C → C succeeds → return response

Canary / weighted round-robin#

CanaryLayer implements weighted round-robin by pre-expanding the rotation: a backend with weight = 3 appears three times in the rotation array. The same lock-free atomic counter selects slots.

// 3 out of 4 slots → stable; 1 out of 4 → canary.
CanaryLayer::new(vec![
    WeightedBackend::new("http://stable:8080", 3),
    WeightedBackend::new("http://canary:8080", 1),
])

See Canary / Traffic Splitting for full details.