Load Balancing Algorithms
A load balancer needs to decide: which server gets the next request? That decision is the algorithm. Different algorithms optimize for different things — fairness, speed, session affinity, or cache efficiency. Choosing the right one depends on your workload.
Round Robin
How It Works
Requests go to each server in order: A → B → C → A → B → C → ... Dead simple. Each server gets an equal share.
Best for: Identical servers handling stateless requests of similar duration.
Weakness: Ignores server load — a server processing a heavy request still gets the next one.
Weighted Round Robin
How It Works
Same as round robin, but servers with higher weights get proportionally more requests. A server with weight 3 gets three requests for every one that a weight-1 server gets.
Best for: Heterogeneous fleets — when some servers are beefier than others.
Weakness: Weights are static; doesn't adapt to real-time load.
Least Connections
How It Works
Route each new request to whichever server currently has the fewest active connections. Naturally adapts to servers that process requests at different speeds.
Best for: Requests with highly variable duration (some take 10ms, others take 5s).
Weakness: Requires tracking connection counts — slightly more state to maintain.
Least Response Time
How It Works
Route to the server with the lowest average response time AND fewest active connections. Combines speed measurement with load awareness.
Best for: When you want the fastest user experience and have mixed-capability backends.
Weakness: Requires continuous latency measurement; can oscillate.
IP Hash
How It Works
Hash the client's IP address to determine which server to use. Same IP always goes to the same server (until the server pool changes).
Best for: Session affinity without cookies — same user hits same server.
Weakness: Adding/removing a server reshuffles many clients. Uneven if traffic is concentrated from a few IPs (corporate NATs).
Consistent Hashing
How It Works
Servers are placed on a virtual ring. Each request is hashed to a point on the ring and routed to the next server clockwise. When a server is added/removed, only ~1/N of requests are redistributed.
Best for: Cache servers (Memcached, Redis) — minimizes cache invalidation when scaling.
Weakness: More complex to implement; can still be uneven without virtual nodes.
Visual Comparison
🏢 Real-World: AWS ALB's "Least Outstanding Requests"
AWS Application Load Balancer offers a routing algorithm called Least Outstanding Requests (LOR). It's similar to least connections but counts only in-flight requests that haven't received a response yet. This naturally routes away from slow instances — if a server is taking 5 seconds to respond, its outstanding request count stays high, and new requests go elsewhere. AWS found this reduces p99 latency by 20-30% compared to round robin for variable-duration workloads.
🏢 Real-World: samwho.dev's Visual Load Balancing
Sam Rose's interactive article on load balancing (samwho.dev/load-balancing) beautifully visualizes how algorithms behave under real conditions. It demonstrates how round robin fails when request durations vary wildly — some servers finish instantly while others queue up requests. The article shows why "least connections" or "PEWMA" (Peak Exponentially Weighted Moving Average) adapts better to real-world traffic patterns.
Interactive: Watch Algorithms in Action
Select an algorithm and watch it distribute requests to 4 servers with varying processing times: