Load Balancing

A load balancer sits between clients and your servers, distributing incoming requests so no single server is overwhelmed. Without one, horizontal scaling gives you capacity that clients cannot reach. With one, you can add or remove servers transparently — clients always hit the same address.

Load balancers use different algorithms to route requests. Round-robin cycles through servers sequentially. Least-connections routes to the server with the fewest active requests — better when requests vary in duration. Consistent hashing maps clients to servers deterministically, minimising cache misses when the server pool changes. Layer 7 load balancers can also route based on URL path or headers — sending /api/* to one cluster and static assets to another.

Before

Single server — one failure kills everything

1Client → Server A (single point of failure)2 3If Server A goes down:4→ All requests fail immediately5→ No capacity to add without downtime6→ Deployment requires taking the app offline

After

Load balancer + server pool

1Client → Load Balancer → Server A (healthy)2                            ↘ Server B (healthy)3                            ↘ Server C (draining)4 5Benefits:6→ Server failure: traffic shifts in seconds7→ Add Server D with zero downtime8→ Deploy by draining one server at a time

Check your understanding

What problem does a load balancer solve for a server pool?Show answerHide answer
Answer
Distributes traffic so clients hit one address while instances can be added/removed.
When is least-connections better than round-robin?Show answerHide answer
Answer
When request durations vary widely — busy servers get fewer new connections.
What does consistent hashing help with?Show answerHide answer
Answer
Keeping client→server affinity more stable when the pool membership changes (fewer remaps).