Horizontal vs Vertical Scaling

Vertical scaling upgrades one host: more CPU, RAM, or faster disk. No load balancer, no distributed session redesign. For a monolithic API or a database with moderate traffic, resizing the instance is often the fastest path to relieve a measured bottleneck.

The ceiling is real: cloud instance families top out, and one machine remains one failure domain. Many resizes still need a maintenance window. Cost often rises faster than linear with size. When not to stop here: you need multi-instance availability, or profiling shows you are already on a large SKU with no headroom.

Horizontal scaling adds instances behind a load balancer. Autoscaling groups add capacity on CPU, request rate, or custom metrics, and remove it when load falls. That only works if the app is stateless on the node: sessions in Redis (or signed cookies), uploads in object storage, no reliance on local disk for durable data.

Request trace: Client → LB → instance A handles GET /orders/42 → Redis session lookup → Postgres → response. Failure path: instance A dies mid-request → LB stops routing new traffic there (health check fail) → retry or next request lands on instance B with the same session store. If sessions lived only in A’s memory, B cannot continue the user’s flow.

Production systems mix strategies by tier. App servers usually go horizontal early. Databases often go vertical first, then read replicas for read scale, then partitioning when writes demand it. Caches and CDNs are horizontal by design.

Evidence-based progression: (1) load-test and find the bottleneck, (2) vertical if one resource is cheap to buy, (3) horizontal app tier + shared session/cache, (4) DB replicas, (5) shard or split only with a measured write ceiling. Skipping steps without metrics is how teams invent distributed complexity before they need it.

Trigger. Instance A, one of four behind the load balancer, crashes mid-request — an OOM kill, a bad deploy, a hardware fault. Symptom. The in-flight request to A fails; the next request from the same user, routed to instance B, either continues seamlessly or forces a re-login, depending entirely on where that user's session actually lived.

Root mechanism. The load balancer's health check needs one or more failed probes before it stops routing new traffic to A — a brief window where new connections can still land on a dead instance, the same detection-delay trade-off covered in Load Balancing: Layer 4 vs Layer 7. Separately, whether B can pick up the user's session depends on whether that session was ever node-local. Recovery: once the health check fails A, all new traffic routes to B, C, and D only; if sessions live in shared Redis, B reads the same session state A would have and the user never notices. Verification: kill one instance mid-session in a local two-process test and confirm the next request still authenticates on the survivor — the practice exercise below is exactly this test. Prevention: treat any node-local state (in-memory sessions, local file uploads, in-process caches assumed durable) as a horizontal-scaling blocker to fix before adding instances, not an acceptable trade-off to discover during an actual instance failure.