NGINX DevOps Load Balancing System Design Networking

NGINX Gateway: Reverse Proxy, Load Balancing, and Edge Features

Q: Caching and rate limiting

A limit_req bucket on /login turns a credential-stuffing flood into 429s instead of exhausted app threads.

Q: Routing and WebSockets

Missing Upgrade/Connection headers fail a WebSocket handshake outright — the same front door needs both a REST personality and a socket personality.

Core Concept LearningJuly 16, 20265 min readUpdated July 21, 2026

Many teams still introduce NGINX as “just a web server.” In production it usually sits in front of your app: clients hit NGINX first; your Node, .NET, or Java process sees only the traffic that survives TLS, routing, rate limits, and cache checks. The Layer 4 vs Layer 7 load-balancing guide places that proxy role on the network stack.

Think of NGINX as an edge gateway. One request enters messy; NGINX applies infrastructure concerns; cleaner traffic reaches backends. This guide maps eight jobs NGINX does well — reverse proxy, load balancing, caching, rate limiting, static files, TLS, routing, and WebSockets — and when to lean on each. For transport-specific trade-offs, compare WebSocket, SSE, and long polling.

NGINX gateway: edge capability map (not a numbered execution order)

Learning path

Follow these checkpoints in order — each builds on the last.

NGINX on the public port absorbs TLS, routing, and connection fan-out so app code only sees business logic.
An upstream pool with health-aware removal keeps one dead instance from taking down the pool it belongs to.
A limit_req bucket on /login turns a credential-stuffing flood into 429s instead of exhausted app threads.
Certificates and static assets live at one edge hop, so an expiring cert or a slow asset never becomes an app-process incident.
Missing Upgrade/Connection headers fail a WebSocket handshake outright — the same front door needs both a REST personality and a socket personality.
Whatever owns your edge — NGINX or a managed ALB — settling TLS, limits, cache, and routing there is one less concern inside the app process.

NGINX as the front door

Put NGINX on the public port (80/443). Terminate TLS there, decide which upstream should handle the path, optionally serve a cached response or a static asset, and only then proxy to application servers. Your app code stays focused on business logic instead of certificate renewals and connection fan-out.

This is the same role people expect from cloud load balancers and API gateways. NGINX is popular because it is fast, config-driven, and runs the same way on a VM, a bare metal box, or as a Kubernetes Ingress controller.

One request: client → edge jobs → ordered backends

Quick reference

Clients talk to NGINX; apps talk to private upstreams.
Edge concerns: TLS, limits, cache, routing — before app code.
Same binary: static site, reverse proxy, or Ingress data plane.
Keep apps on internal ports; never expose every instance publicly.
Config lives in nginx.conf / site files under /etc/nginx/.

Remember this

NGINX on the public port absorbs TLS, routing, and connection fan-out so app code only sees business logic.

Reverse proxy and load balancing

As a reverse proxy, NGINX accepts the client connection and forwards it to one or more upstream servers (proxy_pass). Clients never need the private IPs of your app instances. You can add headers (X-Forwarded-For, X-Forwarded-Proto) so apps still see the original client and scheme.

Load balancing spreads requests across an upstream block — round-robin by default, or least connections / hash-based stickiness when sessions matter. Combine with health-aware removal so a dead instance stops receiving traffic. For deeper L4 vs L7 trade-offs, see the load-balancing guide; NGINX commonly operates as the L7 hop behind a cloud L4 balancer.

Zoom: reverse proxy hides backends · LB spreads load

Quick reference

Reverse proxy: hide backends; centralize access logs and timeouts.
Upstream pool: distribute load; retry on next peer when one fails.
Sticky sessions only when the app cannot share state (prefer shared sessions).
Tune proxy_read_timeout for slow APIs and LLM streams.
Preserve client IP with real IP / forwarded headers — document which hop trusts them.

App exposed directly (no edge)

1# Fragile: clients hit Node on :30002# No shared timeouts, TLS, or next-peer retry3# curl http://app-host:3000/api/health

upstream + proxy_pass

1upstream app_upstream {2  least_conn;3  server 10.0.0.11:3000 max_fails=3 fail_timeout=30s;4  server 10.0.0.12:3000 max_fails=3 fail_timeout=30s;5}6server {7  listen 80;8  location /api/ {9    proxy_pass http://app_upstream;10    proxy_http_version 1.1;11    proxy_set_header Host $host;12    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;13    proxy_set_header X-Forwarded-Proto $scheme;14    proxy_connect_timeout 5s;15    proxy_read_timeout 60s;16    # dead peer → try next; 502 if all fail17  }18}

Remember this

An upstream pool with health-aware removal keeps one dead instance from taking down the pool it belongs to.

Caching and rate limiting

Caching lets NGINX answer repeated GETs from memory or disk without waking the app (proxy_cache). Great for public pages, CDN-origin shielding, and expensive read APIs with clear Cache-Control semantics. Invalidate carefully — stale HTML is worse than a slow miss.

Rate limiting (limit_req, limit_conn) caps how hard clients can hit you. Put limits at the edge so abusive traffic dies before it burns app threads. Return 429 with clear retry guidance. Pair with auth-aware limits later; start with IP or API-key buckets for login and signup paths.

Protect the origin: rate limit, then Hit/Miss branches after cache check

Quick reference

Cache: speed responses; shield origin; honor vary/auth correctly.
Rate limit: control volume; protect login and expensive endpoints.
Bypass cache for authenticated or highly personalized responses.
Burst + rate: allow short spikes without opening a floodgate.
Watch hit ratio and 429 rates — both tell you if the edge is tuned.

No limit (login flood hits app)

1location /login {2  proxy_pass http://app_upstream;3  # Every guess reaches Node/DB4}

limit_req → 429

1limit_req_zone $binary_remote_addr zone=login:10m rate=5r/m;2server {3  location /login {4    limit_req zone=login burst=10 nodelay;5    limit_req_status 429;6    proxy_pass http://app_upstream;7    # curl -i /login under flood → HTTP/1.1 4298  }9  location /public/ {10    proxy_cache api_cache;11    proxy_cache_valid 200 60s;12    proxy_pass http://app_upstream;13  }14}

Remember this

A limit_req bucket on /login turns a credential-stuffing flood into 429s instead of exhausted app threads.

Static files and TLS

NGINX still shines at static files: HTML, JS, CSS, images, fonts. Serving assets directly with sendfile and long cache headers offloads work your app framework does poorly. Many SPAs use NGINX for /assets and fall through to index.html for client routes.

TLS termination at NGINX means certificates and cipher policy live in one place. Redirect HTTP→HTTPS, enable HTTP/2, and (when ready) talk HTTP to trusted internal upstreams — or keep TLS end-to-end if compliance demands it. Automate renewals (Certbot, ACME) so expiry pages never surprise you.

Serve at the edge: TLS termination + static assets

Quick reference

Static: serve assets at the edge; gzip/brotli compress text.
TLS: terminate HTTPS; manage certs once; prefer modern ciphers.
HSTS only after you are sure HTTPS works everywhere.
Separate location blocks for /static vs /api.
Internal mTLS is optional — start with private network + TLS at the edge.

HTTP only on :80

1server {2  listen 80;3  location / { proxy_pass http://app_upstream; }4}

TLS terminate + static assets

1server {2  listen 443 ssl http2;3  ssl_certificate     /etc/nginx/certs/fullchain.pem;4  ssl_certificate_key /etc/nginx/certs/privkey.pem;5  location /assets/ {6    root /var/www;7    expires 7d;8    add_header Cache-Control "public";9  }10  location / {11    proxy_pass http://app_upstream;12  }13}14# Missing/expired cert → browser TLS error before any app log line

Remember this

Certificates and static assets live at one edge hop, so an expiring cert or a slow asset never becomes an app-process incident.

Routing and WebSockets

Routing is how NGINX directs traffic intelligently: by host (api.example.com), path (/api, /admin), or headers. Rewrite paths, split canaries, or send /media to an object-storage-backed upstream while / hits the app. Clear location precedence beats a tangle of regex.

WebSockets need an upgrade-aware proxy: pass Upgrade and Connection headers, and raise read timeouts so long-lived sockets are not cut as idle HTTP. The same front door can route /ws to a socket service and /api to REST workers — one hostname, two upstream personalities.

Smart paths: host/path routing · WebSocket upgrade

Quick reference

Route by host/path before the request reaches app code.
WebSocket: enable upgrade headers; extend timeouts; consider sticky peers.
Canary: weight upstreams or split on a cookie/header.
Keep routing tables boring — readable beats clever regex.
Test upgrades through the real LB; some hops strip WebSocket headers.

WS without Upgrade headers

1location /ws/ {2  proxy_pass http://ws_upstream;3  # Handshake fails: 400 / connection closed4}

Upgrade-aware WebSocket proxy

1location /ws/ {2  proxy_pass http://ws_upstream;3  proxy_http_version 1.1;4  proxy_set_header Upgrade $http_upgrade;5  proxy_set_header Connection "upgrade";6  proxy_read_timeout 3600s;7  proxy_send_timeout 3600s;8}9location /api/ {10  proxy_pass http://app_upstream;11}

Remember this

Missing Upgrade/Connection headers fail a WebSocket handshake outright — the same front door needs both a REST personality and a socket personality.

Edge checklist

When a request hits production, walk the edge in order: TLS → rate limit → cache lookup → static file hit? → route → proxy/load balance → (optional) WebSocket upgrade. Anything you can settle in NGINX is one less concern inside the app process.

NGINX is not mandatory everywhere — managed ALB/Cloudflare/API Gateway may own the edge — but the jobs stay the same. Learning NGINX teaches the vocabulary you will reuse on every cloud. Start with reverse proxy + TLS + one upstream; add cache, limits, and routing as traffic grows.

Edge checklist order: TLS → limits → cache → route → proxy

Quick reference

Minimum viable edge: TLS + reverse proxy + health-checked upstreams.
Add rate limits on auth and write-heavy paths first.
Cache only idempotent, public, or explicitly versioned responses.
Document which hop sets X-Forwarded-* and which app trusts it.
Practice: put a tiny API behind NGINX with TLS and /static assets.

Remember this

Whatever owns your edge — NGINX or a managed ALB — settling TLS, limits, cache, and routing there is one less concern inside the app process.

Key takeaway

NGINX earns its keep as a gateway: reverse proxy and load balancing shape where traffic goes; caching and rate limiting protect performance and capacity; static files and TLS keep the edge fast and encrypted; routing and WebSockets handle smart paths and realtime sockets. Your application should see fewer raw internet concerns — not zero ops, but a clearer boundary.

Practice (25 min): Run NGINX locally (Docker is fine). Terminate TLS with a self-signed cert, proxy_pass to an app on :3000, serve /assets from disk, and add the limit_req block above on /login. Hit it with curl until you see a 429 — then you have felt the edge.