↓Skip to main content
  1. References/
  2. Architecture Design Basics/
  3. Pattern Taxonomy/
  4. Scaling & Performance/

Load Balancing Patterns

·· 308 words· 2 mins
Reference Scaling Networking P1
Table of Contents

🟠 P1 — distributing traffic across service instances

Problem #

Multiple instances need traffic distributed among them.

L4 vs L7 Load Balancing #

LayerOperates onRouting decisions based onExamples
L4TCP/UDP packetsIP address, portAWS NLB, HAProxy
L7HTTP requestsURL path, headers, cookies, contentAWS ALB, NGINX

Algorithms #

AlgorithmMechanismBest for
Round-RobinRotate through instancesEqual-capacity instances
Weighted Round-RobinRotate with weights per instanceMixed-capacity instances
Least ConnectionsRoute to instance with fewest activeLong-lived connections
IP HashHash client IP to determine instanceSession affinity without cookies

Instinct #

L7 for most application traffic (path-based routing, health checks, SSL termination). L4 for high-throughput, latency-sensitive traffic (database connections, gRPC streams).

Sticky sessions are a code smell — they usually mean the service is stateful when it shouldn’t be. Fix the statefulness.

Rule of thumb to choose L4 vs L7 LB: #

Persistent connections (e.g. websockets)? Then use a L4 LB

Else use a L7 LB that gives flexibility for routing traffic to services while also minimising the connection load downstream

  • INTERVIEW: Choosing L4 vs L7 LB comes up specifically in system design interviews for real-time features (chat, live updates, gaming). Pair the LB choice with the protocol choice:
    • WebSocket-based design → L4 LB (preserves TCP session)
    • HTTP-based long-polling or SSE → L7 LB (flexible routing)
  • INSIGHT: DNS provides rudimentary client-side load balancing (rotated IP lists). This avoids single-point-of-failure problems but can’t react quickly to server health changes.
  • PITFALL: Problems with extremely high traffic need specialised hardware LBs that outperform software LBs by orders of magnitude.
  • EXP: I’ve used AWS ELB, NGINX. NGINX is the most flexible for L7; HAProxy is battle-tested for L4.

References #