Load Balancing Algorithms in System Architecture

13. Load Balancing Routing Algorithms A load balancer is only as good as the math it uses to distribute traffic. Once your Layer 4 or Layer 7 load balancer verifies that your backend instances are completely healthy, it must apply a specific Routing Algorithm to determine exactly which server container will handle each incoming request. Choosing the wrong algorithm can lead to an uneven traffic distribution—causing a single backend instance to choke under a wave of heavy processing requests while sibling servers sit completely underutilized. 1. The Sequential Cycle: Round Robin Round Robin is the simplest and most common load-balancing algorithm. It distributes incoming network requests across your backend server pool in a strict, sequential, circular order. ROUND ROBIN ROUTING LOOP INBOUND REQUESTS LOAD BALANCER BACKEND INSTANCES ┌────────────────┐ ┌───────────────┐ ┌──────────────────┐ │ Request #1 ───┼─────────►│ │─────────►│ Backend Node A │ │ Request #2 ───┼─────────►│ Sequential │─────────►│ Backend Node B │ │ Request #3 ───┼─────────►│ Rotator Dial │─────────►│ Backend Node C │ │ Request #4 ───┼─────────►│ │─────────►│ Backend Node A │ └────────────────┘ └───────────────┘ └──────────────────┘ Production Mechanics If you have three identical backend servers (Node A, Node B, Node C): Request 1 is routed to Node A. Request 2 is routed to Node B. Request 3 is routed to Node C. Request 4 loops back around straight to Node A. Weighted Round Robin (The Heterogeneous Upgrade) Standard Round Robin assumes all backend servers have the exact same hardware capacity. In production cloud setups, you might mix instance sizes (e.g., Node A has 4 CPU cores, while Node B and C have only 2 cores). Weighted Round Robin solves this by allowing an engineer to assign a static integer weight to each server. A server with a weight of 2 will receive two consecutive requests before the rotator dial moves to the next server in the list. Core Trade-offs Pros: Exceptionally low CPU overhead. The load balancer doesn't need to track any real-time server metrics or state updates; it simply moves down a list. Cons: Completely blind to server load. If Request 1 triggers a massive, slow database report calculation on Node A, and Requests 2, 3, and 4 are tiny, fast API checks, Round Robin will still send Request 4 straight to Node A, piling traffic onto an already struggling server. 2. Dynamic Load Awareness: Least Connections The Least Connections algorithm is a dynamic routing strategy that sends incoming traffic to whichever server currently has the lowest number of active, open concurrent sessions at that exact millisecond. Production Mechanics The load balancer maintains an active, real-time counter of every open TCP socket link or active HTTP request currently processing on each backend instance. When a new user request arrives at the gateway, the load balancer evaluates the counters and routes the request to the most underutilized node. Weighted Least Connections Similar to Round Robin, this upgrade factors in varying hardware sizes. The load balancer divides each server's active connection count by its assigned weight capacity, ensuring that higher-capacity machines pull a larger share of concurrent workloads. Core Trade-offs Pros: Highly efficient for environments where user requests vary significantly in processing time and complexity (e.g., systems mixing short API reads with long, heavy data processing tasks). It naturally balances out traffic spikes. Cons: Higher processing overhead on the load balancer, which must continuously update its connection tracking table for every single entry and exit event. 3. Geographic & State Pinning: IP Hash The IP Hash algorithm uses mathematical hashing functions to map a client’s unique IP address directly to a specific backend server instance. IP HASH ROUTING MECHANICS CLIENT SOURCE IP HASH ALGORITHM TARGET DESTINATION ┌──────────────────┐ ┌────────────────────┐ ┌──────────────────┐ │ 192.168.1.50 │ ────────► │ Hash(IP) % NodeCount│ ──────► │ Backend Node B │ └──────────────────┘ └────────────────────┘ └──────────────────┘ (Always maps to B) Production Mechanics When a packet arrives, the load balancer extracts the client's source IP address string (e.g., 157.45.12.110 ), runs it through a deterministic hashing algorithm, and performs a modulo operation based on the total number of active backend servers: Server Index = Hash ( Client IP ) ( mod Total Healthy Nodes ) Because the math is entirely deterministic, a user connecting from that specific IP address will always land on the exact same backend server instance across every click they make, provided the server pool remains unchanged. Core Trade-offs Pros: Establishes a stateless relationship between a client and a specific server without forcing the load balancer to store persistent session state tables in its memory. Cons: Can lead to uneven traffic distribution. If a massive office building or university campus routes thousands of internal users through a single shared corporate NAT Gateway public IP, the load balancer will hash that single IP and dump every single one of those users onto the exact same backend node, overloading it completely. 4. Application Session Persistence: Sticky Sessions Sticky Sessions (also known as Session Affinity ) is an application-layer routing mechanism that binds a user's browser session directly to a specific backend server container for the entire duration of their visit. Production Mechanics Unlike IP Hash (which relies blindly on low-level network IP headers), Sticky Sessions operate at Layer 7 using HTTP cookies: A user logs into the application interface. The Layer 7 load balancer intercepts the initial response and injects a unique tracking cookie (e.g., SERVERID=node_C ) straight into the user's browser response headers. For every subsequent click, the browser automatically attaches that cookie to its outbound HTTP request headers. The load balancer reads the cookie value and routes the traffic directly to Node C. When is this Pattern Required? Sticky Sessions are a mandatory fallback when working with Stateful Applications . If your backend server saves a user's active login state, shopping cart array, or temporary working files directly inside its own local RAM cache rather than an external centralized database, that user must hit that exact same machine on every single click. If they get routed to a sibling server, their session data will be missing, forcing them to log in again. The Production Risks Breaks Horizontal Elasticity: If Node C experiences a hardware failure or is shut down by an auto-scaling group, all users pinned to Node C lose their sessions instantly. Creates Traffic Imbalances: If a few highly active users get grouped onto the same backend node, that server will experience high resource utilization while sibling servers sit idle. Load Balancing Algorithms Reference Matrix Algorithmic Strategy Operational Execution Complexity Primary Traffic Optimization Focus Critical Production Downside / Failure Mode Round Robin Minimal. Requires zero state or metric tracking loops. Perfectly distributed request counts across identical hardware setups. Completely blind to active processing load and hardware performance differences. Least Connections Moderate. Requires continuous tracking of active network connections. Distributing unpredictable, long-running requests efficiently. Higher processing overhead on the load balancer's central CPU. IP Hash Low. Relies on a fast, deterministic mathematical calculation. Providing predictable client-to-server mapping without storing session states. Prone to clustering imbalances when dealing with large corporate NAT gateways. Sticky Sessions High. Requires Layer 7 cookie inspection and parsing. Maintaining user persistence on legacy, stateful application setups. Destroys pure horizontal elasticity. Prevents clean auto-scaling downscales. 🧠 System Design Checkpoint We have successfully broken down the routing algorithms that manage high-volume horizontal traffic. Now, we can move from raw load balancing into advanced infrastructure orchestration. Would you like to explore API Gateway Design Patterns (handling rate limiting, authentication, and routing), or investigate reverse-proxy reverse entry architectures next?