Day4

What is Horizontal Scaling?

Horizontal scaling, often referred to as "scaling out," means increasing the capacity of your system by adding more machines (servers) into your infrastructure pool, rather than making a single machine stronger. Instead of trading your delivery truck for a massive semi-truck (vertical scaling), you hire an entire fleet of standard delivery trucks to work together at the same time.

In system design, horizontal scaling is the foundation for building highly available, fault-tolerant, and massive global-scale applications.

Step 1: Implementing the Fleet Infrastructure

To transition a system from a single server to a horizontally scaled architecture, you must introduce components that allow multiple machines to coordinate seamlessly:

• Load Balancers (The Traffic Cop): You place a load balancer (like Nginx, AWS ALB, or HAProxy) in front of your servers. It intercepts all incoming user requests and distributes them evenly across your fleet of application servers so no single machine gets overwhelmed.

• Stateless Application Servers: For horizontal scaling to work smoothly, application servers should ideally be "stateless." This means they do not save user session data (like login info or shopping carts) on their local hard drives. If Server A crashes, the load balancer can send the user to Server B without the user getting logged out.

• Centralized Session Storage: To keep servers stateless, all user sessions are offloaded to a shared, ultra-fast, in-memory data store like Redis or Memcached that all application servers can access.

Step 2: Horizontally Scaling the Database Layer

While scaling the application servers out is relatively easy, scaling a relational database horizontally is the hardest part of system design. Architects use specific strategies to do this:

• Read Replicas: You separate read and write traffic. One "Primary" database handles all data changes (INSERT, UPDATE), while multiple "Replica" databases handle data viewing (SELECT).

• Database Sharding (Horizontal Partitioning): When data grows too large for one database disk, you break tables into smaller pieces across completely different machines based on a key. For example, users with IDs 1 to 1,000,000 are stored on Database Server 1, and IDs 1,000,001 to 2,000,000 are stored on Database Server 2.

• NoSQL Databases: Often, architects transition non-relational data to databases designed natively for horizontal scaling (like Cassandra, MongoDB, or DynamoDB), which inherently support distributed data across clusters.

Advantages of Horizontal Scaling

• Infinite Scaling Ceiling: There is no physical hardware limit. If your traffic doubles next month, you don't need to engineer a futuristic processor; you simply boot up more standard, inexpensive servers.

• High Availability & Fault Tolerance: If you have 10 servers and one breaks down or crashes, the load balancer automatically stops sending traffic to it. Your application stays online, and your users experience zero downtime.

• Cost Inefficiency Fix (Linear Costs): Standard commodity servers are cheap. Buying ten basic servers is often significantly cheaper than trying to build or buy one ultra-powerful, specialized enterprise server.

The Complexities and Disadvantages

Horizontal scaling is powerful, but it drastically complicates your software code:

• Data Consistency Issues (Eventual Consistency): If a user updates their profile picture on the Primary database, it takes a few milliseconds to copy that picture to the Read Replicas. If they refresh their page instantly, they might temporarily see their old picture until the data syncs.

• Network Latency & Partitions: Because machines must talk to each other over a network to process data, network lag is introduced. Additionally, you must design your system to handle situations where a network switch fails and splits your server fleet in half (Network Partitioning).

• Architectural Complexity: Your engineering team must write code capable of handling distributed transactions, complex service-to-service communication, and centralized logging.

Summary Comparison

Traditional relational databases (like MySQL or PostgreSQL) are structurally rigid. They require strict tables, schemas, and complex relationship joins, making them naturally suited for a single machine (vertical scaling).

NoSQL databases (like MongoDB, Cassandra, DynamoDB), on the other hand, were explicitly designed from day one to scale horizontally across hundreds or thousands of commodity servers. They trade away complex relational joins and absolute immediate consistency to gain massive, nearly infinite scale.

Here is exactly how NoSQL databases achieve effortless horizontal scaling:

1. Data Sharding Built into the Engine

In a relational database, horizontal partitioning (sharding) is notoriously complex and usually requires engineers to write custom routing logic. In NoSQL databases, sharding is native and fully automated.

• Partition Key / Shard Key: When you save data into a NoSQL database, you provide a key (like user_id). The database applies a mathematical algorithm (hashing) to that key to generate a number.

• Automatic Distribution: The database automatically assigns ranges of these numbers to different machines (nodes) in the cluster. If you add 5 new servers to the cluster, the database automatically shifts data chunks over to those new servers under the hood, requiring zero manual reshuffling by developers.

2. Non-Relational Data Models (No Joins)

The biggest blocker to horizontal scaling in a traditional database is a JOIN operation. If you need to join an Orders table on Server A with a Customers table on Server B, the servers must send massive amounts of data to each other over a slow network, causing severe performance bottlenecks.

NoSQL solves this by eliminating joins entirely:

• Denormalization: Instead of keeping data in separate linked tables, NoSQL database structures (like Documents or Key-Value pairs) allow you to bundle everything together. A single user document might contain the user's profile, their last 10 orders, and their addresses all in one record.

• Single-Node Operations: Because all the required data is self-contained in a single document, the database can fetch it from one exact server in a single fast request, without ever needing to coordinate network communication with other machines.

3. Peer-to-Peer Cluster Architecture (No Single Master)

Many NoSQL databases (most famously Apache Cassandra) utilize a decentralized masterless or "peer-to-peer" architecture.

• Every Server is Equal: There is no single "Primary" or "Master" database that handles all updates. Every single server in the cluster can accept both read and write requests simultaneously.

• Linear Scaling: If 5 servers can handle 50,000 requests per second, adding another 5 servers will cleanly double your capacity to 100,000 requests per second. If any server dies, the remaining nodes absorb the traffic with zero system downtime.

The Trade-off: CAP Theorem & Eventual Consistency

While horizontal scaling makes NoSQL incredibly fast and massive, you pay for it with data consistency. According to the CAP Theorem, a distributed system can only guarantee two out of three attributes: Consistency, Availability, and Partition Tolerance.

NoSQL choices typically prioritize Availability and Partition Tolerance. This results in Eventual Consistency:

• If a user updates their profile description on Node 1, it takes a fraction of a second to replicate that change to Node 2 and Node 3.

• If another user requests that profile from Node 3 immediately afterward, they might temporarily see the old description. Within milliseconds, all nodes sync up, and the data becomes fully consistent across the cluster.

Summary: When to Use NoSQL for Scaling