Distributed Transactions in System Architecture

32. Distributed Transactions: Ensuring Data Integrity In a microservices architecture, a single business process (like "Book a Flight") often spans multiple services (Flight Service, Payment Service, Booking Service). Unlike a monolith, you cannot wrap these in a single ACID database transaction. If one step fails, you must ensure the entire system remains consistent. This is the challenge of Distributed Transactions . 1. The Saga Pattern The Saga Pattern is the most common way to manage distributed transactions. A Saga is a sequence of local transactions. Each local transaction updates the database and publishes an event or message to trigger the next local transaction in the saga. Success Path: The saga executes Service A $\rightarrow$ Service B $\rightarrow$ Service C. Failure Path (Compensating Transactions): If Service C fails, the saga must execute Compensating Transactions (undo actions) for Service B and Service A (e.g., "Refund Payment," "Cancel Seat Reservation") to return the system to its original state. 2. Distributed Transactions (2PC) The Two-Phase Commit (2PC) is a traditional, strict protocol for atomic transactions across distributed databases. Phase 1 (Prepare): A "coordinator" asks all participating databases if they are ready to commit. They lock the necessary records and vote "Yes" or "No." Phase 2 (Commit): If all participants vote "Yes," the coordinator sends a "Commit" command. If any vote "No," it sends "Rollback." The Reality: While highly consistent, 2PC is a blocking protocol . If the coordinator fails during the process, resources remain locked, causing massive system bottlenecks. It is rarely used in modern, cloud-native microservices for this reason. 3. Idempotency In distributed systems, network errors are common. If you send a "Charge Customer" request and get a timeout, you don't know if the charge succeeded. If you retry and it did succeed, you might charge them twice. Idempotency is the property that ensures an operation can be applied multiple times without changing the result beyond the initial application. The Mechanism: Use Idempotency Keys . The client sends a unique request_id with every POST request. The Server Logic: The server checks if it has already processed a request with that request_id . If yes, it returns the cached original response without executing the business logic again. If no, it processes the request and stores the result. 4. Eventual Consistency In distributed systems, achieving "Strong Consistency" (where all nodes see the same data at the exact same millisecond) is often impossible due to the laws of physics and network latency (the CAP Theorem). We instead opt for Eventual Consistency . The Concept: You accept that for a short period, different services or replicas might have different data. However, you guarantee that if no new updates are made, eventually all nodes will converge to the same value. Production Reality: Most business logic is built to handle this. For example, when you update your profile photo, it is okay if it takes 2 seconds to appear on all global servers; the system remains functional and responsive while the data "catches up." Distributed Transaction Reference Matrix Pattern Consistency Level Complexity Primary Benefit Saga Eventual High Handles long-running business processes. 2PC Strong Very High Absolute consistency (but high latency). Idempotency N/A (Required) Moderate Prevents data duplication on retries. Eventual Cons. Eventual Low Maximum availability and performance.