Caching Basics in System Architecture

Caching Basics: Performance at the Speed of RAM Caching is the practice of storing a copy of frequently accessed data in a high-speed storage layer (usually RAM) to serve future requests instantly, avoiding the latency of re-computing the result or fetching it from a slow, primary data source like a disk-based database. 1. Why Caching? In distributed architecture, database queries and network calls are "expensive" operations that consume significant CPU, memory, and time. Caching provides three core benefits: Latency Reduction: Serving a request from an in-memory cache (e.g., Redis) takes microseconds, whereas a database query might take milliseconds. Database Offloading: By serving 80–90% of requests from the cache, you prevent your database from buckling under high traffic spikes. Cost Efficiency: Offloading your primary database means you can run smaller, cheaper database instances, as the "hot" traffic is being handled by lighter caching layers. 2. Client-Side Caching Client-side caching happens entirely inside the user's browser or mobile application. The goal is to avoid network round-trips entirely by keeping data local to the user. Browser Cache (HTTP Caching): Browsers automatically store static assets (images, CSS, JS) based on Cache-Control headers sent by the server. Example: A server might send Cache-Control: max-age=31536000 , telling the browser to cache that image for one year. Local/Session Storage: Modern web apps store specific application states (like user preferences or shopping cart data) directly in the browser’s localStorage or IndexedDB , allowing the app to load instantly even if the user is offline. 3. Server-Side Caching Server-side caching is where the most significant performance gains are found in backend architecture. Database Caching: Placing an in-memory data store like Redis or Memcached between your application server and your primary SQL/NoSQL database. You store the result of complex SQL queries or expensive computation logic as key-value pairs. CDN (Edge) Caching: As we discussed in our earlier networking module, CDNs cache public assets (like HTML, images, and videos) at "Edge" locations physically closer to the user to reduce the geographic distance the data must travel. Application Caching: Storing the results of a heavy function call (like a recommendation engine algorithm result) directly in the memory of the application server itself to prevent redundant CPU cycles. 4. Cache Invalidation: The Hardest Problem The "Two Hard Problems in Computer Science" are cache invalidation, naming things, and off-by-one errors. If you cache data, you eventually need to update it when the source data changes. If you don't, you serve "stale" (incorrect) data. Common Invalidation Strategies TTL (Time to Live): The simplest strategy. You set a fixed lifespan for a cache entry (e.g., 60 seconds). Once it expires, the cache automatically deletes it. The next request is forced to fetch the latest data from the database. Write-Through Caching: The application updates the primary database and the cache simultaneously. The cache is always consistent, but every write operation now takes slightly longer because it must update two systems. Cache-Aside (Lazy Loading): The application checks the cache. If it’s a "miss," it fetches data from the database and updates the cache. This is the most common pattern for web applications. Event-Driven Invalidation: When a record is updated in the database, the backend triggers an event (e.g., via a Message Queue) that instructs the caching layer to delete the specific key, ensuring the next user sees fresh data. Caching Reference Matrix Cache Type Location Primary Goal Invalidation Strategy Browser Cache Client Device Eliminate network request Cache-Control headers CDN Cache Edge Network Eliminate geographic latency Purge commands / TTL Database Cache Backend (Redis) Eliminate database query time TTL, Write-Through, or Event-Driven App Memory Server CPU Eliminate function runtime TTL