Chapter 5 Interview: Caching

Okay so I'd think about this layer by layer. The first question is: where are the repeated, expensive reads? That's where caching gives you the biggest bang.

At the client layer — browser caching with proper Cache-Control headers. If you've got static assets or API responses that don't change frequently, let the browser hold onto them. This is free performance and reduces load on your servers entirely.

Then at the CDN level — for anything that's the same for many users. Product images, marketing pages, even API responses for popular endpoints. A CDN cache hit means the request never reaches your origin.

At the application layer — this is your Redis or Memcached. I'd cache database query results, computed values, session data. Things like user profiles that get fetched on every request, or a product catalog that changes maybe once a day but gets read thousands of times per second.

And honestly, don't forget the database's own query cache and buffer pool. Sometimes the answer isn't adding a new caching layer — it's giving your DB more memory so its internal cache is effective.

• Client: HTTP cache headers, browser storage for static assets
• CDN: edge caching for geographically distributed users
• Application: Redis/Memcached for hot database results and computed values
• Database: query cache, buffer pool sizing, materialized views

Yeah so there's that famous quote — "there are only two hard things in computer science: cache invalidation and naming things." And honestly it's true, this is genuinely hard.

The simplest approach is TTL-based expiration. You set a timeout and accept that data might be stale for up to that duration. For a lot of use cases — product recommendations, trending lists, dashboards — a 5-minute staleness window is totally fine.

For stricter needs, I'd use write-through or write-behind. Write-through updates the cache synchronously when you write to the database — consistency is strong but writes are slower. Write-behind queues the cache update asynchronously — faster writes but you risk a brief inconsistency window.

The pattern I use most is event-driven invalidation. When data changes, publish an event. Cache subscribers listen and either invalidate or update the relevant keys. This decouples the write path from cache management and works well in distributed systems.

The gotcha is always: what if invalidation fails? You need TTL as a safety net even with active invalidation. Belt and suspenders.

• TTL: simple time-based expiry — good safety net, accepts bounded staleness
• Write-through: update cache on every write — strong consistency, slower writes
• Event-driven: publish change events, subscribers invalidate — decoupled, scalable
• Safety net: always have TTL even with active invalidation

So imagine you've got a super popular cache key — like a celebrity's profile on a social platform. Millions of requests per second hitting that cached value. Now the TTL expires. Suddenly a million requests simultaneously see a cache miss and ALL of them try to rebuild the cache by hitting the database. Your DB gets crushed. That's the thundering herd.

The classic fix is a cache lock — when there's a miss, only ONE request gets a lock to rebuild the cache. Everyone else either waits briefly or gets the stale value. This is sometimes called "request coalescing" or "single-flight."

Another approach I like is stale-while-revalidate. You serve the slightly expired value to current requests while one background process refreshes the cache. Users get fast (slightly stale) responses and the DB sees exactly one query instead of a million.

You can also stagger TTLs with some jitter — instead of everything expiring at exactly 5 minutes, expire at 5 minutes plus a random 0-30 seconds. This prevents synchronized cache expiry across many keys.

• Problem: mass cache miss → simultaneous DB flood on popular keys
• Lock/single-flight: one request rebuilds, others wait or get stale data
• Stale-while-revalidate: serve expired data while refreshing in background
• TTL jitter: randomize expiry to prevent synchronized invalidation

This is a great question to show you're not just a "cache everything" person. Caching can genuinely make things worse in several scenarios.

First — low hit rates. If your data is highly unique per user or per request, most cache lookups are misses. Now you're paying the cost of a cache check PLUS the database query. You've added latency and infrastructure cost for nothing.

Second — write-heavy workloads. If data changes more often than it's read, you're constantly invalidating and rebuilding the cache. The overhead of cache maintenance exceeds the benefit of cache hits.

Third — consistency requirements. In financial systems or inventory management during flash sales, serving stale data isn't just annoying — it's a correctness bug. You oversell inventory because the cache said there were 5 items left when there are actually 0.

Fourth — the complexity tax. Every caching layer is code you maintain, bugs you debug, infrastructure you monitor. I've seen teams spend weeks debugging stale cache issues that wouldn't exist if they'd just optimized their database queries instead.

• Low hit rate: unique/random access patterns make caching overhead without benefit
• Write-heavy: constant invalidation negates read performance gains
• Consistency-critical: stale data causes correctness bugs, not just UX issues
• Complexity cost: debugging cache bugs and maintaining infrastructure

Okay so a social media feed is interesting because it's personalized, frequently updated, and read WAY more than written. Every time someone opens the app, they fetch their feed — but new posts come in maybe every few minutes.

I'd probably go with a fan-out-on-write approach for most users. When someone posts, you precompute and push that post into the cached feeds of all their followers. So when a user opens the app, their feed is already assembled in cache — it's just a Redis list read. Super fast.

But here's the catch — for users with millions of followers (celebrities), fan-out-on-write is insane. You'd update millions of cached feeds for every tweet. So you do a hybrid: fan-out-on-write for regular users, fan-out-on-read for celebrity accounts. When rendering a feed, you merge the precomputed feed with fresh queries for celebrity posts.

For the cache itself, I'd use sorted sets in Redis — score by timestamp, trim to the latest N posts. TTL of maybe 24 hours as a safety net, but actively update on new posts. And I'd cache individual post objects separately so they're shared across feeds and you're not duplicating content everywhere.

• Fan-out-on-write: precompute feeds in cache at post time — fast reads
• Hybrid approach: fan-out-on-read for high-follower accounts to avoid write amplification
• Redis sorted sets: score by time, ZRANGEBYSCORE for pagination
• Separate post cache: cache post content independently, reference by ID in feeds