CDNs: Caching at the Edge

📘 Chapter 5: Caching ⏱️ 9 min read 🏗️ Lesson 022

A user in Tokyo requests your website hosted in Virginia. The data travels ~11,000 km across undersea cables, through routers, adding 150-200ms of latency each way — and there's nothing you can do about the speed of light. Unless you put the data closer to the user.

What a CDN Is

A Content Delivery Network is a geographically distributed network of proxy servers that cache content close to end users. Instead of every request traveling to your origin server, most requests are served by the nearest edge node — a server that's physically close to the user.

CDN Key Concepts

  • Origin server — Your actual server with the source-of-truth content
  • Edge node (PoP) — A CDN server in a specific geographic location (Point of Presence)
  • Cache hit — Edge serves content directly without contacting origin
  • Cache miss — Edge doesn't have the content, fetches from origin, caches it
CDN: Serving Content from the Edge ORIGIN US-East EU Asia SA AU US-W 👤 5ms 👤 8ms 👤 12ms Edge Node (PoP) Origin Server → User connects to nearest edge (low latency)
Figure 1: Users connect to the nearest CDN edge node instead of the distant origin server, dramatically reducing latency.

What CDNs Cache

CDNs started with static assets but now cache far more:

Cacheable Content Types

  • Static assets: Images, CSS, JavaScript, fonts, videos — the classic CDN use case
  • HTML pages: Full rendered pages (great for content sites, blogs, e-commerce listings)
  • API responses: JSON responses that don't change per-user (product catalogs, feed data)
  • Dynamic content: Even personalized pages can be cached at the edge with techniques like Edge-Side Includes (ESI)

Push vs Pull CDNs

Two Approaches

  • Pull CDN: Edge fetches content from origin on first request (cache miss), then serves it locally for subsequent requests. Most common. Simple to set up.
  • Push CDN: You proactively upload content to the CDN before users request it. Better for large files (videos) or predictable content. More control, more operational burden.

Most CDNs are pull-based. You point DNS at the CDN, and it automatically fetches and caches from your origin.

How CDN Cache Keys Work

The CDN needs to know when two requests should receive the same cached response. The cache key determines this:

Cache Key = URL + Relevant Headers

  • Default: Full URL (scheme + host + path + query string)
  • Vary header: Adds additional dimensions (e.g., Vary: Accept-Encoding means gzip and non-gzip are cached separately)
  • Custom: You can configure CDNs to include/exclude query params, cookies, or custom headers in the cache key

Headers That Control CDN Behavior

Header Purpose Example
Cache-Control Primary caching directive max-age=3600, public
s-maxage CDN-specific max age (overrides max-age for shared caches) s-maxage=86400
Vary Which headers create separate cache entries Vary: Accept-Encoding, Accept-Language
ETag Content fingerprint for conditional requests ETag: "abc123"
stale-while-revalidate Serve stale while refreshing in background stale-while-revalidate=60

Real-World: Netflix Open Connect

🏢 Netflix's Own CDN

Netflix accounts for ~15% of global internet traffic. They built their own CDN called Open Connect:

  • Custom hardware: Netflix places its own servers (Open Connect Appliances) inside ISP networks worldwide. Each box holds 100-200TB of content.
  • Proactive pushing: During off-peak hours, Netflix pre-positions content that's predicted to be popular in each region (new releases, trending shows).
  • Result: When you press play, the video streams from a server that's often in your ISP's own building — single-digit milliseconds away.
  • Scale: Thousands of servers in 1,000+ ISP locations across 150+ countries.
  • Why build their own? At Netflix's scale, commercial CDNs were too expensive and couldn't provide the quality control Netflix needed for video streaming.

CDN for API Responses

CDNs aren't just for images anymore. Caching API responses at the edge can dramatically reduce latency:

When to Cache APIs at the Edge

  • Good candidates: Product catalog, public user profiles, search results, feed data, configuration endpoints
  • Bad candidates: User-specific data (unless using Vary: Cookie or token-based cache keys), real-time data, write endpoints
  • Technique: Set Cache-Control: public, s-maxage=60 on responses. CDN caches for 60s. Users within that window get edge-served responses.
  • Personalization trick: Separate public (cacheable) and private (user-specific) data into different endpoints. Cache the public parts aggressively.

Multi-CDN Strategies

Why Use Multiple CDNs?

  • Redundancy: If one CDN has an outage, traffic fails over to another
  • Performance: Different CDNs may be faster in different regions
  • Cost optimization: Route traffic to cheapest CDN per region
  • Implementation: DNS-based routing (e.g., NS1, Route53) directs users to the best-performing CDN based on real-time latency data

Interactive: CDN Latency Simulator

Place origin servers and CDN edges, then simulate requests from different locations. See how CDN placement affects latency:

Select a configuration and click a location to see latency.