Design a URL Shortener (End-to-End)

📘 Chapter 15: Putting It All Together ⏱️ 9 min read 🏗️ Lesson 062

The URL shortener is the "Hello World" of system design interviews. It looks simple — map short codes to long URLs — but it exercises ID generation, caching, database selection, and read-heavy optimization at scale.

Requirements Gathering

Functional Requirements

  • Given a long URL, generate a short unique link
  • Redirect short link to the original URL
  • Support custom aliases (e.g., short.ly/my-brand)
  • Optional: analytics (click counts, geo, referrer)
  • Links expire after configurable TTL

Non-Functional Requirements

  • Scale: 100M new URLs/day, 10B redirects/day (100:1 read/write)
  • Latency: <100ms redirect
  • Availability: 99.99% — redirects must never fail
  • Durability: links persist for 5 years minimum

Back-of-Envelope Estimation

MetricCalculationResult
Total URLs (5 years)100M × 365 × 5~180 billion
Storage180B × 500 bytes avg~90 TB
Write QPS100M / 86400~1,200/sec
Read QPS1,200 × 100~120,000/sec
Short code length62⁷ = 3.5 trillion7 chars (base62) suffices

System Architecture

URL Shortener Architecture Client Browser/App Load Balancer Write API Shorten URL + ID Generator Read API Redirect (301/302) POST /shorten GET /:code Cache Redis Cluster (hot URLs) Database NoSQL (DynamoDB / Cassandra) short_code → long_url cache miss Analytics Queue async log
Figure 1: Separate read and write paths. Reads (redirects) dominate 100:1, so caching hot URLs is critical.

Key Design Decisions

ID Generation: Base62 Encoding

Characters: a-z A-Z 0-9 = 62 chars. A 7-character code gives 62⁷ ≈ 3.5 trillion unique URLs.

ApproachProsCons
Counter + base62No collisions, short codesSingle point of failure, predictable
Hash (MD5/SHA) + truncateStateless, distributedCollision handling needed
Pre-generated key rangesFast, no coordinationWasted keys, extra service

Deep Dive: Caching & Redirects

301 vs 302 Redirects

  • 301 (Permanent): Browser caches it — fewer requests to your servers, but you lose analytics visibility
  • 302 (Temporary): Browser always hits your server — full analytics, but higher load

Decision: Use 302 if analytics matter; 301 if minimizing infra cost is priority.

Consistent Hashing for Cache

With 120K reads/sec, a single Redis node won't suffice. Distribute across a cluster using consistent hashing — adding/removing nodes only remaps ~1/N of keys, preventing cache stampedes.

How Bit.ly Works at Scale

  • Processes billions of link clicks monthly
  • Uses a distributed ID generator (similar to Twitter Snowflake)
  • Heavy caching layer — top 20% of links serve 80% of traffic
  • Separate analytics pipeline: click events → Kafka → real-time counters
  • Multi-region deployment with DNS-based routing for low-latency redirects

Interactive: Design Step by Step

Build Your URL Shortener

Make design choices at each stage and see the architecture evolve.