Stateless Services & Session Management

📘 Chapter 7: Scalability Patterns ⏱️ 9 min read 🏗️ Lesson 029

In Lesson 011, we introduced stateless vs stateful services. Now we go deeper: where does state hide in supposedly stateless services, and how do you manage the state that must exist (like user sessions) without sacrificing scalability?

Where State Hides

Your "Stateless" Service Probably Isn't

State sneaks in through many back doors:

  • In-memory caches: A local HashMap of recently-fetched user profiles. If the next request hits a different server, cache miss.
  • File uploads: Saving /tmp/upload_123.jpg locally. A subsequent request to process it lands on a different server — file not found.
  • WebSocket connections: Client connects to Server A. If Server A dies, the connection and all subscription state is lost.
  • Scheduled jobs: A cron that runs on one instance. If that instance scales down, the job disappears.
  • Local disk logs: Writing to ./app.log. Logs scatter across N servers — impossible to correlate.
  • Rate limit counters: Tracking "this IP sent 50 requests" in local memory. Each server counts separately — limits are N× too generous.

Session Management Approaches

Session Storage Approaches Compared Server-Side (Redis/Memcached) Server A Server B Redis Cluster ✓ Revocable ✓ Small cookie (just ID) ✗ Extra hop (network latency) Client-Side (JWT) Server A Server B JWT in Cookie/Header header.payload.signature ✓ No server lookup needed ✓ Scales infinitely ✗ Can't revoke until expiry Encrypted Cookie Server A Server B AES-encrypted cookie All state in the cookie ✓ No external storage ✓ Simple implementation ✗ 4KB cookie size limit Quick Comparison Feature Redis JWT Enc. Cookie Instant revocation ✓ Yes ✗ No* ✗ No No external infra ✗ Needs Redis ✓ Yes ✓ Yes Large session data ✓ Unlimited ~ Limited (header size) ✗ 4KB max Read latency ~1ms (network hop) 0ms (local verify) 0ms (local decrypt) * JWT can use short expiry + refresh tokens, or maintain a blocklist (which requires server state)
Figure 1: Three session storage approaches — each trades off differently between scalability, revocability, and simplicity.

JWT Deep Dive

Structure: header.payload.signature

A JWT is three Base64-encoded parts separated by dots:

  • Header: {"alg": "HS256", "typ": "JWT"} — which signing algorithm
  • Payload: {"sub": "user_123", "role": "admin", "exp": 1700000000} — claims (data)
  • Signature: HMAC-SHA256(base64(header) + "." + base64(payload), secret) — tamper-proof seal

Advantages

  • No server lookup — verify locally with the secret key
  • Contains all user info needed for authorization (role, permissions)
  • Works across services — any service with the key can verify

Disadvantages

  • Can't revoke: Once issued, valid until expiry. A compromised token is dangerous.
  • Size: A JWT with claims is 800+ bytes vs a 32-byte session ID
  • Stale data: If user's role changes, the JWT still has the old role until re-issued

Mitigation: Use short-lived access tokens (15 min) + long-lived refresh tokens (stored server-side, revocable).

Redis for Sessions

Why Redis Is the Most Popular Session Store

  • Speed: Sub-millisecond reads — barely slower than local memory
  • TTL: SET session:abc123 data EX 3600 — sessions auto-expire after 1 hour
  • Atomic operations: INCR, SETNX for safe concurrent access
  • Cluster mode: Redis Cluster shards across nodes for HA and horizontal scale
  • Pub/Sub: Notify other services when a session is invalidated

Pattern: Client sends session cookie → app server looks up session:{id} in Redis → gets user data → processes request. Any server can handle any request.

File Uploads in a Stateless World

Never Store Files Locally

If a user uploads to Server A and their next request hits Server B, the file isn't there. Solutions:

  • Stream directly to object storage: Use presigned URLs (S3, GCS) — the client uploads straight to the storage service, never touching your app servers.
  • Proxy through app server: Receive the file and immediately write to S3. Never write to local disk.
  • Shared filesystem (NFS/EFS): All servers mount the same storage. Works but adds a single point of failure and latency.

Best practice: Presigned URLs. Your server generates a URL, the client uploads directly. Zero load on your app servers.

🏢 Real-World: How Spotify Manages Sessions

Spotify runs thousands of stateless microservices. Here's how they handle session state:

  • Access tokens: Short-lived OAuth2 tokens (1 hour). Every API call includes the token. Services verify locally via a shared public key (asymmetric JWT).
  • Refresh tokens: Stored server-side in a distributed session store. When access tokens expire, the client uses the refresh token to get a new one.
  • Revocation: When a user changes their password, all refresh tokens for that user are invalidated. Access tokens naturally expire within an hour.
  • Cross-device: Each device has its own session. You can see all active sessions in your account settings and revoke any one.
  • Playback state: "Now playing" state is stored in a dedicated service (not in the session) — this lets you seamlessly hand off playback between devices.

Interactive: Design a Session Management System

Choose your session approach and see the trade-off analysis for your use case: