Design a Chat System (End-to-End)
Chat is deceptively complex. The core problem isn't sending messages — it's managing millions of persistent connections, guaranteeing message ordering, and handling offline delivery gracefully.
Requirements
Functional
- 1-on-1 messaging and group chat (up to 500 members)
- Online/offline presence indicators
- Message history with pagination
- Read receipts, typing indicators
- Push notifications for offline users
Non-Functional
- Scale: 50M DAU, each sending ~40 messages/day = 2B messages/day
- Latency: <200ms end-to-end delivery for online users
- Reliability: No message loss — at-least-once delivery
- Ordering: Messages appear in correct order within a conversation
Real-Time Transport: WebSocket vs Alternatives
| Protocol | How It Works | Best For |
|---|---|---|
| WebSocket | Full-duplex persistent connection | Chat — bidirectional, low overhead ✓ |
| Long Polling | Client polls, server holds until data | Fallback when WS unavailable |
| SSE | Server-to-client only stream | Notifications — unidirectional only |
Key insight: WebSockets are stateful — each connection is pinned to a specific server. This complicates load balancing and deployment.
System Architecture
Message Storage & Ordering
Partitioning Strategy
Partition by conversation_id — all messages in a conversation live on the same partition, ensuring ordering within a conversation.
Schema: (conversation_id, message_id) → {sender, body, timestamp, status}
Message ID: Snowflake-style (timestamp + server_id + sequence) — globally unique and time-sortable.
Delivery Guarantees
- At-least-once: Server retries until client ACKs receipt
- Client-side dedup: Clients ignore messages with already-seen IDs
- Ordering: Message IDs are monotonically increasing per conversation
Group Chat Fan-Out
| Approach | How | Best For |
|---|---|---|
| Write-time fan-out | Copy message to each member's inbox | Small groups (<500) — fast reads |
| Read-time fan-out | Members pull from shared conversation | Large channels (1000+) — efficient writes |
WhatsApp uses write-time for groups (max 1024). Discord uses read-time for servers with millions of members.
WhatsApp: 100B Messages/Day with Erlang
- Each server handles 2M+ connections using Erlang's lightweight processes
- Messages stored only until delivered — no permanent server-side history (end-to-end encrypted)
- Mnesia DB for routing tables (which user → which server)
- Simple protocol: message + ACK + retry = guaranteed delivery
Discord: Massive Servers with Eventual Consistency
- Channels with millions of members — read-time fan-out is essential
- Messages stored in Cassandra, partitioned by channel + time bucket
- Eventual consistency accepted — message may appear slightly out of order across clients
- Presence uses a heartbeat + gossip protocol across gateway servers