Inter-Service Communication (Sync vs Async)

📘 Chapter 10: Microservices & Service Architecture ⏱️ 9 min read 🏗️ Lesson 042

Once you split a monolith into services, every function call becomes a network call. The choice between synchronous and asynchronous communication fundamentally shapes your system's reliability, latency, and coupling.

Synchronous Communication

Request-Response (REST, gRPC)

The caller sends a request and blocks until the response arrives. Simple mental model — like a function call over the network.

  • REST/HTTP: Text-based, universal, easy to debug with curl. Higher overhead (~1-10ms per hop).
  • gRPC: Binary (protobuf), strongly typed, multiplexed over HTTP/2. Lower latency (~0.5-3ms), ideal for internal service-to-service calls.

When to use: When you need an immediate answer to proceed (e.g., "is this user authorized?").

Asynchronous Communication

Messages & Events (Kafka, RabbitMQ, SQS)

The caller sends a message and moves on without waiting. The receiver processes it later. Decouples services in time.

  • Commands: "PlaceOrder" — directed at a specific service, expects action.
  • Events: "OrderPlaced" — broadcast to anyone interested, publisher doesn't know who listens.

When to use: When you don't need an immediate response, or when downstream processing can be deferred.

Visual Comparison

Synchronous Chain Async via Broker API Gateway Orders Payments Inventory waits... waits... waits... Total latency = sum of all hops Orders Message Broker "OrderPlaced" Payments Inventory Notifications fire & forget Consumers process independently Publisher doesn't wait or know about them
Figure 1: Sync chains accumulate latency and coupling. Async via broker decouples sender from receivers in time and identity.

The Synchronous Death Spiral

Synchronous Death Spiral Service A Service B Service C ⚠️ slow / down C is slow → B's threads fill up waiting → B times out → A's threads fill up waiting on B → A becomes unresponsive One slow service takes down the entire call chain. Solution: timeouts + circuit breakers + bulkheads
Figure 2: Without protection, one slow downstream service cascades failures upstream through blocked threads.

Resilience Patterns

PatternWhat It DoesAnalogy
TimeoutAbort if no response in N msHanging up after 30 rings
Retry (w/ backoff)Retry transient failures with exponential delayCalling back in 1s, 2s, 4s…
Circuit BreakerStop calling a failing service; fail fast insteadElectrical breaker tripping
BulkheadIsolate thread pools per dependencyShip compartments — one floods, ship floats

Comparison Table

SynchronousAsynchronous
CouplingTemporal + spatialOnly spatial (via schema)
LatencyAdditive across chainOnly initial publish
Failure handlingCascading by defaultIsolated — broker buffers
DebuggingEasier (stack trace)Harder (trace IDs, dead letters)
ConsistencyStronger (immediate)Eventual

Real-World Examples

🏢 Twitter — Async Timeline Fanout

When you tweet, it doesn't synchronously update all followers' timelines:

  • Tweet is written to your timeline (sync, fast)
  • An event "TweetCreated" is published to a fanout service
  • Fanout asynchronously pushes the tweet to each follower's cached timeline
  • For users with millions of followers (celebrities), fanout is deferred further — read-time merge instead
  • Result: tweet publish is fast (~50ms), fanout happens in background over seconds

🏢 Grab — gRPC + Kafka Hybrid

Grab (Southeast Asia's super-app) uses a mix:

  • gRPC sync for real-time queries: "Where is my driver?" "Is this promo valid?"
  • Kafka async for state changes: ride completed → trigger payment, send receipt, update driver stats
  • Key principle: sync for reads that need fresh data, async for commands/events that can be processed later

Interactive: Communication Flow Builder

Scenario: A user places an order. Choose sync or async for each step, then simulate failures.