Service Discovery & Service Mesh

📘 Chapter 10: Microservices & Service Architecture ⏱️ 8 min read 🏗️ Lesson 043

In a microservices world, services come and go — scaling up, scaling down, restarting after crashes. How does Service A find Service B? Hardcoded IPs won't work. You need service discovery, and at scale, a service mesh to handle cross-cutting concerns.

The Discovery Problem

In production, a service might have 20 instances across 5 machines. Instances are ephemeral — containers start, die, and reschedule. The caller needs to know: which IP:port is alive right now?

Client-Side vs Server-Side Discovery

Client-Side Discovery Server-Side Discovery Service A Registry (Consul, Eureka) 1. query B inst.1 B inst.2 B inst.3 2. direct call Client picks instance & load-balances Service A Load Balancer (DNS / LB) Registry (K8s DNS, Consul) B inst.1 B inst.2 B inst.3 LB/router picks instance — client is unaware
Figure 1: Client-side discovery — client queries registry and picks an instance. Server-side — a router/LB handles it transparently.
Client-SideServer-Side
ExampleNetflix Eureka + RibbonKubernetes DNS + kube-proxy
ProsNo extra hop, flexible LBLanguage-agnostic, simple client
ConsClient library per languageExtra network hop

Service Mesh Architecture

A service mesh extracts networking concerns (discovery, load balancing, retries, mTLS, observability) into a sidecar proxy deployed alongside each service instance. The application code knows nothing about it.

Service Mesh — Sidecar Pattern Control Plane (Istio/Linkerd) Service A Envoy Proxy Pod/Container Service B Envoy Proxy Pod/Container mTLS, retries, tracing, LB Control plane pushes config (routing rules, policies) to all sidecars
Figure 2: Service mesh — sidecar proxies handle all network concerns. Application code just calls localhost.

Tools Landscape

ToolCategoryKey Feature
Kubernetes DNSServer-side discoveryBuilt-in, zero config for K8s services
ConsulService registry + meshMulti-DC, health checks, KV store
IstioFull service meshTraffic shaping, mTLS, canary deploys
LinkerdLightweight meshSimpler than Istio, low resource overhead
EnvoyProxy (data plane)L7 proxy, used by Istio/Consul/AWS App Mesh

Real-World Examples

🏢 Lyft — Created Envoy for Service Mesh

  • Lyft had 100+ services with inconsistent retry/timeout behavior per language (Python, Go, Java)
  • Built Envoy as a universal sidecar proxy — one place for all network policies
  • Every service talks to localhost:port; Envoy handles discovery, mTLS, retries, circuit breaking
  • Result: consistent networking behavior across all languages, centralized observability

🏢 Kubernetes — Built-in Service Discovery

  • Every K8s Service gets a DNS name: orders.default.svc.cluster.local
  • kube-proxy maintains iptables/IPVS rules to route to healthy pods
  • For most teams, K8s DNS is enough — no extra tooling needed
  • Graduate to Istio/Linkerd when you need: mTLS between services, canary routing, per-route retries

Interactive: Service Registry Simulation

Watch services register and deregister. Click buttons to simulate scaling events.