How containers find each other in a world where nothing stays at the same address.

1. The Problem

Containers are ephemeral. They restart, scale, move between hosts, and get new IPs every time. Hardcoding addresses is impossible when:

• A service scales from 2 to 10 replicas in seconds.
• A crashed container restarts on a different host with a different IP.
• Rolling updates replace containers one-by-one — old IPs disappear, new ones appear.
• Multiple environments (dev, staging, prod) have completely different topologies.

Services need to find each other dynamically. This is the service discovery problem.

2. DNS-Based Discovery

The simplest and most common approach: use DNS names instead of IP addresses.

Docker Compose

Every service name in docker-compose.yml becomes a DNS name on the shared network:

services:
  web:
    image: nginx
  api:
    image: myapp
    # Can reach the database at hostname "db"
  db:
    image: postgres

From the api container: ping db resolves to the db container's IP.

Docker Swarm

Service names resolve to a Virtual IP (VIP). The VIP is stable — it doesn't change when tasks restart. Swarm's internal load balancer routes VIP traffic to healthy tasks.

Kubernetes

A Service object creates a DNS entry: <service>.<namespace>.svc.cluster.local. The cluster DNS (CoreDNS) resolves it to the Service's ClusterIP, which load-balances to backing Pods.

How Embedded DNS Works

1. Container's /etc/resolv.conf points to the embedded DNS server (127.0.0.11 in Docker).
2. DNS server intercepts lookups for container/service names.
3. Returns current IP(s) for matching containers on the same network.
4. External names are forwarded to upstream DNS servers.

3. Client-Side vs Server-Side Load Balancing

Client-Side Load Balancing

The client receives a list of all backend IPs and decides which one to call:

• Client maintains its own connection pool and selection algorithm (round-robin, least connections, random).
• Examples: Netflix Ribbon, gRPC built-in LB, Envoy sidecar.
• Pros: No single point of failure, lower latency (no extra hop).
• Cons: Every client must implement LB logic, harder to update policies uniformly.

Server-Side Load Balancing

A single endpoint (the load balancer) accepts all traffic and distributes it:

• Client connects to one address — the LB decides where to forward.
• Examples: Nginx, HAProxy, AWS ALB/NLB, Kubernetes Service (kube-proxy).
• Pros: Clients stay simple, centralized policy control.
• Cons: Extra network hop, LB can become a bottleneck or SPOF.

4. Docker's Built-in Load Balancing

Swarm Routing Mesh

Publish a port on a Swarm service and every node in the cluster accepts traffic on that port — even nodes not running the service's tasks. The ingress network routes traffic to a healthy task.

# Every node listens on :8080, routes to web tasks
docker service create --name web --replicas 3 -p 8080:80 nginx

VIP-Based Internal Load Balancing

Each Swarm service gets a Virtual IP. Internal requests to the service name resolve to this VIP. The Linux kernel's IPVS (or iptables) distributes packets across tasks.

DNS Round-Robin in Compose

On user-defined networks, Docker returns multiple A records when a service is scaled:

# Scale to 3 replicas
docker compose up -d --scale api=3

# From another container:
dig api    # Returns 3 A records (round-robin)

⚠️ DNS round-robin has TTL caching issues — clients may cache stale IPs. VIP-based load balancing (Swarm) or a proper reverse proxy is more reliable.

5. Reverse Proxies & Ingress

Container-aware reverse proxies auto-discover backends by watching the Docker/K8s API:

Label-Based Routing (Traefik Example)

services:
  traefik:
    image: traefik:v3.0
    command: --providers.docker
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

  app:
    image: myapp
    labels:
      - "traefik.http.routers.app.rule=Host(`app.example.com`)"

  api:
    image: myapi
    labels:
      - "traefik.http.routers.api.rule=Host(`api.example.com`)"

Kubernetes Ingress

An Ingress resource defines external routing rules; an Ingress Controller (Nginx, Traefik, Envoy) implements them:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: app-service
            port:
              number: 80

6. Service Mesh

A service mesh adds a sidecar proxy (typically Envoy) next to every application container. All traffic flows through the sidecar, which handles:

• Service discovery — automatically locates backends.
• Load balancing — advanced algorithms (weighted, locality-aware).
• Retries & timeouts — automatic retry with exponential backoff.
• Circuit breaking — stop sending to unhealthy services.
• mTLS — encrypt all service-to-service traffic, zero app changes.
• Observability — distributed tracing, metrics, access logs for free.

Popular Service Meshes

💡 When is a service mesh worth it? When you have 10+ services AND need mTLS everywhere AND want uniform observability without changing app code. For 3–5 services, a reverse proxy is simpler.

7. Health-Aware Routing

Load balancers should only send traffic to healthy instances:

Health Check Types

• HTTP check — GET /health returns 200.
• TCP check — Port accepts connections.
• gRPC check — gRPC health protocol response.
• Command check — Run a binary inside the container.

Integration with Load Balancing

# Docker Compose health check
services:
  api:
    image: myapi
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 10s
      timeout: 3s
      retries: 3
      start_period: 15s

Unhealthy containers are removed from DNS and the load balancer pool until they pass again.

Graceful Connection Draining

1. Mark container as "draining" — stop sending new requests.
2. Wait for in-flight requests to complete (configurable timeout).
3. Terminate the container only after connections close.
4. Kubernetes: preStop hook + terminationGracePeriodSeconds.

Hands-On: Traefik Reverse Proxy with Docker Compose

version: "3.8"

services:
  traefik:
    image: traefik:v3.0
    command:
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--entrypoints.web.address=:80"
    ports:
      - "80:80"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

  whoami-a:
    image: traefik/whoami
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami-a.rule=Host(`app-a.localhost`)"
      - "traefik.http.routers.whoami-a.entrypoints=web"

  whoami-b:
    image: traefik/whoami
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.whoami-b.rule=Host(`app-b.localhost`)"
      - "traefik.http.routers.whoami-b.entrypoints=web"

# Start all services
docker compose up -d

# Test routing by hostname
curl -H "Host: app-a.localhost" http://localhost
curl -H "Host: app-b.localhost" http://localhost

# Scale a service and watch load balancing
docker compose up -d --scale whoami-a=3

# Repeated requests hit different instances
for i in $(seq 1 6); do
  curl -s -H "Host: app-a.localhost" http://localhost | grep Hostname
done

Industry Spotlight

Knowledge Check

Key Takeaways

• Never hardcode IPs — use DNS names or service registries for discovery.
• Docker Compose gives you free DNS discovery — service names resolve automatically on user-defined networks.
• VIP > DNS round-robin — VIPs avoid TTL caching issues and are the default in Swarm.
• Client-side LB removes the proxy hop but adds complexity to every client.
• Reverse proxies (Traefik, Nginx) auto-discover containers and route by hostname/path — ideal for 3–15 services.
• Service meshes add mTLS, observability, and advanced traffic control — but only justify the complexity at scale (10+ services).
• Always use health checks — route traffic only to containers that can serve it.
• Graceful draining prevents dropped connections during deployments.