gRPC & Protocol Buffers

📘 Chapter 11: API Design ⏱️ 8 min read 🏗️ Lesson 047

When services talk to each other thousands of times per second, JSON over HTTP/1.1 becomes a bottleneck. Every message must be serialized to text, parsed back, and sent over a connection that can only handle one request at a time. gRPC eliminates these costs: binary serialization (Protocol Buffers), HTTP/2 multiplexing, and generated client/server code in any language. It's the default for high-performance internal communication.

What Is gRPC?

High-Performance RPC Framework

  • gRPC = Google Remote Procedure Call — open-sourced in 2015
  • Uses HTTP/2 for transport (multiplexing, header compression, bidirectional streaming)
  • Uses Protocol Buffers (protobuf) for serialization (binary, strongly typed, schema-driven)
  • Code generation: Define your API in a .proto file → generate client stubs and server skeletons in 10+ languages
  • Built-in support for deadlines, cancellation, load balancing, and health checks
gRPC Architecture: Code Generation Flow service UserService { rpc GetUser(...) returns (...) } user.proto protoc compiler Generated Client Stub client = UserServiceClient(channel) user = client.GetUser(req) Python / Go / Java / ... Generated Server Skeleton class UserService(pb2_grpc...): def GetUser(self, req, ctx): Implement business logic here ⟵ HTTP/2 + Protobuf binary frames ⟶
Figure 1: Define the API once in a .proto file; the compiler generates type-safe client and server code in any language.

Protocol Buffers: Binary Serialization

Smaller, Faster Than JSON

// user.proto
syntax = "proto3";

message User {
  int32 id = 1;
  string name = 2;
  string email = 3;
  repeated Order orders = 4;
}

message Order {
  int32 id = 1;
  float total = 2;
  string status = 3;
}

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
  rpc ListUsers (ListUsersRequest) returns (stream User);
  rpc CreateUser (CreateUserRequest) returns (User);
}

Each field has a number (not a name) on the wire — that's what makes it so compact. Field names exist only in the schema for code generation.

Size Comparison

The same user object:

JSON{"id":42,"name":"Alice","email":"alice@ex.com"}47 bytes
Protobuf08 2A 12 05 41 6C 69 63 65 1A 0C...24 bytes

~50% smaller for simple messages, up to 10x smaller for large nested objects with repeated fields. Plus binary parsing is orders of magnitude faster than JSON string parsing.

Four Streaming Types

gRPC Streaming Patterns Unary Client Server 1 request → 1 response Server Streaming Client Server 1 request → stream of responses Client Streaming Client Server Stream of requests → 1 response Bidirectional Client Server Both stream independently Standard request/reply GetUser, CreateOrder Real-time feeds Stock prices, log tailing Batch uploads File upload, sensor data Chat, multiplayer Real-time collaboration
Figure 2: gRPC supports four communication patterns — from simple request/reply to full-duplex bidirectional streaming.

Advantages Over REST

  • ~10x smaller payloads: Binary protobuf vs text JSON
  • Strongly typed: Generated code catches errors at compile time, not runtime
  • HTTP/2 multiplexing: Multiple RPC calls over a single TCP connection, no head-of-line blocking
  • Bidirectional streaming: Both sides can send data concurrently
  • Deadlines/timeouts: Built into the protocol — propagate through call chains
  • Language agnostic: One .proto generates Go, Java, Python, C++, Rust, etc.

When to Use (and When Not)

✅ Use gRPC When:

  • Internal service-to-service communication (microservices)
  • Low-latency, high-throughput requirements
  • Polyglot environments (teams using different languages)
  • Streaming data (real-time updates, log streaming)

❌ Don't Use gRPC When:

  • Browser clients: Limited native support (need gRPC-Web proxy)
  • Human-readable debugging: Binary messages aren't curl-friendly
  • Simple CRUD APIs: Overhead of proto definitions isn't worth it
  • Third-party/public APIs: REST or GraphQL are more accessible

gRPC-Web & Connect

Bridging to Browsers

gRPC-Web: An official spec that enables browser clients to call gRPC services through an Envoy proxy that translates HTTP/1.1 to HTTP/2. Limited to unary and server-streaming (no client or bidi streaming).

Connect (Buf): A newer protocol that's gRPC-compatible but also works as plain HTTP — no proxy needed. Supports JSON encoding for debugging while keeping binary for production. Growing rapidly in adoption.

🌍 Google's Internal Usage

Google processes 10 billion gRPC calls per second across their internal infrastructure. Every Google service communicates via gRPC — from Search to YouTube to Cloud. The framework was extracted from their internal "Stubby" RPC system that's been running since 2001.

🌍 Netflix's Hybrid Approach

Netflix uses gRPC for all backend service-to-service communication (high performance, type safety between 1000+ microservices) while exposing REST and GraphQL externally for their client apps and third-party integrations. The pattern: gRPC inside, REST/GraphQL at the edge.

Interactive: Proto Definition & Size Comparison

🎮 Define a Service & Compare Sizes

See how a protobuf message compares to JSON in size. Add fields to the message and watch the size difference grow.

Message fields:
Size comparison: