grpc inside a kubernetes cluster
inside a kubernetes cluster, the protocol overhead becomes visible because the network round trip is already short.
for our APIs, clients can keep using REST and WSS at the edge. between the proxy and the applications, we can choose a protocol designed for service-to-service traffic.
gRPC is built on HTTP/2. services and messages are defined in a proto schema, client and server code is generated, and protobuf is generally used for serialization.
a unary call still looks like one request followed by one response. server streaming sends multiple responses, client streaming sends multiple requests, and bidirectional streaming keeps both directions open independently.
on the wire, every gRPC call is an HTTP/2 stream. each message adds a 1-byte compression flag and a 4-byte big-endian length before the protobuf payload. multiple calls can share the same TCP connection without waiting for the previous response to finish.
this already separates it from the usual REST over HTTP/1.1 and JSON path. HTTP/1.1 needs several connections for real concurrency. JSON repeats field names in every payload and spends CPU converting text back into typed values.
REST over HTTP/2 removes the connection limitation and also gets multiplexing and compressed headers. comparing gRPC against a new HTTP/1.1 connection for every request mostly benchmarks a bad HTTP client. the remaining difference comes from protobuf, generated contracts, gRPC framing and streaming semantics.
the latency of one internal call can be reduced to a simple model:
total = queue + encode + transport + handler + decode
when the handler performs a 20 ms database query, changing the protocol has little impact. when a service forwards hundreds of thousands of small cached messages per second, encode, decode, allocations and bytes on the wire become the workload.
i ran a codec-only benchmark on an Apple M5 using Rust release builds, prost 0.14.4, serde_json 1.0.151 and five million iterations per test. the message contained a chain id, address, price, liquidity, timestamp, sequence and boolean status.
protobuf encoded it in 82 bytes. JSON needed 181 bytes. across repeated runs, protobuf encoding was about 4.7x faster and decoding about 3.2x faster at the median.
that does not mean a gRPC endpoint is 4.7x faster than a REST endpoint. this benchmark excludes HTTP framing, syscalls, TLS, scheduling and application work. it isolates the codec.
the size difference still matters. at 100,000 messages per second, removing 99 bytes per message saves about 9.9 MB/s in one direction before transport overhead. the CPU difference compounds at the same rate, especially in Node.js where parsing and serialization compete with application code on the same event loop.
streaming changes the equation again. a long-lived gRPC stream avoids creating a new RPC and sending a new set of headers for every update. HTTP/2 flow control also slows the sender when the receiver stops consuming, instead of allowing an application queue to grow forever.
WSS can be even smaller when it carries a custom binary frame. it can also carry protobuf. but WSS only gives us an ordered bidirectional byte stream. message types, generated clients, deadlines, cancellation, status codes, retries, health checking and reconnection semantics remain application code.
gRPC standardizes most of this surface. deadlines can propagate through a call chain. cancellation can stop work that no client needs anymore. health services can remove an unhealthy backend. protobuf field numbers give the protocol an explicit compatibility model.
none of those guarantees configure themselves. gRPC has no deadline by default, so every call needs one. retries must be limited to operations that can safely run more than once. keepalive detects dead connections, while application health still requires a separate health signal.
kubernetes adds one important trap. a ClusterIP service balances TCP connections, not individual HTTP/2 streams. a gRPC channel can keep one connection alive for hours, so thousands of calls may stay pinned to one pod while other replicas receive nothing.
use client-side round robin with a headless service, xDS, a service mesh, or an L7 proxy that understands gRPC. the default pick-first behavior is rarely the distribution expected from a large deployment.
a stream also cannot move after it starts. if its pod dies, the stream fails. reconnecting is easy; resuming correctly requires a cursor, sequence number or application-level acknowledgement so messages are neither silently skipped nor applied twice.
HTTP/2 has a concurrent stream limit per connection too. once it is reached, new calls queue in the client. high-QPS services and long-lived streams may need separate channels or a channel pool. one overloaded stream should not block an unrelated class of traffic.
benchmarking the real migration means keeping everything else equal: same handler, same payload values, warm connections, same TLS or mTLS path, same pod placement and same concurrency.
measure p50 and p99 latency, QPS per core, CPU in both processes, allocations, bytes per message, open connections and reconnect behavior during a pod termination. repeat with the service mesh enabled if production uses one.
for our architecture, the useful shape is REST and WSS outside, then gRPC between the proxy and the applications. unary calls fit ordinary reads. server streams fit price and transaction updates. bidirectional streams fit subscriptions where the client changes filters without opening a new connection.
the largest gain appears when one warm connection carries many typed messages. fewer bytes, less parsing, fewer allocations and built-in flow control all compound.
gRPC will not make database work or pricing computation disappear. it removes protocol work around them, which matters once that work is repeated millions of times inside the cluster.