Why can a TCP load balancer mishandle gRPC, and can a browser call it?

The 30 second answer

gRPC keeps a long-lived HTTP/2 connection and multiplexes many RPCs as streams on that connection. A TCP load balancer only sees the connection, so all of those calls can stick to one backend. I would use an HTTP/2-aware proxy or client-side balancing that picks a backend per call. Browsers cannot speak native gRPC, because they do not expose raw HTTP/2 frames; gRPC-Web plus a proxy such as Envoy covers unary and server streaming, but not typical client or bidirectional streaming.

gRPC usually keeps a long-lived HTTP/2 connection and multiplexes many RPCs as streams on that connection. A layer-3 or layer-4 load balancer that only sees TCP connections will pin all of those RPCs to one backend for as long as the connection lives. That is the standard load-balancing interview trap.

Official guidance distinguishes:

Proxy load balancing, which can be L3/L4 (cheap, but connection-based) or L7 (the proxy parses HTTP/2 and can send different streams from one client to different backends).
Client-side load balancing, where the client is given a list of backends and an LB policy chooses a subchannel per RPC, not per connection.

In the gRPC client, load balancing is per call. Built-in policies include pick_first (the default: connect to addresses in order until one is ready) and round_robin (a subchannel per address, picks rotating among READY subchannels). round_robin is selected via service config.

The practical consequence: if you put a TCP-only load balancer in front of gRPC and expect even traffic, you will not get it. Use an HTTP/2-aware proxy, or client-side balancing, often with a resolver or xDS.

Browsers and gRPC-Web. Browsers do not expose the control needed to speak native gRPC-over-HTTP/2: there is no way to force HTTP/2, and no access to raw HTTP/2 frames. gRPC-Web is a related protocol that uses ordinary browser HTTP APIs (XHR or Fetch) and a proxy, commonly Envoy's gRPC-Web filter, to translate to backend gRPC. gRPC-Web supports unary and server-streaming calls. Client-streaming and bidirectional streaming are not part of the mandated gRPC-Web feature set in current browser transports. Official getting-started material still uses a proxy such as Envoy in front of the gRPC server.

Alternate interview scenarios

"All of our gRPC traffic from one client is hitting a single pod. Why?"

I would first ask whether the load balancer is TCP-only. If it is, that is the expected behaviour with a multiplexed HTTP/2 connection. The fix is an L7 HTTP/2-aware proxy, or client-side round_robin (or a similar per-call policy), not adding more TCP backends behind the same connection-hashing balancer. I would also check that the client is not stuck on pick_first, which is the default and will stay on the first ready address.

"Can the frontend call our gRPC search service directly?"

Not with native gRPC. I would put Envoy or another gRPC-Web proxy in front, and I would only promise unary or server-streaming calls from the browser. If the product needs a bidirectional stream in the browser, I would not claim gRPC-Web already provides it.

Sources (5)