Server-side connection management

[lucaspin]

Is your feature request related to a problem? Please describe.

I'm running into an issue regarding load balancing fairness. The situation is:

• On the server side of the equation, I have multiple replicas of a GRPC.Server, running behind a Kubernetes headless service.
• On the client side of the equation, I have a gRPC gateway translating HTTP calls into gRPC calls that go into the gRPC server above. The gRPC gateway is configured to use the round_robin load balancing policy, so each gRPC request is sent to one of the known IPs exposed by the headless Kubernetes service.

Everything works well and load is distributed fairly until the number of replicas for the gRPC server is scaled up. When that happens, the new pods receive no traffic unless I restart the gRPC clients, forcing them to re-establish the connections again.

Describe the solution you'd like

One of the solutions to this problem is described in the server-side connection management gRPC proposal. For this particular issue with load balancing fairness, having the ability to specify the MAX_CONNECTION_AGE and MAX_CONNECTION_AGE_GRACE options for the gRPC server could help force the clients to re-create connections after a certain period of time.

Describe alternatives you've considered

Another alternative would be to handle the re-creation of connections on the client side. The drawback with that is there's no way to handle that easily unless you take full control of the connection management on the client side, which is not always possible, especially when using libs like grpc-gateway.

Additional context

For gRPC servers implemented in Go, you can specify the maximum connection age with these parameters.

[sleipnir]

@lucaspin A possible workaround if you have the appropriate infrastructure in your kubernetes cluster is to use layer 7 routing instead of the native Kubernetes routing which is layer 4. Istio and Cilium are examples of products that add this capability to your kubernetes cluster

[lucaspin]

@sleipnir thanks for the suggestion. That's certainly something I'd like to use, but right now it wouldn't be possible to add a layer 7 load balancer in that mix, at least not without complicating things a bit.