A68: Random subsetting with rendezvous hashing LB policy #423
Conversation
s-matyukevich
changed the title
A68-random-subsetting.md
Outdated
| * When the lb policy is initialized it also creates a random 32-byte long `salt` string. | ||
| * After every resolver update the policy picks a new subset. It does this by implementing `rendezvous hashing` algorithm: | ||
| * Concatenate `salt` to each address in the list. | ||
| * For every resulting entity compute [MurmurHash3](https://en.wikipedia.org/wiki/MurmurHash) hash, which produces 128-byte output. |
There was a problem hiding this comment.
There is no dependency on murmur from grpc, at least in Go, as of today. You can use xxhash which is depended upon by ring_hash.
There was a problem hiding this comment.
updated to use xxhash, This changes the algorithms slightly as we can use random pre-generated seed instead of concatenating salt to each address. The new version is even simpler.
|
|
||
| * The policy receives a single configuration parameter: `subset_size`, which must be configured by the user. | ||
| * When the lb policy is initialized it also creates a random 32-byte long `salt` string. | ||
| * After every resolver update the policy picks a new subset. It does this by implementing `rendezvous hashing` algorithm: |
There was a problem hiding this comment.
I think it would help to define the algorithm in pseudo code. See https://github.com/grpc/proposal/blob/master/A55-xds-stateful-session-affinity.md#lb-policy-for-stateful-session-affinity or https://github.com/grpc/proposal/blob/master/A42-xds-ring-hash-lb-policy.md#aggregated-connectivity-state for examples of gRFCs that do this.
A68-random-subsetting.md
Outdated
|
|
||
| ### Handling Parent/Resolver Updates | ||
|
|
||
| When the resolver updates the list of addresses, or the LB config changes, Random subsetting LB will run the subsetting algorithm, described above, to filter the endpoint list. Then it will create a new resolver state with the filtered list of the addresses and pass it to the child LB. Attributes and service config from the old resolver state will be copied to the new one. |
There was a problem hiding this comment.
I think you should replace addresses with endpoints to take A61 into consideration.
A68-random-subsetting.md
Outdated
|
|
||
| ## Proposal | ||
|
|
||
| Introduce a new LB policy, `random_subsetting`. This policy selects a subset of addresses and passes them to the child LB policy. It maintains 2 important properties: |
There was a problem hiding this comment.
I think you need to replace addresses with endpoints to account for https://github.com/grpc/proposal/blob/master/A61-IPv4-IPv6-dualstack-backends.md, where each endpoint may have multiple addresses.
A68-random-subsetting.md
Outdated
| * The policy receives a single configuration parameter: `subset_size`, which must be configured by the user. | ||
| * When the lb policy is initialized it also creates a random 32-byte long `salt` string. | ||
| * After every resolver update the policy picks a new subset. It does this by implementing `rendezvous hashing` algorithm: | ||
| * Concatenate `salt` to each address in the list. |
There was a problem hiding this comment.
You'll have to decide which address, in case the endpoint has more than one (I think you can use the first address?).
There was a problem hiding this comment.
Updated the doc to use the first address.
| As described in [gRFC A52](https://github.com/grpc/proposal/blob/master/A52-xds-custom-lb-policies.md), gRPC has an LB policy registry, which maintains a list of converters. Every converter translates xDS LB policy to the corresponding service config. In order to allow using the Random subsetting LB policy via xDS, the only thing that needs to be done is providing a corresponding converter function. The function implementation will be trivial as the fields in the xDS LB policy will match exactly the fields in the service config. | ||
|
|
||
| ## Rationale | ||
| ### Alternatives Considered: Deterministic subsetting |
There was a problem hiding this comment.
You should probably discuss the trade offs of doing this kind of subsetting in the control plane, since it was discussed in the original proposal.
There was a problem hiding this comment.
Yeah, but I posted a link to the tl;dr; of the discussion, so you think this is not enough?
There was a problem hiding this comment.
I'm thinking of the option of doing random subsetting in the control plane by sending different EDS responses (with different subsets) to each dataplane, or the equivalent with other resolvers. It is simple to implement with xDS and works for Envoy and gRPC. IIRC the main argument for not going that route is the need to have an xDS infrastructure (this is a big barrier for our orgs, and probably others), and existing limitations of https://github.com/envoyproxy/go-control-plane.
This was discussed in https://github.com/grpc/proposal/pull/383/files#r1308024474.
In order to understand this proposal, I think users will need to understand the trade off of doing it as a balancer in each data plane rather than directly in service discovery.
Co-authored-by: Antoine Tollenaere <atollena@gmail.com>
Co-authored-by: Antoine Tollenaere <atollena@gmail.com>
Co-authored-by: Antoine Tollenaere <atollena@gmail.com>
… into random-subsetting
|
Bump on this. It has been almost a month since the proposal was submitted and no one from gRPC maintainers commented on it yet. cc @markdroth and @ejona86 since you both reviewed previous version of the proposal and have full context. |
Replaces #383
Related to #430 which describes an LB policy that could be used in combination with random subsetting to correct the resulting server-side load imbalance.