A76: Improvements to the Ring Hash LB Policy #412

atollena · 2024-01-22T08:29:11Z

Based on discussion in grpc/grpc#33356.

A76-ring-hash-improvements.md

murgatroid99 · 2024-01-22T17:49:21Z

A76-ring-hash-improvements.md

+`hash_key` name resolver attribute changes.
+
+The xDS resolver will be changed so that when converting EDS responses to
+resolver endpoints, it will set the `hash_key` name resolver attribute to the


EDS responses are handled in a load balancing policy, so I'm not sure if "name resolver attribute" is the right terminology here.

OK, I interpreted your comment as two things:

It's not the xDS resolver, but the cluster resolver LB policy that does this, so I replaced the term.

"name resolver attributes" are not really a thing. I replaced this with "Endpoint attributes", which is metadata attached to each endpoint, used for example to store locality IDs and weight, and which IIUC are language specific, but all languages have this feature. I don't know if this is clear enough for cross-language interpretations -- please let me know.

1 seems to be in motion with A74, which removes the cluster resolver LB policy in favor of the "xDS resolver".

murgatroid99 · 2024-01-22T18:02:56Z

A76-ring-hash-improvements.md

+      uint64 min_ring_size = 1;        // Optional, defaults to 1024.
+      uint64 max_ring_size = 2;        // Optional, defaults to 4096, max is 8M.
+
+      string request_metadata_key = 3; // Optional, defaults to the empty string.


In the context of a Protobuf 3 message, an unset string field is the same as an empty string value, so stating that the field "defaults to the empty string" is redundant with the default Protobuf behavior. In addition, "defaults to the empty string" seems to imply that the empty string value will be used in some way, but earlier it says that "not set" and "empty" are treated the same. I think it would be clearer to say something like "Optional, unused if unset".

Yes, replaced with "Optional, unused if unset.". I also removed mentions of "not set" in the implementation section, since there is no way to distinguish not set from empty.

murgatroid99 · 2024-01-22T18:08:13Z

A76-ring-hash-improvements.md

+
+After the addition of this field, the `ring_hash` LB policy config will be:
+
+    message RingHashLoadBalancingConfig {


To get protobuf syntax highlighting, instead of indenting the code, use a fenced code block, with the language name proto.

murgatroid99 · 2024-01-22T18:12:53Z

A76-ring-hash-improvements.md

+`GRPC_EXPERIMENTAL_XDS_RING_HASH_ENDPOINT_HASH_KEY` environment variable. This
+will protect from the case where an xDS control plane is already setting the
+`LbEndpoint.Metadata` `envoy.lb` `hash_key` field, in which case deploying this
+new behavior would churn all endpoint hash keys. This environment variable will


Deploying a new version of gRPC will clear all ring_hash state anyway, so I don't understand how this environment variable would do anything to prevent churn.

Deploying with the environment variable unset will keep the current behavior of hashing the IP address instead of the new behavior of using the value from LbEndpoint.Metadata. As a result, the locations of existing endpoints on the ring will be guaranteed to be the same as before, and requests will continue being routed to the same endpoint. It will churn connections, if that's what you mean, but not endpoint locations on the ring. Suddenly changing all locations on the ring may have unintended consequences, removing the locality that ring hash may have been used for in the first place.

I'm thinking of a better phrasing, and the best I could think of was to replace "churn" with "change". Let me know if this is still unclear.

markdroth

Thanks for writing this!

Overall, the design looks good, but I think there is one significant issue related to where we compute the request hash, since I don't think it will work to do it in the picker. That will probably need a bit more discussion to resolve.

Please let me know if you have any questions. Thanks!

A76-ring-hash-improvements.md

markdroth

Thanks for making those updates! We'll get back to you on the open question once Eric gets back.

A76-ring-hash-improvements.md

atollena · 2024-03-22T09:30:20Z

@markdroth just checking in on this, I've made the updates regarding handling the empty header. I'll start testing an implementation for this internally, but in the meantime would appreciate another review. Thanks!

markdroth

Sorry for the delay! This looks really good overall. Comments are mostly things to improve clarity of the doc.

Please let me know if you have any questions. Thanks!

A76-ring-hash-improvements.md

markdroth · 2024-03-22T23:27:46Z

A76-ring-hash-improvements.md

+
+This proposal extends the following existing gRFC:
+
+* [gRFC A42: xDS Ring Hash LB Policy][A42]


It's probably also worth referencing A61, which significantly simplified the ring_hash picker logic.

OK, thanks for pointing this out, I wasn't aware of A61's impact on ring hash. I spent some time to understand how that affects the implementation I started in Go. Go implements A62 (pick first sticky transient failure) but not A61, so it cannot take advantage of this without further refactoring the ring hash policy. I'll sync up with the Go team to see what the best path forward is, but I imagine it'd be best to start with implementing at least part of A61 (delegating to pick_first).

After a quick discussion with @dfawley I decided to try to implement this before A61. I think the proposal could be more clear in this case, at least for Go, because there is a difference between immediately attempting to connect (which happens only for subchannels in "idle" state), and queuing a connect for subchannels in transient failure, where a connection attempt will be triggered when the subchannel returns to idle state after the connection backoff.

I implemented the new behavior of scanning forward for a ready subchannel, either when:

the subchannel picked randomly is idle (immediate connection attempt triggered) or,

the subchannel picked randomly is in transient failure (connection attempt queued)

I think it has the desired behaviour of making sure we don't trigger more than one connection attempt on each pick if there is a ready subchannel, not adding latency if there is a ready subchannel, and converge to random. But I also considered only implementing the scanning forward for ready subchannel when we triggered an immediate connection attempt, which happens when either:

the picked subchannel is idle

the picked subchannel is in transient failure, and the second is idle

the picked suchannel is in transient failure, the second is also in transient failure, and we found an idle connection before a ready one when scanning the ring forward.

This ambiguity will disappear when all implementations have A61 implemented. This is planned for this quarter for Go, IIUC, but I imagine there may be reasons for some implementations to want to implement A76 before A61. My question is whether you think it's worth adding a note about it in this gRFC to lift this ambiguity.

My inclination is that this logic is already complicated enough to understand, so I'd prefer to avoid muddying the waters by trying to do this without having first implemented the relevant parts of A61. From first glance, what you describe here seems reasonable, but I think I'd have to stare at it for a lot longer to convince myself it didn't have any potential pitfalls.

@dfawley, is it not feasible to implement just the ring_hash part of A61 before implementing this change?

A76-ring-hash-improvements.md

markdroth

This looks great to me!

@ejona86 @dfawley Would you two please do a review pass as well? Thanks!

A76: Improvements to the Ring Hash LB Policy

e558476

This was referenced Jan 22, 2024

Support using the ring hash balancer without xDS grpc/grpc#33356

Open

A76: Improvements to the Ring Hash LB Policy atollena/proposal#1

Closed

Comments from Sergey

5407438

atollena force-pushed the A76 branch from a1d2530 to 5407438 Compare January 22, 2024 13:52

markdroth self-assigned this Jan 22, 2024

murgatroid99 reviewed Jan 22, 2024

View reviewed changes

comments from Michael

cc3af7f

markdroth reviewed Jan 22, 2024

View reviewed changes

feedback from Mark

01449d7

markdroth reviewed Jan 23, 2024

View reviewed changes

A76-ring-hash-improvements.md Outdated Show resolved Hide resolved

better handling of empty hash header

cf65a14

markdroth reviewed Mar 22, 2024

View reviewed changes

feedback from Mark

202a225

markdroth reviewed Apr 9, 2024

View reviewed changes

s-matyukevich mentioned this pull request Apr 11, 2024

Random subsetting with rendezvous hashing LB policy s-matyukevich/proposal#2

Closed

ginayeh requested review from ejona86 and dfawley April 23, 2024 16:14

This was referenced Apr 23, 2024

ringhash: make endpoint hash key configurable, and set it from EDS grpc/grpc-go#7161

Open

ringhash: allow setting request hash key explicitly grpc/grpc-go#7170

Open

add implementation link

0de42f2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

A76: Improvements to the Ring Hash LB Policy #412

A76: Improvements to the Ring Hash LB Policy #412

atollena commented Jan 22, 2024 •

edited by markdroth

murgatroid99 Jan 22, 2024

atollena Jan 22, 2024 •

edited

murgatroid99 Jan 22, 2024

atollena Jan 22, 2024 •

edited

murgatroid99 Jan 22, 2024

atollena Jan 22, 2024

murgatroid99 Jan 22, 2024

atollena Jan 22, 2024

markdroth left a comment

markdroth left a comment

atollena commented Mar 22, 2024

markdroth left a comment

markdroth Mar 22, 2024

atollena Apr 2, 2024 •

edited

atollena Apr 29, 2024

markdroth May 6, 2024

markdroth left a comment


		After the addition of this field, the `ring_hash` LB policy config will be:

		message RingHashLoadBalancingConfig {


		This proposal extends the following existing gRFC:

		* [gRFC A42: xDS Ring Hash LB Policy][A42]

A76: Improvements to the Ring Hash LB Policy #412

Are you sure you want to change the base?

A76: Improvements to the Ring Hash LB Policy #412

Conversation

atollena commented Jan 22, 2024 • edited by markdroth

Choose a reason for hiding this comment

atollena Jan 22, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atollena Jan 22, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markdroth left a comment

Choose a reason for hiding this comment

markdroth left a comment

Choose a reason for hiding this comment

atollena commented Mar 22, 2024

markdroth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

atollena Apr 2, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

markdroth left a comment

Choose a reason for hiding this comment

atollena commented Jan 22, 2024 •

edited by markdroth

atollena Jan 22, 2024 •

edited

atollena Jan 22, 2024 •

edited

atollena Apr 2, 2024 •

edited