NOTICE: Upcoming API changes to experimental Name Resolver (resolver) and LB Policy (balancer) packages

[dfawley]

To support the upcoming IPv4/IPv6 Dualstack design (gRFC A61), some changes will be required in the resolver and balancer. We will also be taking this opportunity to streamline the APIs a bit, in particular improving the LB policy API to make it easier to build and manage child LB policies (used extensively for our xDS support).

The design is not finalized, however, the following changes are currently planned:

• Dualstack support:

  • Add an abstraction for "endpoints" in resolver results, which will each contain a list of addresses (resolver: State: add Endpoints and deprecate Addresses #6471)
  • The default pick_first LB policy will perform the "happy eyeballs" algorithm (RFC8305) to establish a connection (pickfirst: Implement Happy Eyeballs #7725, pickfirst: Interleave IPv6 and IPv4 addresses for happy eyeballs #7742)
  • Update every leaf LB policy to use pick_first as its child to handle endpoint connections
    • leastrequest (leastrequest: Delegate subchannel creation to pickfirst #7969)
    • weighted_round_robin (balancer/weightedroundrobin: Switch Weighted Round Robin to use pick first instead of SubConns #7826)
    • roundrobin
    • ringhash
  • Internally change the implementation of health checking so it applies in pick_first instead of inside the SubConn (balancer: Add a SubConn.RegisterHealthListener API #7780, pickfirst: Register a health listener when used as a leaf policy #7832, health, grpc: Deliver health service updates through the health listener #7900)
    • Remove health checking implementation in the channel.

• General improvements / cleanups

  • Provide SubConn state updates via a callback instead of by calling balancer.UpdateSubConnState (balancer: add StateListener to NewSubConnOptions for SubConn state updates #6481)
  • Deprecate ClientConn.RemoveSubConn and add balancer.SubConn.Shutdown as its replacement (balancer: add SubConn.Shutdown; deprecate Balancer.RemoveSubConn #6493)
    • Remove ClientConn.RemoveSubConn
  • Deprecate resolver.Address.BalancerAttributes and add resolver.Endpoint.Attributes as its replacement (resolver: State: add Endpoints and deprecate Addresses #6471)
    • Remove resolver.Address.BalancerAttributes
  • Remove resolver.Address.Metadata.
    • The State.Attributes or Endpoint.Attributes fields should be used to communicate between resolver and LB policy. The Address.Attributes field should be used to communicate with the transport layer.
  • Remove the ability to specify multiple addresses per SubConn
    • This requires changing grpclb to delegate to the pick first LB policy instead of relying on the subchannel's implementation of pick first.
  • Remove SubConn.UpdateAddresses and ClientConn.UpdateAddresses
  • Introduce requirement that LB policies that create SubConns be responsible for shutting them down when Balancer.Close is called.
    • Note that with pick_first as the universal leaf policy, only pick_first will need to be updated to do this.
    • This can simplify graceful switch, or other parent policies that might be tracking created subchannels for this purpose.

We will do our best to make these changes in a way that allows several releases / months between adding the new version of an API and removing the old version.

Please keep in mind that all these APIs are still experimental, even the newly added ones. Until we can guarantee long-term support for them, they will remain this way. Note that any packages outside this repo that use an experimental symbol or import an experimental package should be considered experimental themselves, as the removal of a symbol would break compilation of that entire package.

[dfawley]

I think we can check off more of these items, right @arjan-bal ? Some or all of the remaining items may need to wait for things like etcd still using Address.Metadata (#8225)

[arjan-bal]

I've checked off the items that are complete. We need to wait for the the old pickfirst to be removed before we can remove some deprecated APIs, and health check code in the channel. We may be able to work on the requirement for LB policies to close their channel.