Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A67: xDS dynamic parameters #381

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
48 changes: 34 additions & 14 deletions A67-xds-dynamic-parameters.md
Expand Up @@ -4,7 +4,7 @@
* Approver: @markdroth
* Status: Draft
* Implemented in:
* Last updated: 2023-07-21
* Last updated: 2024-03-18
* Discussion at:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please go ahead and create the mailing list thread, and then add a link to it here, as per the gRFC process.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still need to take care of this.


## Abstract
Expand Down Expand Up @@ -104,6 +104,9 @@ resources {

* CNCF xDS proposal
[TP2: Dynamically Generated Cacheable xDS Resources](https://github.com/cncf/xds/blob/main/proposals/TP2-dynamically-generated-cacheable-xds-resources.md).
* gRPC proposal[gRFC A47: xDS Federation](https://github.com/grpc/proposal/blob/master/A47-xds-federation.md).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can remove the words "gRPC proposal". Anyone reading this already knows what a gRFC is. :)

Describes the common structure of the XdsClient.


## Proposal

Expand Down Expand Up @@ -205,13 +208,21 @@ function ConstructADSRequest(authority_resource_names)

### Temporary environment variable protection
allanrbo marked this conversation as resolved.
Show resolved Hide resolved

During initial development, the GRPC_EXPERIMENTAL_XDS_DYNAMIC_PARAMETERS
environment variable will control whether ADS requests will use the
`resource_locators` field with dynamic parameters populated, instead of the
`resource_names` field. This will help guard against if an xDS server does not
yet support dynamic parameters, but the
[bootstrap generator](https://github.com/GoogleCloudPlatform/traffic-director-grpc-bootstrap)
already sets the `dynamic_parameters` fields.
During development, the dynamic parameter functionality will be guarded by the
GRPC_EXPERIMENTAL_XDS_DYNAMIC_PARAMETERS env var. Specifically, this env var
will control two things:

- Whether we read the new `dynamic_parameters` fields in the bootstrap
config (and therefore whether we populate the `resource_locators` field in
ADS requests). This guards against cases where the bootstrap file is
generated by a separate tool with a release process independent of that of
the application (e.g., the Traffic Director bootstrap generator), such
that a newer bootstrap config may be used with an older version of gRPC.
- Whether we look at the new `resource_names` field in ADS responses. This
guards against xDS servers that may incorrectly start sending new fields
to clients that are not yet ready to understand them.

The env var protection will be removed once this feature passes interop tests.

## Rationale

Expand All @@ -225,11 +236,20 @@ changing the parameters.

### No metadata reuse

An idea in TP2 is to reuse the node metadata field as dynamic parameters. I
propose we do not do this. Clients already have node metadata configured in
their bootstrap. If we reuse node metadata as dynamic parameters, clients will
begin sending dynamic parameters without knowing whether a given control plane
supports dynamic parameters.
An idea in TP2 is to reuse the node metadata field as dynamic parameters.

This idea is especially useful as a possible migration path for Envoy, where
servers use node metadata to determine which resources to send for wildcard
LDS and CDS subscrioptions. In contrast, node metadata is not as important in
the gRPC xDS ecosystem, so this migration path is not considered important.

Additionally, converting node metadata to dynamic parameters would be somewhat
complex to implement, since node metadata can have structured values, whereas
dynamic parameter values are always just strings.

Additionally, an explicit knob in the bootstrap config that determines whether
to use dynamic parameters is desirable for control. Using the presence of the
`dynamic_parameters` field gives us that.

### No merging of top-level and per-authority dynamic parmeters

Expand All @@ -243,4 +263,4 @@ want this parameter for the xdstp case.

## Implementation

Allan Boll (@allanrbo) plans to implement this in C++ in 2023 Q3.
Allan Boll (@allanrbo) plans to implement this in C++ in 2024 Q3.