Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xds: fix a concurrency issue in CSDS ClientStatus responses #8795

Merged
merged 7 commits into from Jan 12, 2022

Conversation

sergiitk
Copy link
Member

@sergiitk sergiitk commented Dec 29, 2021

Fixes an issue with ClientXdsClient.getSubscribedResourcesMetadata()
executed out of shared synchronization context, and leading to:

  • each individual config dump containing outdated data when
    an xDS resource is updated during CsdsService preparing the response
  • config dumps for different services being out-of-sync with each
    other when any of the related xDS resources is updated during
    CsdsService preparing the response

The fix replaces getSubscribedResourcesMetadata(ResourceType type)
with atomic getSubscribedResourcesMetadataSnapshot() returning
a snapshot of all resources for each type as they are
at the moment of a CSDS request.

Fix false negative in fetchClientConfig_unexpectedException,
where expected exception was actually caused by missing
bootstrap data, not by throwing error from the local
class implementation.

@sergiitk
Copy link
Member Author

Fixes #8642

@Override
Map<String, ResourceMetadata> getSubscribedResourcesMetadata(ResourceType type) {
Map<ResourceType, Map<String, ResourceMetadata>> getSubscribedResourcesMetadataSnapshot() {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this was needed because I discovered this test to be false negative, see PR description:

Fix false negative in fetchClientConfig_unexpectedException,
where expected exception was actually caused by missing
bootstrap data, not by throwing error from the local
class implementation.

Fixes an issue with ClientXdsClient.getSubscribedResourcesMetadata()
executed out of shared synchronization context, and leading to:

- each individual config dump containing outdated data when
  an xDS resource is updated during CsdsService preparing the response
- config dumps for different services being out-of-sync with each
  other when any of the related xDS resources is updated during
  CsdsService preparing the response

The fix replaces getSubscribedResourcesMetadata(ResourceType type)
with atomic getSubscribedResourcesMetadataSnapshot() returning
a snapshot of all resources for each type as they are
at the moment of a CSDS request.
@sergiitk
Copy link
Member Author

Ready for another review.

@sergiitk sergiitk enabled auto-merge (squash) January 12, 2022 01:33
@sergiitk sergiitk merged commit 7c4fe69 into grpc:master Jan 12, 2022
@sergiitk sergiitk deleted the csds-fix-concurrency branch January 12, 2022 16:49
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Apr 13, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants