Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

eds: fix priority timeout failure when EDS removes all priorities #3830

Merged
merged 4 commits into from Aug 24, 2020

Conversation

menghanl
Copy link
Contributor

Without this fix, when the EDS response removes all priorities, after the timeout, the priority check panics because priority is unset.

…es all priorities

Without this fix, when the EDS response removes all priorities, after the timeout, the priority check panics because priority is unset.
@easwars
Copy link
Contributor

easwars commented Aug 21, 2020

Where does the panic happen?

Should we rather fix the func passed to timer.AfterFunc so that it can handle this case?

// Test the case where the first and only priority is removed.
func (s) TestEDSPriority_FirstPriorityUnavailable(t *testing.T) {
const testPriorityInitTimeout = time.Second
defer func() func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow .... never seen this before defer func() func().
Cool

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It isn't much different from: assign new value, then defer func() to reassign old value back.

The only advantage is, it limits the scope of old, so you can just name it old, instead of oldPriorityInitTimeout..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

defer func(t time.Duration) {
  defaultPriorityInitTimeout = t
}(defaultPriorityInitTimeout)
defaultPriorityInitTimeout = testPriorityInitTimeout

?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

clab2 := testutils.NewClusterLoadAssignmentBuilder(testClusterNames[0], nil)
edsb.handleEDSResponse(parseEDSRespProtoForTesting(clab2.Build()))

// Wait after the init timer timeout, to ensure it doesn't fail.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe wait for 2 times the value of testPriorityInitTimeout to be double sure that the timeout has fired?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@easwars easwars assigned menghanl and unassigned easwars Aug 21, 2020
@menghanl
Copy link
Contributor Author

menghanl commented Aug 22, 2020

The panic happens here:

Maybe it should not panic, but should return a value. Will need to think what the correct value to return.

@easwars
Copy link
Contributor

easwars commented Aug 22, 2020

Maybe it should not panic, but should return a value. Will need to think what the correct value to return.

Yeah, I also think that we should not panic from the equal method. The others might still be OK for now.

@menghanl
Copy link
Contributor Author

Made equal return right result for unset values instead of panic. PTAL.

For the other two (> and <), I'm leaving them as TODO for now, it's not clear what they should return. I checked the places they are called, they both seem to be pretty safe.

@menghanl menghanl merged commit 410880d into grpc:master Aug 24, 2020
@menghanl menghanl deleted the xds_priority_unset branch August 24, 2020 18:09
menghanl added a commit to menghanl/grpc-go that referenced this pull request Aug 24, 2020
…rities (grpc#3830)

Without this fix, when the EDS response removes all priorities, after the timeout, the priority check panics because priority is unset.
menghanl added a commit to menghanl/grpc-go that referenced this pull request Aug 24, 2020
…s all priorities (grpc#3830)

Without this fix, when the EDS response removes all priorities, after the timeout, the priority check panics because priority is unset.
menghanl added a commit that referenced this pull request Aug 24, 2020
…erry pick of #3830) (#3839)

Without this fix, when the EDS response removes all priorities, after the timeout, the priority check panics because priority is unset.
menghanl added a commit that referenced this pull request Aug 24, 2020
…erry pick of #3830) (#3840)

Without this fix, when the EDS response removes all priorities, after the timeout, the priority check panics because priority is unset.
@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 25, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants