Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix buffer overflow error in KryoBackedDecoder #18338

Merged
merged 1 commit into from Sep 30, 2021

Conversation

jbartok
Copy link
Member

@jbartok jbartok commented Sep 21, 2021

Fixes #18316

@jbartok jbartok added this to the 7.3 RC1 milestone Sep 21, 2021
@jbartok
Copy link
Member Author

jbartok commented Sep 21, 2021

@adammurdoch pls. take a look. I'm sure this fix solves the problem in a sense:

  • when using the reproducer from the linked issue, there is a worker process that hangs and in its error logs I can clearly see the UnsupportedOperationException from the decoder
  • there is clearly an unhandled situation in the decoder (count > length)
  • I can reproduce the failure via unit tests, but only in a hacky and ephemeral way unfortunately, by changing the default buffer size of the decoder from 4096 to something smaller (have not found a way to force the buffer size of the decoder used in tests to a different value; and that would also be unrealistic)
  • I can use the issue reproducer to confirm that it works after this fix

BUT, I don't see how this situation can occur and that worries me. The encoder is written in such a way that it should never write more data than there is space in the decoding buffer... (Hence your initial assumptions.) Unless the decoder tries a read without fully consuming the data from its buffer, but I can't see that in tests. Any insight on that front?

@jbartok jbartok self-assigned this Sep 21, 2021
@jbartok jbartok requested a review from donat September 22, 2021 07:14
@jbartok jbartok modified the milestones: 7.3 RC1, 7.4 RC1 Sep 23, 2021
@jbartok
Copy link
Member Author

jbartok commented Sep 30, 2021

Further investigation revealed that there is nothing mysterious about this failure. The read method that has been adjusted in the fix is called in various scenarios to fill a buffer (it's used via various nested InputStreams). Considering how diverse the ways it can be called are, one can't make assumptions about what values the offset and length will have. And the original implementation does precisely that, it assumes that it will always be called with a 0 offset and maxim length, which is simply not true.

@donat turning the reproducer into an integration test turned out to not be feasible, but remote debugging cleared the mystery out of it.

Copy link
Member

@donat donat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jbartok
Copy link
Member Author

jbartok commented Sep 30, 2021

@bot-gradle test and merge

@bot-gradle
Copy link
Collaborator

OK, I've already triggered a build for you.

@bot-gradle bot-gradle merged commit 1258a71 into master Sep 30, 2021
@jbartok jbartok deleted the jbartok/fix-buffer-overflow branch September 30, 2021 14:03
@KroArtem
Copy link

KroArtem commented Jan 6, 2022

@jbartok , 👋 is there any chance for this fix to appear in 6.9.x branch? Due to some reasons we can't update our project to 7.x right now (working on it) but this problem is important and affects 6.x branch.

@ljacomet , I see you were backporting several fixes to 6.9.2, are there any plans for a new bugfix release?

@ljacomet
Copy link
Member

Hello @KroArtem ,

There is no current plans for having a 6.9.3 for now. However backporting this if there is one might be doable. I filed #19523 to keep track of this.

@KroArtem
Copy link

Thanks for the info @ljacomet

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

From version 6.3 to 6.4 long messages to "Gradle ANTLR Worker" stopped work.
5 participants