New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[NOREVIEW][NOMERGE] HTTP/3 stress data corruption hunt #101624
Conversation
Tagging subscribers to this area: @dotnet/ncl |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
Okay, this is interesting.
We hit validation because we apparently a byte was somehow lost. (we failed validation on the pipe which reads body contents). When I backtrace to what was read form the pipe which reads raw HTTP frames, I see
The reconstructed contents in a more readable form are
So the HTTP frame type got corrupted somehow, so the server skips the "unknown" HTTP frame and then processes the Now the issue is how the frame could've got corrupted. The HTTP3 Frame pipe runs validation code which asserts that the expected HTTP3 frame bytes have been received so the corruption must have happened after the contents were written to the pipe. |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
/azp run runtime-libraries stress-http |
Azure Pipelines successfully started running 1 pipeline(s). |
Hunt successfull, root cause found |
This PR is just for running http stress pipeline with custom code which will hopefully allow us to diagnose #76183.