Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

test: fix flaky test-crypto-timing-safe-dqual-benchmarks #38476

Closed
wants to merge 1 commit into from
Closed

test: fix flaky test-crypto-timing-safe-dqual-benchmarks #38476

wants to merge 1 commit into from

Conversation

Trott
Copy link
Member

@Trott Trott commented Apr 29, 2021

Fixes: #38226

@github-actions github-actions bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Apr 29, 2021
@nodejs-github-bot

This comment has been minimized.

Sign in to view
@nodejs-github-bot
Copy link
Collaborator

CI: https://ci.nodejs.org/job/node-test-pull-request/37820/

@Trott
Copy link
Member Author

Trott commented Apr 30, 2021

You can see in #38226 where I bisected to determine that adding Proxy to bootstrap primordials is what caused this test to start failing much more frequently. I don't really understand the mechanics here, but maybe @not-an-aardvark or @aduh95 would be able to explain. (Or maybe not and it's a mystery.)

I still don't know if the issue here is with the test or with there really being a timing problem in Node.js core, but I'm guessing the latter but that it only manifests with a very fast CPU.

Regardless, fast-track to fix CI? Maybe leave the underlying issue open and investigate what's going on here?

@Trott
Copy link
Member Author

Trott commented Apr 30, 2021

(dqual-> equal in the commit message, but that can happen on landing.)

@Trott Trott added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Apr 30, 2021
@Trott
Copy link
Member Author

Trott commented Apr 30, 2021

https://ci.nodejs.org/job/node-stress-single-test/293/ is a stress test to show improvement. It will probably still show some failures, but not nearly as many as https://ci.nodejs.org/job/node-stress-single-test/273/ from this morning which ran against master and failed 941 times out of 1000 runs.

@Trott
Copy link
Member Author

Trott commented Apr 30, 2021

@nodejs/testing

@Trott Trott requested review from jasnell and aduh95 April 30, 2021 02:37
@Trott
Copy link
Member Author

Trott commented Apr 30, 2021

CI is green. Let's land this?

Stress test failed 192 times out of 1000 runs, which isn't great, but is way better than the current master branch which failed 941 times out of 1000 runs.

@aduh95
Copy link
Contributor

aduh95 commented Apr 30, 2021

Would it be worth trying to identify which Proxy call (line 112 or line 124, or both) is actually causing the issue? It can happen in a follow up PR though.

aduh95
aduh95 approved these changes Apr 30, 2021
View reviewed changes
Copy link
Contributor

@aduh95 aduh95 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a very surprising result, but the stress test results look promising.

@jasnell jasnell added the fast-track PRs that do not need to wait for 48 hours to land. label Apr 30, 2021
@github-actions
Copy link
Contributor

Fast-track has been requested by @jasnell. Please 👍 to approve.

@jasnell jasnell added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. and removed needs-ci PRs that need a full CI run. labels Apr 30, 2021
@jasnell
Copy link
Member

jasnell commented Apr 30, 2021

Landed in c7ccab3

@jasnell jasnell closed this Apr 30, 2021
jasnell pushed a commit that referenced this pull request Apr 30, 2021
@Trott @jasnell
test: fix flaky test-crypto-timing-safe-dqual-benchmarks
c7ccab3 
Fixes: #38226

PR-URL: #38476
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Ruy Adorno <ruyadorno@github.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@Trott
Copy link
Member Author

Trott commented Apr 30, 2021

Would it be worth trying to identify which Proxy call (line 112 or line 124, or both) is actually causing the issue? It can happen in a follow up PR though.

I've got another fix in the works that should make it possible to add back the Proxy() stuff removed here. Stay tuned.

targos pushed a commit that referenced this pull request May 3, 2021
@Trott @targos
test: fix flaky test-crypto-timing-safe-dqual-benchmarks
9ad611c 
Fixes: #38226

PR-URL: #38476
Reviewed-By: Antoine du Hamel <duhamelantoine1995@gmail.com>
Reviewed-By: Ruy Adorno <ruyadorno@github.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
@targos targos mentioned this pull request May 3, 2021
Merged
@Trott Trott deleted the hopeful branch September 25, 2022 17:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ruyadorno ruyadorno ruyadorno approved these changes

@jasnell jasnell jasnell approved these changes

@aduh95 aduh95 aduh95 approved these changes

Assignees
No one assigned
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. fast-track PRs that do not need to wait for 48 hours to land. flaky-test Issues and PRs related to the tests with unstable failures on the CI. test Issues and PRs related to the tests.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

investigate flaky test-crypto-timing-safe-equal-benchmarks
6 participants
@Trott @nodejs-github-bot @aduh95 @jasnell @ruyadorno @targos