Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate flaky wpt/test-user-timing test #40449

Open
mhdawson opened this issue Oct 13, 2021 · 4 comments
Open

Investigate flaky wpt/test-user-timing test #40449

mhdawson opened this issue Oct 13, 2021 · 4 comments
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI.

Comments

@mhdawson
Copy link
Member

mhdawson commented Oct 13, 2021

Version

master

Platform

arm + x64

Subsystem

wpt

What steps will reproduce the bug?

Run test wpt/test-user-timing - https://ci.nodejs.org/job/node-test-binary-arm-12+/12718/RUN_SUBSET=2,label=pi2-docker/testReport/(root)/test/wpt_test_user_timing/

How often does it reproduce? Is there a required condition?

intermittent

What is the expected behavior?

test passes

What do you see instead?

test fails

Additional information

No response

@mhdawson
Copy link
Member Author

@Mesteery Mesteery added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Oct 13, 2021
@mhdawson
Copy link
Member Author

From the reliability report this is one of the most problematic intermittent tests causing CI failures with 9 reported failures.

@mhdawson
Copy link
Member Author

PR to mark flaky - #41203

@mhdawson mhdawson reopened this Dec 16, 2021
@joyeecheung
Copy link
Member

https://ci.nodejs.org/job/node-test-binary-arm-12+/14763/RUN_SUBSET=0,label=pi3-docker/testReport/junit/(root)/test/wpt_test_user_timing_/

Not sure if it's the same issue, here is another failure, looks like the measurements have a slightly higher error than anticipated.

---- user_timing_exists.any.js ----
[PASS] self.performance.clearMeasures is defined.
{
  'idlharness.any.js': { skip: 'idlharness cannot recognize Node.js environment' },
  'invoke_with_timing_attributes.worker.js': { skip: 'importScripts not supported' },
  'performance-measure-invalid.worker.js': { skip: 'importScripts not supported' },
  'mark.any.js': {
    fail: {
      unexpected: [
        'assert_approx_equals: expected 6722.132648468018 +/- 20 but got 6698.316417694092'
      ]
    }
  }
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI.
Projects
None yet
Development

No branches or pull requests

3 participants