Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[frontend] Flaky tests in CompareV2 resulting in false failures in PRs #10811

Open
HumairAK opened this issue May 9, 2024 · 0 comments
Open

[frontend] Flaky tests in CompareV2 resulting in false failures in PRs #10811

HumairAK opened this issue May 9, 2024 · 0 comments
Labels
area/frontend kind/bug

Comments

@HumairAK
Copy link
Contributor

HumairAK commented May 9, 2024 •
edited

Tests in CompareV2.test.tsx seem to be very Flaky, and keep resulting in random failures in PRs, I constantly have to re-run these tests to get it fully passing. Some examples of failure errors I've seen:

failures.1 
Summary of all failing tests

FAIL src/pages/CompareV2.test.tsx (91.424 s)

● CompareV2 › Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs

thrown: "Exceeded timeout of 5000 ms for a test.

Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

410 | });

411 |

> 412 | it('Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs', async () => {

| ^

413 | const getRunSpy = jest.spyOn(Apis.runServiceApiV2, 'getRun');

414 | runs = [newMockRun(MOCK_RUN_1_ID), newMockRun(MOCK_RUN_2_ID), newMockRun(MOCK_RUN_3_ID)];

415 | getRunSpy.mockImplementation((id: string) => runs.find(r => r.run_id === id));

at src/pages/CompareV2.test.tsx:412:3

at Object.<anonymous> (src/pages/CompareV2.test.tsx:34:1)

Test Suites: 1 failed, 113 passed, 114 total

Tests: 1 failed, 1611 passed, 1612 total

Snapshots: 427 passed, 427 total

Time: 238.994 s
failures.2 
● CompareV2 › Parameters and Scalar metrics tab initially enabled with loading then error, and switch tabs

Unable to find an element with the text: An error is preventing the Scalar Metrics from being displayed.. This could be because the text is broken up by multiple elements. In this case, you can provide a function for your text matcher to make your matcher more flexible.

<body>
....
OMITTED
....

426 | screen.getByText('There are no Parameters available on the selected runs.');

427 | screen.getByText('An error is preventing the Scalar Metrics from being displayed.');

428 |

at waitForWrapper (node_modules/@testing-library/react/node_modules/@testing-library/dom/dist/wait-for.js:173:27)

at Object.<anonymous> (src/pages/CompareV2.test.tsx:425:18)

Examples:

Love this idea? Give it a 👍.
@HumairAK HumairAK changed the title [frontend] Flakey tests in CompareV2 resulting in false failures in PRs [frontend] Flaky tests in CompareV2 resulting in false failures in PRs May 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/frontend kind/bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@HumairAK