Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integration test setup upgrades #10351

Merged
merged 14 commits into from Feb 15, 2021
Merged

Integration test setup upgrades #10351

merged 14 commits into from Feb 15, 2021

Conversation

arindam1993
Copy link
Contributor

@arindam1993 arindam1993 commented Feb 4, 2021

Fixes #8902

A bunch of quality of life updates for the integration tests:

Screen Shot 2021-02-04 at 3 31 34 PM

  • Catch and report errors thrown during a test run via tap using t.error(e). This accounts for any synchronously thrown errors.
  • Ensure that tests, irrespective of timeouts or errors always cleanup map and container so next test can proceed smoothly.
  • Tests that throw errors async during operations will now report errors correctly in the report.

Screen Shot 2021-02-09 at 12 54 08 PM

  • Tests "should" 🤞 be less flaky:
    • the wait operation now waits for atleast one frame to be rendered,
    • Synchronous calls to the private map._render() have also been removed and replaced with async wait-for-one-frame via a map event.

I also discovered that a few tests were silently throwing errors, I've fixed them and updated their expected outputs.

Launch Checklist

  • briefly describe the changes in this PR
  • apply changelog label ('bug', 'feature', 'docs', etc) or use the label 'skip changelog'

@arindam1993 arindam1993 added the skip changelog Used for PRs that do not need a changelog entry label Feb 5, 2021
Copy link
Contributor

@ryanhamley ryanhamley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. The new error handling should be a nice improvement

@andycalder
Copy link
Contributor

andycalder commented Feb 14, 2021

I've been having tons of trouble with flaky tests so I'm very happy to see this PR!

@mourner
Copy link
Member

mourner commented Feb 15, 2021

@arindam1993 needs a rebase / yarn update to resolve dependency conflicts.

@arindam1993 arindam1993 merged commit 2779f8d into main Feb 15, 2021
@arindam1993 arindam1993 deleted the integration-test-error branch February 15, 2021 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip changelog Used for PRs that do not need a changelog entry testing 💯 workflow 💅
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Browser query-test runner doesnt report test timeouts and crashed tests correctly
4 participants