avoid test flake in "delay" test; closes #2469 #2470
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
boneskull
force-pushed
the
issue/2469
branch
from
ceee1a5
to
058d00c
Compare
|
failure is sauce flake. I think the sauce connect tunnel is not getting shut down quickly enough. see #2478 |
1999
pushed a commit
to 1999/mocha
that referenced
this pull request
Sep 19, 2016
…roperty-currentretry * upstream/master: helpful error when necessary suite callback omitted; closes mochajs#1744 Fix an issue and add relevant tests when describe and xdescribe fail when not supplied with a callback (issue mochajs#1744). rename more fixtures; closes mochajs#2383 Report non-match to STDERR and exit if no files are matched (mochajs#2450) Exit process with correct error codes (mochajs#2445) fix test files not using .spec suffix fix test fixtures not using .fixture suffix always halt execution when async skip() called; closes mochajs#2465 (mochajs#2479) avoid test flake in "delay" test; closes mochajs#2469 (mochajs#2470) revert accidental change to bin paths disregard function names when testing under Node.js 6.5.0; closes mochajs#2467 (mochajs#2477) lints more files; add more files to lint check; closes mochajs#2457 adds *.orig to .gitignore Exclude the --inspect flag fix check-leaks to catch a leak whose name begins with 'd'
1999
pushed a commit
to 1999/mocha
that referenced
this pull request
Sep 22, 2016
…-files-cache * upstream/master: attempt windows-friendly reproducible case for mochajs#2315 fix: fix uncaught TypeError if error occurs on next tick, closes mochajs#2315 (mochajs#2439) helpful error when necessary suite callback omitted; closes mochajs#1744 Fix an issue and add relevant tests when describe and xdescribe fail when not supplied with a callback (issue mochajs#1744). rename more fixtures; closes mochajs#2383 Report non-match to STDERR and exit if no files are matched (mochajs#2450) Exit process with correct error codes (mochajs#2445) fix test files not using .spec suffix fix test fixtures not using .fixture suffix always halt execution when async skip() called; closes mochajs#2465 (mochajs#2479) avoid test flake in "delay" test; closes mochajs#2469 (mochajs#2470) revert accidental change to bin paths disregard function names when testing under Node.js 6.5.0; closes mochajs#2467 (mochajs#2477) lints more files; add more files to lint check; closes mochajs#2457 adds *.orig to .gitignore Exclude the --inspect flag fix check-leaks to catch a leak whose name begins with 'd'
|
Wait, what? How come that test didn't work? Do you have any insight @boneskull? It would be nice to have a link to an example of this test causing the run to fail. |
|
I'm just curious, and it kinda sucks having to nuke the test with no good workaround. |
|
We have a limited number of concurrent sessions (5) with SauceLabs. If we limit the concurrent sessions within Karma itself, our tests should not flake, if and only if we have only one (1) build running at any given time. If we have multiple builds running, we cannot ensure we won't exceed our concurrent session limit. I researched this, and there seems to be no way to limit the number of concurrent builds on Travis CI. You can limit the number of concurrent jobs (which is only part of a build), but this is not necessary, and slows CI way down, since it has to run all jobs in serial. Potential solutions:
From my conversations with @drazisil, it seems like CircleCI wouldn't be helpful to us, given the use case. But I'd love to know his opinion after having looked at our Travis config, and the problems we're having. |
|
@boneskull But I thought we were only using Karma (for the browser tests) on one of the TravisCI jobs (?) eg Node 7 |
|
Thought I'd pipe in and offer to help, should you decide to move part of the work to CircleCI. I'm pretty familiar with that service, and like it a lot. I've done some work with Sauce Labs in the past, on a python project. @boneskull BTW, I just recently sent you a note on gitter. I'm looking for a good place to plug in! |
This was referenced Feb 2, 2018
jimthedev
pushed a commit
to commitizen/cz-cli
that referenced
this pull request
May 24, 2018
This Pull Request updates dependency [mocha](https://github.com/mochajs/mocha) from `v3.1.2` to `v3.5.3` <details> <summary>Release Notes</summary> ### [`v3.5.3`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​353--2017-09-11) [Compare Source](mochajs/mocha@v3.5.2...v3.5.3) #### 🐛 Fixes - [#​3003]: Fix invalid entities in xUnit reporter first appearing in v3.5.1 ([@​jkrems]) [#​3003]: `mochajs/mocha#3003 --- ### [`v3.5.2`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​352--2017-09-10) [Compare Source](mochajs/mocha@v3.5.1...v3.5.2) #### 🐛 Fixes - [#​3001]: Fix AMD-related failures first appearing in v3.5.1 ([@​boneskull]) [#​3001]: `mochajs/mocha#3001 --- ### [`v3.5.1`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​351--2017-09-09) [Compare Source](mochajs/mocha@v3.5.0...v3.5.1) #### 📰 News - 📣 Mocha is now sponsoring [PDXNode](http://pdxnode.org)! If you're in the [Portland](https://wikipedia.org/wiki/Portland,_Oregon) area, come check out the monthly talks and hack nights! #### 🐛 Fixes - [#​2997]: Fix missing `xit` export for "require" interface ([@​solodynamo]) - [#​2957]: Fix unicode character handling in XUnit reporter failures ([@​jkrems]) #### 🔩 Other - [#​2986]: Add issue and PR templates ([@​kungapal]) - [#​2918]: Drop bash dependency for glob-related tests ([@​ScottFreeCode]) - [#​2922]: Improve `--compilers` coverage ([@​ScottFreeCode]) - [#​2981]: Fix tpyos and spelling errors ([@​jsoref]) [#​2997]: `mochajs/mocha#2997 [#​2957]: `mochajs/mocha#2957 [#​2918]: `mochajs/mocha#2918 [#​2986]: `mochajs/mocha#2986 [#​2922]: `mochajs/mocha#2922 [#​2981]: `mochajs/mocha#2981 [@​solodynamo]: https://github.com/solodynamo [@​jkrems]: https://github.com/jkrems [@​jsoref]: https://github.com/jsoref --- ### [`v3.5.0`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​350--2017-07-31) [Compare Source](mochajs/mocha@v3.4.2...v3.5.0) #### 📰 News - Mocha now has a [code of conduct](https://github.com/mochajs/mocha/blob/master/.github/CODE_OF_CONDUCT.md) (thanks [@​kungapal]!). - Old issues and PRs are now being marked "stale" by [Probot's "Stale" plugin](https://github.com/probot/stale). If an issue is marked as such, and you would like to see it remain open, simply add a new comment to the ticket or PR. - **WARNING**: Support for non-ES5-compliant environments will be dropped starting with version 4.0.0 of Mocha! #### 🔒 Security Fixes - [#​2860]: Address [CVE-2015-8315](https://nodesecurity.io/advisories/46) via upgrade of [debug](https://npm.im/debug) ([@​boneskull]) #### 🎉 Enhancements - [#​2696]: Add `--forbid-only` and `--forbid-pending` flags. Use these in CI or hooks to ensure tests aren't accidentally being skipped! ([@​charlierudolph]) - [#​2813]: Support Node.js 8's `--napi-modules` flag ([@​jupp0r]) #### 🔩 Other - Various CI-and-test-related fixes and improvements ([@​boneskull], [@​dasilvacontin], [@​PopradiArpad], [@​Munter], [@​ScottFreeCode]) - "Officially" support Node.js 8 ([@​elergy]) [#​2860]: `mochajs/mocha#2860 [#​2696]: `mochajs/mocha#2696 [#​2813]: `mochajs/mocha#2813 [@​charlierudolph]: https://github.com/charlierudolph [@​PopradiArpad]: https://github.com/PopradiArpad [@​kungapal]: https://github.com/kungapal [@​elergy]: https://github.com/elergy [@​jupp0r]: https://github.com/jupp0r --- ### [`v3.4.2`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​342--2017-05-24) [Compare Source](mochajs/mocha@v3.4.1...v3.4.2) #### 🐛 Fixes - [#​2802]: Remove call to deprecated `os.tmpDir` ([@​makepanic]) - [#​2820]: Eagerly set `process.exitCode` ([@​chrisleck]) #### 🔩 Other - [#​2778]: Move linting into an npm script ([@​Munter]) [@​chrisleck]: https://github.com/chrisleck [@​makepanic]: https://github.com/makepanic [@​Munter]: https://github.com/Munter [#​2778]: `mochajs/mocha#2778 [#​2802]: `mochajs/mocha#2802 [#​2820]: `mochajs/mocha#2820 --- ### [`v3.4.1`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​341--2017-05-14) [Compare Source](mochajs/mocha@v3.3.0...v3.4.1) Fixed a publishing mishap with git's autocrlf settings. --- ### [`v3.3.0`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​330--2017-04-24) [Compare Source](mochajs/mocha@v3.2.0...v3.3.0) Thanks to all our contributors, maintainers, sponsors, and users! ❤️ As highlights: - We've got coverage now! - Testing is looking less flaky \o/. - No more nitpicking about "mocha.js" build on PRs. #### 🎉 Enhancements - [#​2659]: Adds support for loading reporter from an absolute or relative path ([@​sul4bh]) - [#​2769]: Support `--inspect-brk` on command-line ([@​igwejk]) #### 🐛 Fixes - [#​2662]: Replace unicode chars w/ hex codes in HTML reporter ([@​rotemdan]) #### 🔍 Coverage - [#​2672]: Add coverage for node tests ([@​c089], [@​Munter]) - [#​2680]: Increase tests coverage for base reporter ([@​epallerols]) - [#​2690]: Increase tests coverage for doc reporter ([@​craigtaub]) - [#​2701]: Increase tests coverage for landing, min, tap and list reporters ([@​craigtaub]) - [#​2691]: Increase tests coverage for spec + dot reporters ([@​craigtaub]) - [#​2698]: Increase tests coverage for xunit reporter ([@​craigtaub]) - [#​2699]: Increase tests coverage for json-stream, markdown and progress reporters ([@​craigtaub]) - [#​2703]: Cover .some() function in utils.js with tests ([@​seppevs]) - [#​2773]: Add tests for loading reporters w/ relative/absolute paths ([@​sul4bh]) #### 🔩 Other - Remove bin/.eslintrc; ensure execs are linted ([@​boneskull]) - [#​2542]: Expand CONTRIBUTING.md ([@​boneskull]) - [#​2660]: Double timeouts on integration tests ([@​Munter]) - [#​2653]: Update copyright year ([@​Scottkao85], [@​Munter]) - [#​2621]: Update dependencies to enable Greenkeeper ([@​boneskull], [@​greenkeeper]) - [#​2625]: Use trusty container in travis-ci; use "artifacts" addon ([@​boneskull]) - [#​2670]: doc(CONTRIBUTING): fix link to org members ([@​coderbyheart]) - Add Mocha propaganda to README.md ([@​boneskull]) - [#​2470]: Avoid test flake in "delay" test ([@​boneskull]) - [#​2675]: Limit browser concurrency on sauce ([@​boneskull]) - [#​2669]: Use temporary test-only build of mocha.js for browsers tests ([@​Munter]) - Fix "projects" link in README.md ([@​boneskull]) - [#​2678]: Chore(Saucelabs): test on IE9, IE10 and IE11 ([@​coderbyheart]) - [#​2648]: Use `semistandard` directly ([@​kt3k]) - [#​2727]: Make the build reproducible ([@​lamby]) [@​boneskull]: https://github.com/boneskull [@​c089]: https://github.com/c089 [@​coderbyheart]: https://github.com/coderbyheart [@​craigtaub]: https://github.com/craigtaub [@​epallerols]: https://github.com/epallerols [@​greenkeeper]: https://github.com/greenkeeper [@​igwejk]: https://github.com/igwejk [@​kt3k]: https://github.com/kt3k [@​lamby]: https://github.com/lamby [@​Munter]: https://github.com/Munter [@​rotemdan]: https://github.com/rotemdan [@​seppevs]: https://github.com/seppevs [@​sul4bh]: https://github.com/sul4bh [#​2470]: `mochajs/mocha#2470 [#​2542]: `mochajs/mocha#2542 [#​2621]: `mochajs/mocha#2621 [#​2625]: `mochajs/mocha#2625 [#​2648]: `mochajs/mocha#2648 [#​2653]: `mochajs/mocha#2653 [#​2659]: `mochajs/mocha#2659 [#​2660]: `mochajs/mocha#2660 [#​2662]: `mochajs/mocha#2662 [#​2669]: `mochajs/mocha#2669 [#​2670]: `mochajs/mocha#2670 [#​2672]: `mochajs/mocha#2672 [#​2675]: `mochajs/mocha#2675 [#​2678]: `mochajs/mocha#2678 [#​2680]: `mochajs/mocha#2680 [#​2690]: `mochajs/mocha#2690 [#​2691]: `mochajs/mocha#2691 [#​2698]: `mochajs/mocha#2698 [#​2699]: `mochajs/mocha#2699 [#​2701]: `mochajs/mocha#2701 [#​2703]: `mochajs/mocha#2703 [#​2727]: `mochajs/mocha#2727 [#​2769]: `mochajs/mocha#2769 [#​2773]: `mochajs/mocha#2773 --- ### [`v3.2.0`](https://github.com/mochajs/mocha/blob/master/CHANGELOG.md#​320--2016-11-24) [Compare Source](mochajs/mocha@v3.1.2...v3.2.0) #### 📰 News ##### Mocha is now a JS Foundation Project! Mocha is proud to have joined the [JS Foundation](https://js.foundation). For more information, [read the announcement](https://js.foundation/announcements/2016/10/17/Linux-Foundation-Unites-JavaScript-Community-Open-Web-Development/). ##### Contributor License Agreement Under the foundation, all contributors to Mocha must sign the [JS Foundation CLA](https://js.foundation/CLA/) before their code can be merged. When sending a PR--if you have not already signed the CLA--a friendly bot will ask you to do so. Mocha remains licensed under the [MIT license](https://github.com/mochajs/mocha/blob/master/LICENSE). #### 🐛 Bug Fix - [#​2535]: Fix crash when `--watch` encounters broken symlinks ([@​villesau]) - [#​2593]: Fix (old) regression; incorrect symbol shown in `list` reporter ([@​Aldaviva]) - [#​2584]: Fix potential error when running XUnit reporter ([@​vobujs]) #### 🎉 Enhancement - [#​2294]: Improve timeout error messaging ([@​jeversmann], [@​boneskull]) - [#​2520]: Add info about `--inspect` flag to CLI help ([@​ughitsaaron]) #### 🔩 Other - [#​2570]: Use [karma-mocha](https://npmjs.com/package/karma-mocha) proper ([@​boneskull]) - Licenses updated to reflect new copyright, add link to license and browser matrix to `README.md` ([@​boneskull], [@​ScottFreeCode], [@​dasilvacontin]) [#​2294]: `mochajs/mocha#2294 [#​2535]: `mochajs/mocha#2535 [#​2520]: `mochajs/mocha#2520 [#​2593]: `mochajs/mocha#2593 [#​2584]: `mochajs/mocha#2584 [#​2570]: `mochajs/mocha#2570 [@​Aldaviva]: https://github.com/Aldaviva [@​jeversmann]: https://github.com/jeversmann [@​ughitsaaron]: https://github.com/ughitsaaron [@​villesau]: https://github.com/villesau [@​vobujs]: https://github.com/vobujs Thanks to all our contributors, sponsors and backers! Keep on the lookout for a public roadmap and new contribution guide coming soon. --- </details> --- This PR has been generated by [Renovate Bot](https://renovatebot.com).
This was referenced Sep 23, 2018
1 task
This was referenced Dec 6, 2018
This was referenced Dec 23, 2018
sgilroy
pushed a commit
to TwineHealth/mocha
that referenced
this pull request
Feb 27, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.