Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] WIP: run tests concurrently; see #2839 #4198

Closed
wants to merge 92 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
92 commits
Select commit Hold shift + click to select a range
ef40c9e
WIP: concurrency based on worker threads; see #2839 [ci skip]
boneskull Mar 3, 2020
0a62f3f
somewhat reasonable re-implementation using child processes
boneskull Mar 12, 2020
db4b9a9
revert refactor of setting test.speed in runner
boneskull Mar 12, 2020
7eac21a
pull in promise.allsettled shim
boneskull Mar 12, 2020
d4a649e
remove concurrency stuff from bundle
boneskull Mar 12, 2020
61ab96e
string tweak in package-scripts.js
boneskull Mar 13, 2020
fbf7d8a
browserify fixes
boneskull Mar 13, 2020
485d6ab
add some tests
boneskull Mar 18, 2020
e604fa0
add a lot of unit test coverage around serialization/deserialization
boneskull Mar 23, 2020
d154141
increase timeout for options tests
boneskull Mar 23, 2020
76105e7
upgrade unexpected-eventemitter
boneskull Mar 24, 2020
def4815
add some tests for BufferedRunner
boneskull Mar 24, 2020
20c6604
add more buffered-runner tests
boneskull Mar 25, 2020
5092f08
add test for Buffered reporter
boneskull Mar 26, 2020
da7e206
remove useless calls to rewiremock
boneskull Mar 26, 2020
ce015ee
ignoring some missed coverage
boneskull Mar 26, 2020
e3a38a9
add unit-ish tests for worker
boneskull Mar 26, 2020
947a00c
add some assertions for RawRunResult to match JSONRunResult
boneskull Mar 27, 2020
b161fc0
add assertion to check exit code of a RawResult
boneskull Mar 27, 2020
45233d2
add a debug stmt to Buffered reporter
boneskull Mar 27, 2020
8aaf72f
fix issue with serialization of cyclic objects
boneskull Mar 27, 2020
9090d41
add a couple debugs to worker
boneskull Mar 27, 2020
552d92b
add a simple integration test for --jobs
boneskull Mar 27, 2020
269bccc
refactor/reorganize some of the integration test helpers
boneskull Mar 27, 2020
9a2a816
add tests for --parallel with --retries and --allow-uncaught
boneskull Mar 27, 2020
b4c7b7f
increase default test timeout to 1s
boneskull Mar 27, 2020
2cb5768
fix some flaky tests when run in wallaby
boneskull Mar 29, 2020
4cfafda
integration test helper: add `command` prop to RawRunResult for easy …
boneskull Mar 29, 2020
1b05f6f
tweak a couple debug statements
boneskull Mar 29, 2020
c6e560d
serialization: add missing functions
boneskull Mar 29, 2020
2c3b564
--file and --parallel are now mutually exclusive
boneskull Mar 29, 2020
10cd2c4
add --parallel test for --bail and for each reporter
boneskull Mar 29, 2020
4cfe37c
fix stack prop access on custom errors
boneskull Mar 29, 2020
d5d6e89
add --parallel test for ESM
boneskull Mar 29, 2020
291b40a
fix serialization of circular data structures
boneskull Mar 30, 2020
42e466c
--parallel is mutually exclusive with --sort
boneskull Mar 31, 2020
c9c3779
don't test --parallel w/ "base" reporter
boneskull Mar 31, 2020
fc912e8
remove unnecessary bail handling
boneskull Mar 31, 2020
c8e8938
remove cruft from wallaby config
boneskull Mar 31, 2020
aaef65f
fix "sort" issue
boneskull Mar 31, 2020
1a37e18
increase timeout test timeout for appveyor
boneskull Apr 1, 2020
1c0971c
cleanup .mocharc.yml
boneskull Apr 1, 2020
9d75852
add a watch-ignore rule for using --watch with mocha's own tests
boneskull Apr 7, 2020
622d334
fix event listener leak in worker.spec.js
boneskull Apr 7, 2020
e52fea8
fix return value of BufferedRunner#run
boneskull Apr 7, 2020
938ce25
create a FileCollectionOptions typedef
boneskull Apr 7, 2020
9a8adf6
fix debug statements to use plain strings
boneskull Apr 7, 2020
525cb7c
reporter fixes for --parallel
boneskull Apr 8, 2020
c9bfa67
implement --parallel --watch
boneskull Apr 8, 2020
a775327
integration test helper improvements
boneskull Apr 8, 2020
9ffe03f
implement an alternative way to define root hooks; works w/ `--parallel`
boneskull Apr 8, 2020
926a934
remove warning about --parallel run with a single test file
boneskull Apr 8, 2020
7210ca0
rootHooks fix
boneskull Apr 8, 2020
c79c359
another rootHooks fix
boneskull Apr 8, 2020
fb6300b
test actually checks rootHooks behavior
boneskull Apr 8, 2020
41dcbc9
try to force color in CI
boneskull Apr 9, 2020
0415bb7
json-stream reporter is incompatible with --parallel
boneskull Apr 9, 2020
8d1a173
update lockfile after rebase
boneskull Apr 9, 2020
7f35825
better failure modes
boneskull Apr 10, 2020
da006c7
utils: add a test for async functions in type()
boneskull Apr 10, 2020
43ae67f
all for async definition of mocha root hooks via --require
boneskull Apr 10, 2020
e289e40
improvified some debug statements
boneskull Apr 10, 2020
28158ef
remove some done TODOs
boneskull Apr 14, 2020
63d703a
refactor: rename param name in Mocha#reporter method for clarity
boneskull Apr 14, 2020
680240f
refactor validatePlugin() to throw nicer errors
boneskull Apr 14, 2020
4b96461
exit fixes
boneskull Apr 14, 2020
97561ad
fix jsdoc types
boneskull Apr 14, 2020
a0423a0
avoid flake in --require test via burly regexp
boneskull Apr 14, 2020
118c137
improve another regexp in --require test
boneskull Apr 14, 2020
6afce46
update some debug statements in bin/mocha
boneskull Apr 15, 2020
813016b
fix landing reporter to not spam SIGINT handlers
boneskull Apr 15, 2020
bcc0e4e
fix integration test helper to not completely ignore SIGINT
boneskull Apr 15, 2020
88887b6
BufferedRunner fixes
boneskull Apr 16, 2020
6545108
try to fix ESM test
boneskull Apr 16, 2020
e10b5e6
tweak nyc
boneskull Apr 16, 2020
fdf9916
add warning if no multiple cores
boneskull Apr 16, 2020
d8c7fbc
get some debug info
boneskull Apr 16, 2020
e23b4e2
fix default job count, job tests
boneskull Apr 16, 2020
52368a9
fix test timeouts in slow parallel tests
boneskull Apr 16, 2020
a509d1c
reduce debug noise from worker
boneskull Apr 16, 2020
18f7030
fix ESM parallel tests
boneskull Apr 16, 2020
cba2d7e
fix node arg passing to worker processes
boneskull Apr 16, 2020
4ad404f
fix string that should have been a var in BufferedRunner
boneskull Apr 17, 2020
7e2717b
fix --require integration test
boneskull Apr 17, 2020
25d57ec
remove debugging from travis CI
boneskull Apr 17, 2020
e934030
add a serial-mode run (and job names?)
boneskull Apr 17, 2020
4328619
update package-scripts.js to respect MOCHA_PARALLEL=0
boneskull Apr 17, 2020
80d5c06
allow emoji on website via markdown-it-emoji
boneskull Apr 20, 2020
da44ea1
squelch warning about missing language in a fenced code block
boneskull Apr 20, 2020
f5a8a76
website: add styles for a "normal" single-column ul
boneskull Apr 20, 2020
c8236d5
add docs for parallel mode and root hook plugins
boneskull Apr 20, 2020
0b13c92
fix bad anchors
boneskull Apr 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 2 additions & 0 deletions .eleventy.js
Expand Up @@ -31,6 +31,8 @@ module.exports = function(eleventyConfig) {

markdown.use(require('markdown-it-prism'));

markdown.use(require('markdown-it-emoji'));

eleventyConfig.setLibrary('md', markdown);

return {
Expand Down
27 changes: 16 additions & 11 deletions .eslintrc.yml
Expand Up @@ -16,17 +16,22 @@ rules:
- safe
overrides:
- files:
- scripts/**/*.js
- package-scripts.js
- karma.conf.js
- .wallaby.js
- .eleventy.js
- bin/*
- lib/cli/**/*.js
- test/node-unit/**/*.js
- test/integration/options/watch.spec.js
- test/integration/helpers.js
- lib/growl.js
- 'scripts/**/*.js'
- 'package-scripts.js'
- 'karma.conf.js'
- '.wallaby.js'
- '.eleventy.js'
- 'bin/*'
- 'lib/cli/**/*.js'
- 'test/node-unit/**/*.js'
- 'test/integration/options/watch.spec.js'
- 'test/integration/helpers.js'
- 'lib/growl.js'
- 'lib/buffered-runner.js'
- 'lib/worker.js'
- 'lib/reporters/buffered.js'
- 'lib/serializer.js'
- 'test/reporters/buffered.spec.js'
parserOptions:
ecmaVersion: 2018
env:
Expand Down
3 changes: 2 additions & 1 deletion .markdownlint.json
Expand Up @@ -6,5 +6,6 @@
"first-header-h1": false,
"first-line-h1": false,
"commands-show-output": false,
"single-h1": false
"single-h1": false,
"fenced-code-language": false
}
17 changes: 11 additions & 6 deletions .mocharc.yml
@@ -1,7 +1,12 @@
require: test/setup
ui: bdd
require: 'test/setup'
ui: 'bdd'
global:
- okGlobalA,okGlobalB
- okGlobalC
- callback*
timeout: 300
- 'okGlobalA,okGlobalB'
- 'okGlobalC'
- 'callback*'
timeout: 1000
parallel: true
watch-ignore:
- '.*'
- 'docs/_dist/**'
- 'node_modules'
13 changes: 11 additions & 2 deletions .nycrc
Expand Up @@ -4,7 +4,16 @@
"text-summary"
],
"exclude": [
"lib/browser",
"test/**/*"
"coverage/**",
"packages/*/test{,s}/**",
"**/*.d.ts",
"test{,s}/**",
"test{,-*}.{js,cjs,mjs,ts}",
"**/*{.,-}test.{js,cjs,mjs,ts}",
"**/__tests__/**",
"**/{karma,rollup,webpack}.config.js",
"**/{babel.config,.eslintrc,.mocharc}.{js,cjs}",
"lib/browser/**",
"package-scripts.js"
]
}
28 changes: 20 additions & 8 deletions .travis.yml
Expand Up @@ -38,15 +38,22 @@ jobs:
include:
- script: COVERAGE=1 npm start test.node
after_success: npm start coveralls
name: 'Latest Node.js (with coverage)'

- script: MOCHA_PARALLEL=0 npm start test.node.unit
name: 'Latest Node.js (in serial mode)'

- &node
script: npm start test.node
node_js: '12'
name: 'Node.js v12'

- <<: *node
node_js: '10'
name: 'Node.js v10'

- script: npm start test.bundle test.browser
name: 'Browser'
node_js: 12
install: npm ci # we need the native modules here
addons:
Expand All @@ -59,14 +66,15 @@ jobs:

- stage: lint
script: npm start lint
name: 'JS & Markdown'

# smoke tests use default npm.
- &smoke
stage: smoke
env: null
before_install: true
install: npm install --production

name: 'Latest Node.js'
script: ./bin/mocha --no-config --reporter spec test/sanity/sanity.spec.js
cache:
directories:
Expand All @@ -75,19 +83,23 @@ jobs:

- <<: *smoke
node_js: '12'
name: 'Node.js v12'

- <<: *smoke
node_js: '10'
name: 'Node.js v10'

- stage: precache
script: true
name: 'Prime cache'

notifications:
email: false
urls:
# for gitter mochajs/mocha
- secure: fUrHenYJs+pTuLtgBRoYyrlyfVekxaIGmLWq7bhUUqBj/7p5eCkQFn13LlPht0/4WWZOiPBcdTN7tKnz3Ho7ATUJhAchvOWDUgL5gtTvOzeCHbPuCvHz/VLK6hMoPdbLA45M864NDLotfHvyh62WgQaVw9iPc80eb+umaDPrYiU=
# for gitter mochajs/contributors
- secure: rGMGYWBaZgEa9i997jJHKzjI8WxECqLi6BqsMhvstDq9EeTeXkZFVfz4r6G3Xugsk3tFwb/pDpiYo1OK36kA5arUJTCia51u4Wn+c7lHKcpef/vXztoyucvw6/jXdVm/FQz1jztYYbqdyAOWC2BV8gYvg5F8TpK05UGCe5R0bRA=
on_success: change
on_failure: always
webhooks:
urls:
# for gitter mochajs/mocha
- secure: fUrHenYJs+pTuLtgBRoYyrlyfVekxaIGmLWq7bhUUqBj/7p5eCkQFn13LlPht0/4WWZOiPBcdTN7tKnz3Ho7ATUJhAchvOWDUgL5gtTvOzeCHbPuCvHz/VLK6hMoPdbLA45M864NDLotfHvyh62WgQaVw9iPc80eb+umaDPrYiU=
# for gitter mochajs/contributors
- secure: rGMGYWBaZgEa9i997jJHKzjI8WxECqLi6BqsMhvstDq9EeTeXkZFVfz4r6G3Xugsk3tFwb/pDpiYo1OK36kA5arUJTCia51u4Wn+c7lHKcpef/vXztoyucvw6/jXdVm/FQz1jztYYbqdyAOWC2BV8gYvg5F8TpK05UGCe5R0bRA=
on_success: change
on_failure: always
24 changes: 12 additions & 12 deletions .wallaby.js
Expand Up @@ -36,7 +36,7 @@ module.exports = () => {
// running mocha instance is not the same as mocha under test,
// running mocha is the project's source code mocha, mocha under test is instrumented version of the source code
const runningMocha = wallaby.testFramework;
runningMocha.timeout(200);
runningMocha.timeout(1000);
// to expose it/describe etc. on the mocha under test
const MochaUnderTest = require('./');
const mochaUnderTest = new MochaUnderTest();
Expand All @@ -47,17 +47,17 @@ module.exports = () => {
mochaUnderTest
);
// to make test/node-unit/color.spec.js pass, we need to run mocha in the project's folder context
const childProcess = require('child_process');
const execFile = childProcess.execFile;
childProcess.execFile = function() {
let opts = arguments[2];
if (typeof opts === 'function') {
opts = {};
Array.prototype.splice.call(arguments, 2, 0, opts);
}
opts.cwd = wallaby.localProjectDir;
return execFile.apply(this, arguments);
};
// const childProcess = require('child_process');
// const execFile = childProcess.execFile;
// childProcess.execFile = function() {
// let opts = arguments[2];
// if (typeof opts === 'function') {
// opts = {};
// Array.prototype.splice.call(arguments, 2, 0, opts);
// }
// opts.cwd = wallaby.localProjectDir;
// return execFile.apply(this, arguments);
// };
require('./test/setup');
},
debug: true
Expand Down
24 changes: 21 additions & 3 deletions bin/mocha
Expand Up @@ -108,7 +108,9 @@ if (Object.keys(nodeArgs).length) {
unparse(mochaArgs, {alias: aliases})
);

debug(`exec ${process.execPath} w/ args:`, args);
debug(
`forking child process via command: ${process.execPath} ${args.join(' ')}`
);

const proc = spawn(process.execPath, args, {
stdio: 'inherit'
Expand All @@ -126,9 +128,25 @@ if (Object.keys(nodeArgs).length) {

// terminate children.
process.on('SIGINT', () => {
proc.kill('SIGINT'); // calls runner.abort()
proc.kill('SIGTERM'); // if that didn't work, we're probably in an infinite loop, so make it die.
// XXX: a previous comment said this would abort the runner, but I can't see that it does
// anything with the default runner.
debug('main process caught SIGINT');
proc.kill('SIGINT');
// if running in parallel mode, we will have a proper SIGINT handler, so the below won't
// be needed.
if (!args.parallel || args.jobs < 2) {
// win32 does not support SIGTERM, so use next best thing.
if (require('os').platform() === 'win32') {
proc.kill('SIGKILL');
} else {
// using SIGKILL won't cleanly close the output streams, which can result
// in cut-off text or a befouled terminal.
debug('sending SIGTERM to child process');
proc.kill('SIGTERM');
}
}
});
} else {
debug('running Mocha in-process');
require('../lib/cli/cli').main(unparse(mochaArgs, {alias: aliases}));
}
18 changes: 18 additions & 0 deletions docs/css/style.css
Expand Up @@ -171,6 +171,13 @@ ul {
padding: 0 15px;
}

ul.single-column,
ul.single-column > li > ul {
column-count: 1;
margin-top: 0;
padding-right: 0;
}

ul li {
border-bottom: 1px solid #eee;
break-inside: avoid;
Expand All @@ -179,6 +186,17 @@ ul li {
padding: 5px 0;
}

ul.single-column li,
ul.single-column li > ul > li {
list-style: disc;
padding: 0;
border-bottom: 0;
}

ul.single-column li > ul > li {
list-style: circle;
}

code {
font: 14px monaco, monospace;
line-height: 1.8;
Expand Down