Skip to content

Comparison with mocha.parallel

Guilherme Alan Ritter edited this page Aug 21, 2018 · 13 revisions

mocha.parallel is a tool which allows you to run mocha tests in parallel. There's much more difference between mocha-parallel-tests and mocha.parallel: internally mocha.parallel is more like a tool which allows you to run your tests in parallel, while mocha-parallel-tests is more like a wrapper on top of mocha which is running all tests in subprocesses.

API

This is one of the primary differences between these two. mocha.parallel requires you to use its API which resembles mocha to run tests while mocha-parallel-tests is just a parallel tests runner which ideally doesn't require you to change your tests written for mocha: you can just run them with mocha-parallel-tests.

When you're using mocha.parallel you write tests in a bit different way than with mocha although the API looks quite similar:

var parallel = require('mocha.parallel');

parallel('delays', function() {
  it('test1', function(done) {
    setTimeout(done, 500);
  });

  it('test2', function(done) {
    setTimeout(done, 500);
  });

  it('test3', function() {
    return Promise.delay(500);
  });
});

The pros are you can see what's happening in "parallel". The cons: your test is specific to mocha.parallel only and its limitations because mocha.parallel has nothing in common with mocha and is not using its API even although mocha is defined as its peerDependency. Actually, mocha.parallel API gives you a minimal subset of mocha APIs.

With mocha-parallel-tests your tests are ordinary mocha tests, and you don't need to import something specific. The only thing which you should keep in mind is that your test spec files are executed at the same time each in its separate process. Also mocha-parallel-tests is heavily using mocha APIs and is extending it, not re-implementing.

mocha-parallel-tests: ★★★
mocha.parallel: ★☆☆

Running tests

mocha-parallel-tests runs each test spec file in its separate process. This means that for each file mocha-parallel-tests spawns a new process with child_process.fork() and talks to the subprocess through the IPC channel. The test spec file processing is completely isolated from other tests which can be either a great advantage or a sudden discomfort for you depending on how you design tests. Also, bear in mind that creating node.js new processes is an expensive operation and because of that mocha-parallel-tests limits --max-parallel by the number of your logical CPU cores although you can change this.

mocha.parallel runs all tests in the same process. It doesn't offer true parallelism using multiple processes, which is mostly okay for asynchronous tests where spawning multiple processes is not necessary because your main process can execute some code while waiting for I/O data. However, CPU-heavy tests can't be run using this way because CPU-heavy tasks block the CPU and you can achieve parallel execution in this case.

mocha-parallel-tests: ★★☆
mocha.parallel: ★★☆

Mocha API compatibility

mocha.parallel is re-implementing mocha APIs from scratch which means that you can consider writing your code like you're using mocha, but you unfortunately many mocha methods are not implemented. There's also no Mocha.Runner, Mocha.Reporter and other APIs available: only top-level APIs like parallel() and parallel.only() are supported.

mocha-parallel-tests is more like a wrapper on top of mocha. It introduces a couple of its specific methods (mostly to communicate with subprocesses and set --max-parallel value but from end-user perspective it's just a mocha API because it extends mocha.

mocha-parallel-tests: ★★★
mocha.parallel: ★☆☆

Mocha CLI compatibility

mocha.parallel doesn't offer you a CLI to run tests: you're supposed to write a "main" file which invokes the parallel() function which in turn runs all your tests. This "main" file is then executed with node like this: node main.spec.js. This is not very flexible although you can debug your tests with native node debugger.

Another way of getting mocha goodness while using mocha.parallel is writing your test in mocha.parallel style while running it with mocha. In this case it function comes from mocha while you can use mocha.parallel's parallel function.

mocha-parallel-tests has a CLI which is more or less compatible with mocha CLI. Some of the options are probably not supported but this can be fixed easily. The problem why some of them are not supported is because mocha's CLI is a monolith file and mocha-parallel-tests has to re-implement this from scratch.

mocha-parallel-tests: ★★☆
mocha.parallel: ★★☆

Performance

As you can see it's quite an incorrect way to compare the performance of these two libraries because mocha.parallel is using one process, specific API to run tests and doesn't provide "true" parallelism, while mocha-parallel-tests forks subprocesses and is a mocha extension. However here're some ideas you can take:

  • asynchronous tests can benefit from using one process with mocha.parallel: it doesn't create a subprocess for each parallel test, and this is fine for I/O-heavy tests. Although you're limited regarding supported reporters (i.e. you can't choose whatever mocha or even custom reporter for mocha that you like most)
  • running I/O-heavy tests with mocha-parallel-tests can be a good thing to try if you want to keep your mocha tests and make them a bit faster. However, the price that you pay here is the RAM which is allocated for each subprocess.
  • running synchronous CPU-intensive tests is a task for mocha-parallel-tests because it runs each CPU-intensive task in a separate subprocess. However, you can see the benefit here only on multi-core CPU machines: single-core CPU doesn't give you the advantage here.