Async test setup

[KoryNunn]

As per #234

Attempting to call additional tests after some delay causes tape to hang.

This branch resolves by the addition of the endWaitTime option.

By default, there is no wait time, and tape acts as usual.

Setting endWaitTime to a value eg 50 causes tape to wait 50ms between t 'end' events before emitting 'done'. This allows tests that get set up asynchronously after other tests have completed, to run.

endWaitTime is a suggestion, there are probably better names for it.

[yoshuawuyts]

sound reasonable to me; implementation looks good

[ghost]

This would make tests very timing sensitive, which sounds like a fragile feature. What's so bad about moving async operations inside the test:

test('whatever', function (t) {
  asyncSetup(function () {
     // ...
  })
})

or moving async setup to a pre-test:

test('setup', function (t) {
  asyncSetup(function () { t.end() })
})

test('whatever', function (t) {
  // ...
})

With similar techniques for sub-tests. Is the more about subtests in async handlers?

[KoryNunn]

I'm trying to use tape for integration testing, where each set of tests need to run in series, with a db flush in between.

The issue is, because that flush takes some time, tape decides that the tests must be finished.

I'm not a huge fan of this "just wait a bit" solution, and I'm also investigating other solutions.

I'll give the pre-test implementation a go, my only reservation would be having a test that only serves to prevent tape from finishing.

[KoryNunn]

@substack I gave the setup test implementation a go, but it doesn't work, because the results only attempt to complete after a t.end, so no tests are run after the setup test begins.

This PR (#236) does work for that case, and also solves the test added in the current PR.

[Raynos]

I think this is a bad idea.

There are better approaches.

@KoryNunn Please look at https://www.npmjs.com/package/tape-cluster

[KoryNunn]

@Raynos I agree, please see #236 and let me know what you think.

[nathanmarks]

FWIW, this is how I am handling async test setup:

I have the following helper function that runs my DB migrations on an sqlite DB.

You could also run seeds here, or any other async setup. (this is only for a single test, but with some more code you can get something setup that can handle all sorts of flushing/resetting/seeding etc)

import Test from 'blue-tape';
import Bookshelf from 'lib/db';
import KnexCleaner from 'knex-cleaner';

export function TestWithDB (name, callback, testFn = Test) {
  return testFn(name, t => {
    return Bookshelf.knex.migrate.latest()
      .then(() => callback(t))
      .then(() => Bookshelf.knex.migrate.rollback())
      .then(() => KnexCleaner.clean(Bookshelf.knex));
  });
}

TestWithDB.only = function (name, callback) {
  return TestWithDB(name, callback, Test.only);
};

Test.onFinish(() => {
  return Bookshelf.knex.destroy();
});

and to use it....

TestWithDB('Tool runner integration', t => {
  // DB is setup and can be used
  // using blue-tape so return a promise here
});

[mLuby]

Surprised there isn't an test.onStart analog to test.onFinish. I'd think @substack's legitimate concerns over timing sensitivity/fragility would apply equally to both. Since onFinish is allowed why not onStart (which I'm happy to PR 😇).

• Regarding "moving async operations inside the test", you may want the setup to occur only once. Yes there are ways around it but they're gross.
• Regarding "moving async setup to a pre-test", this breaks when .only is used.

(Also I can open this as a new issue if preferred…)