Async test fails with timeout instead of assertion error

[gurdiga]

This test:

    it('returns the correct value', function(done) {
      var returnValue = 5;

      aPromise.then(function() {
        expect(returnValue).to.equal(42);
        done();
      });

    });

This test fails with timeout of 2000ms exceeded instead of assertion error. I guess that’s because expect() call throws an error, and the done() never gets executed, and I’m wondering if there is a better way to test this kind of code.

[NickHeiner]

Does aPromise ever resolve? If not, there's not much choice but to throw a timeout.

[gurdiga]

@NickHeiner Yes, it resolves; and then expect() finds returnValue not being equal(42) and throws.

[hallas]

@gurdiga how do you know it throws if you get a timeout and not an assertion error?

[gurdiga]

@hallas @NickHeiner Here is the running thing: http://jsfiddle.net/gurdiga/p9vmj/.

[hallas]

@gurdiga it seems to me that your promise has its own error catching. Try to add a .catch(done) to your promise and I think it'll work as intended.

[gurdiga]

@hallas Wow: that was the answer! :) aPromise.finally() seems to be the perfect fit to put done into: it also needs to be called when the promise is resolved. ;)

Thank you!

[gurdiga]

I feel stupid.

I think I finally got it: when anything throws in any of the promise’s handler functions, be it the one passed to .then(), to .catch() or to .finally(), the error is handled by the promise library. This way, the test runner never sees the real error, the done() is never called (because something before it threw an assertion error), and so the time out error is all you get from the test runner.

To get out of the promise, I use setTimeout():

    it('returns the correct value', function(done) {
      var returnValue = 5;

      aPromise.then(function() {
        setTimeout(function() {
          expect(returnValue).to.equal(42);
          done();
        });
      });
    });

I found this is the only way to get proper error messages and test runner behavior.

With done passed to .catch() or .finally() the test is considered passed in any case, so if there are assertion errors, you’ll never see them.

[DSheffield]

I ran into a similar problem, and eventually realized you shouldn't use done when testing asynchronous functions with promises, instead, just return the promise. So you should be able to do this without the timeout, e.g.:

it('returns the correct value', function() {
    var returnValue = 5;

    return aPromise.then(function() {
        expect(returnValue).to.equal(42);
    });
});

[stevenvachon]

I think that this issue still exists. I'm getting a timeout issue that cannot be resolved by returning a promise because my module does not use promises.

it("works", function(done) {
    new Something()
    .on("eventA", function(result) {
        expect(result).to.be.true;
    })
    .on("eventB", function(result) {
        expect(result).to.be.false;
        done();
    });
});

• Wrapping the instance in a Promise seems excessive.
• Wrapping each assertion in a try/catch also seems excessive and, more importantly, results in Error: done() called multiple times.

Ideas:
http://staxmanade.com/2015/11/testing-asyncronous-code-with-mochajs-and-es7-async-await/

[ScottFreeCode]

As regards the blog post's recommendations, I don't know about async function error handling, but for plain promises the try-catch recommendation is weird: the original promise attempt with no try-catch was nearly correct, it only needed to use .catch(done) instead of using the second parameter to then as done. (Granted, since Mocha has direct support for promises, you can also just return the promise, but for what it's worth...) The problem in the initial promise example wasn't absence of try-catch, it was that the second handler to then is not called with exceptions thrown by the first handler, whereas a following catch is; I don't know what the rationale was for promises to be designed that way, but it's how promises are. Additionally, if there had been multiple thens for some reason, only a single final .catch(done) would be needed, which is a point of superiority over try-catch inside the handlers (on top of the fact that .catch(done) is less boilerplatey to begin with).

As for your API:

1. Are you sure both events are being called, and in the right order? If they were not, how would your test turn that into a normal failure?
2. What normally happens to exceptions that are thrown from your event handlers? If they don't propagate out the way they do in synchronous code and instead are meant to be listened for on the API (e.g. with .on("error", function(error) {...})), then they'll never reach Mocha unless you listen for them and have the listener call done with the error (or just use done with the listener if the listener is passed the error as its first parameter, e.g. .on("error", done). Presumably that would also only need to be written once per test, rather than once per event handler, like .catch(done) in a promise.

[stevenvachon]

1. Yes, and I use an "end"/"drain" event to check if booleans in the other events were set.
2. The timeout happens. I'm trying to find a lean and clean alternative.

[ScottFreeCode]

Sorry, I still don't know how your API is supposed to report errors.

[stevenvachon]

Yeah, so far I've just been relying on the timeouts for when somethings fails, then manually digging to find out how/why. Fortunately, things rarely break, but that's not an excuse for the lack of a better design (on my part, mostly).

[lsphillips]

@stevenvachon: Forgive me in advance, but I don't see an immediate issue with your example. The assertions made in your event listeners should be handled by Mocha via the uncaughtException mapping (unless the event emitter implementation is catching listener errors and emitting an error event or something, which then is still easy to solve).

Now if your implementation under the hood is using Promises, but emits events rather than exposes the Promise, your assertions will indeed be "eaten". The way I get around this problem is to use unhandledRejection.

I usually put this in a setup script that is run before my tests:

process.on('unhandledRejection', function (reason)
{
    throw reason;
});

Note: This may need some extra elbow grease to work in browsers.

I hope to to see Mocha support this like it does uncaughtException as this is a common use case; just because I use Promises doesn't mean I want to return them to the caller!

[monolithed]

Having the same issue with mocha@3.1.2

    it('Convert files into base64', (resolve) => {
        let files = Promise.all(promises);

        return files
            .then(([actual, expected]) => {
                assert.equal(actual, expected, 'Files not are equal');
                resolve();
            })
            .catch(error => resolve);
    });

   Error: timeout of 2000ms exceeded. Ensure the done() callback is being called in this test.

[ScottFreeCode]

The .catch is wrong. error => resolve is equivalent to function(error) { return resolve }, which means resolve won't be called and the error is ignored. What you want is to call resolve with the error, which would be error => resolve(error). Of course, passing a callback function X that simply calls function Y with the same arguments that X is called with is equivalent to just passing Y as the callback, so even .catch(error => resolve(error)) could be simplified to .catch(resolve). (You would only need to not pass resolve directly if you were passing it to then and, therefore, needed to avoid passing then's result parameter to resolve to prevent it being treated as an error: then(()=>resolve()) rather than just .then(resolve); but since you are using the then callback for the assertions, this doesn't come up.)

(Also, idiomatically, resolve here should probably be named something along the lines of done, since it handles both success and failure and judges which is which based on whether it was called with an argument or not. Hence the name in Mocha's error message. But that may be a moot point; read on.)

However, in this case you can simplify it even further by just returning the promise and not using the test done parameter at all, since Mocha will wait for the promise to succeed or fail as indicating test success or failure (provided there's no done parameter to the test function; behavior in the event both are used is still being hashed out):

it('Convert files into base64', () => {
    let files = Promise.all(promises);
    return files
        .then(([actual, expected]) => {
            assert.equal(actual, expected, 'Files not are equal');
        })
});

[stevenvachon]

@lsphillips that works for me. Thanks!! I hope to see mocha support this by default too. I just created #2640.

[mattstabeler]

Took me a while to work this out! Based on answers above, these are the two options:

npm install --save mocha expect.js q
./node_modules/mocha/bin/mocha test.spec.js

// test.spec.js

var $q = require('q');
var expect = require('expect.js');

describe('tests with done', function(){
    it('returns the correct value from promise', function(done) {
      var returnValue = 5;
      var def = $q.defer();
      def.promise.then((val) => {
        expect(val).to.equal(42);
        done();
      }).catch(done);
      def.resolve(returnValue)
    });
})

describe('tests returning promises', function(){
    it('returns the correct value from promise', function() {
      var returnValue = 5;
      var def = $q.defer();
      def.resolve(returnValue)
      return def.promise.then((val) => {
        expect(val).to.equal(42);
      });
    });
})

  tests with done
    1) returns the correct value from promise

  tests returning promises
    2) returns the correct value from promise


  0 passing (15ms)
  2 failing

  1) tests with done returns the correct value from promise:
     Error: expected 5 to equal 42
      at Assertion.assert (node_modules/expect.js/index.js:96:13)
      at Assertion.be.Assertion.equal (node_modules/expect.js/index.js:216:10)
      at def.promise.then (tests/test.spec.js:9:24)
      at _fulfilled (node_modules/q/q.js:854:54)
      at self.promiseDispatch.done (node_modules/q/q.js:883:30)
      at Promise.promise.promiseDispatch (node_modules/q/q.js:816:13)
      at node_modules/q/q.js:570:49
      at runSingle (node_modules/q/q.js:137:13)
      at flush (node_modules/q/q.js:125:13)
      at _combinedTickCallback (internal/process/next_tick.js:67:7)
      at process._tickCallback (internal/process/next_tick.js:98:9)

  2) tests returning promises returns the correct value from promise:
     Error: expected 5 to equal 42
      at Assertion.assert (node_modules/expect.js/index.js:96:13)
      at Assertion.be.Assertion.equal (node_modules/expect.js/index.js:216:10)
      at def.promise.then (tests/test.spec.js:22:24)
      at _fulfilled (node_modules/q/q.js:854:54)
      at self.promiseDispatch.done (node_modules/q/q.js:883:30)
      at Promise.promise.promiseDispatch (node_modules/q/q.js:816:13)
      at node_modules/q/q.js:570:49
      at runSingle (node_modules/q/q.js:137:13)
      at flush (node_modules/q/q.js:125:13)
      at _combinedTickCallback (internal/process/next_tick.js:67:7)
      at process._tickCallback (internal/process/next_tick.js:98:9)

[RuchitaGarde]

@gurdiga Thanks for the setTimeout() idea! I had a similar problem, but now I can get proper error messages atleast!

[itumoraes]

In my scenario I used Nightmare to end2end tests. The solution for me was use .catch(done). You can call done(error) inside other catch callback like the example below.

describe('Clicking in any bad reputation tag', () => {
    it('open the bad reputation modal', (done) => {
      nightmare
        .select('#per-page', '50')
        .waitForAjax()
        .click('[data-reputation="bad"]')
        .evaluate(function() {
          return document.querySelector('.vue-modal .ls-modal-title').innerText
        })
        .then(function(title) {
          title.should.equal('Sua segmentação teve uma avaliação ruim!')
          done()
        })
        .catch((error) => {
          screenshot(nightmare)
          done(error)
        })
    })
  })

[vkarpov15]

@itumoraes that works, but you can do this instead:

describe('Clicking in any bad reputation tag', () => {
    it('open the bad reputation modal', () => {
      return nightmare
        .select('#per-page', '50')
        .waitForAjax()
        .click('[data-reputation="bad"]')
        .evaluate(function() {
          return document.querySelector('.vue-modal .ls-modal-title').innerText
        })
        .then(function(title) {
          title.should.equal('Sua segmentação teve uma avaliação ruim!')
        })
    })
  })

You don't need to call done() if you return a promise. See my blog post Using Async/Await with Mocha, Express, and Mongoose

[selvakumar1994]

i used the below script but i got the same Timeout exceed error.

Myscript :

describe("getBillingDetail", async function (){
this.timeout(55000);
it.only("check given valid Jobname",async function (done){
this.timeout(55000);
var result = await url.getBillingDetail('12254785565647858');
console.log(result);
assert.equal(result,true);
});
});

Error: Timeout of 55000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves.

[Munter]

Stop spelling the same thing in multiple closed issues. Don't pass a done callback into async functions. Read the documentation on async tests

[selvakumar1994]

@Munter i removed the done callback,but these error is occur again

[outsideris]

It looks like that your promise never resovled.

[hexmarkrecords]

@stevenvachon: Forgive me in advance, but I don't see an immediate issue with your example. The assertions made in your event listeners should be handled by Mocha via the uncaughtException mapping (unless the event emitter implementation is catching listener errors and emitting an error event or something, which then is still easy to solve).

Now if your implementation under the hood is using Promises, but emits events rather than exposes the Promise, your assertions will indeed be "eaten". The way I get around this problem is to use unhandledRejection.

I usually put this in a setup script that is run before my tests:

process.on('unhandledRejection', function (reason)
{
    throw reason;
});

Note: This may need some extra elbow grease to work in browsers.

I hope to to see Mocha support this like it does uncaughtException as this is a common use case; just because I use Promises doesn't mean I want to return them to the caller!

this is a pretty solution. thanks

[enjikaka]

I just had issues with this as well;

it('test', async () => {
      await foreignPromise();
      const result = true;
      console.log(result);
      expect(result).to.equal(true);
      console.log('postexpect');
  });

would timeout with Error: Timeout of 2000ms exceeded. For async tests and hooks, ensure "done()" is called; if returning a Promise, ensure it resolves and print true, postexpect. Looks like mocha can't understand this.

Wrapping it in a normal function and returning the async function/promise instead worked:

it('test', () => {
   return async () => {
      await foreignPromise();
      const result = true;
      console.log(result);
      expect(result).to.equal(true);
      console.log('postexpect');
  }
});

or

it('test', () => async () => {
      await foreignPromise();
      const result = true;
      console.log(result);
      expect(result).to.equal(true);
      console.log('postexpect');
  });

[ticapix]

@stevenvachon: Forgive me in advance, but I don't see an immediate issue with your example. The assertions made in your event listeners should be handled by Mocha via the uncaughtException mapping (unless the event emitter implementation is catching listener errors and emitting an error event or something, which then is still easy to solve).

Now if your implementation under the hood is using Promises, but emits events rather than exposes the Promise, your assertions will indeed be "eaten". The way I get around this problem is to use unhandledRejection.

I usually put this in a setup script that is run before my tests:

process.on('unhandledRejection', function (reason)
{
    throw reason;
});

Note: This may need some extra elbow grease to work in browsers.

I hope to to see Mocha support this like it does uncaughtException as this is a common use case; just because I use Promises doesn't mean I want to return them to the caller!

Perfect for async/await and express !
Thank you