Feature: Prepare for jest-circus

[thernstig]

Next version of Jest will default to using jest-circus as their test runner. Already now many projects change the default runner to instead use jest-circus due to it's superioirity, see https://github.com/facebook/jest/tree/master/packages/jest-circus

jest-circus supports new events fired that can be listened to in test environments via the handleTestEvent function, see https://jestjs.io/docs/en/configuration#testenvironment-string

As part of this, another PR is ongoing to make them async, see jestjs/jest#9397

With all this in mind, I have two things that would be cool if jest-playwright is a bit proactive to support:

1. Support for jest-circus. Maybe this already works?
2. Support for async handleTestEvent. This could be done easily by a user of jest-playwright by extending the environment, as such https://github.com/smooth-code/jest-puppeteer#extend-puppeteerenvironment. I am thus wondering if this should be the recommended way also in jest-playright (in the readme) or via some other means to catch these events.

note: We want to use this ourselves to take screenshots upon failures.

[mmarkelov]

@thernstig I just made some research on it.

So you can extend Environment like in jest-puppeteer:

jest.config.js:

module.exports = {
    verbose: true,
    testEnvironment: "./custom-preset.js",
    testRunner: 'jest-circus/runner',
};

custom-preset.js:

const PuppeteerEnvironment = require('jest-playwright-preset/lib/PlaywrightEnvironment').default

class CustomEnvironment extends PuppeteerEnvironment {
    async setup() {
        await super.setup()
    }
    async handleTestEvent(event, state) {
        if (event.name === 'test_fn_failure') {
            const testName = state.currentlyRunningTest.name;

            // Take a screenshot at the point of failure
            await this.global.page.screenshot({ path: `${testName}.png` });
        }
    }

    async teardown() {
        // await this.global.page.close()
    }
}

module.exports = CustomEnvironment

I just need to find out why there is some problems with it right now

[thernstig]

@mmarkelov Yes that is what I meant with 2) in my original post. Should that maybe be added to the README like it is in jest-puppeteer?

Regarding 1), when you get 2) working, is there anything else that needs to be fixed for jest-circus to work you think?

[celeryclub]

@thernstig I'm using this plugin with jest-circus and everything has been working well for me. Here's how I'm taking screenshots of test failures.

const PlaywrightEnvironment = require('jest-playwright-preset/lib/PlaywrightEnvironment').default;

class CustomEnvironment extends PlaywrightEnvironment {
  async handleTestEvent(event) {
    if (event.name === 'test_done' && event.test.errors.length > 0) {
      const parentName = event.test.parent.name.replace(/\W/g, '-');
      const specName = event.test.name.replace(/\W/g, '-');

      await this.global.page.screenshot({ path: `screenshots/${parentName}_${specName}.png` });
    }
  }
}

module.exports = CustomEnvironment;

[thernstig]

@celeryclub Thanks! Yes, async handleTestEvent just landed in yesterday in Jest 25.3.0 https://github.com/facebook/jest/releases/tag/v25.3.0 (MR jestjs/jest#9397)

Will try this out today!

[mmarkelov]

@thernstig yeah! I just checked #85 (comment) and seems like it's work fine. I'll add some info about this approach in documentation

[thernstig]

I have now also verified everything works nicely with jest-circus. Should we close this issue, or keep it open till maybe some info has been added in README.md about extending PlaywrightEnvironment?

Here is an example of a suggestion that could go into the README:

// CustomEnvironment.js
const PlaywrightEnvironment = require('jest-playwright-preset/lib/PlaywrightEnvironment')
  .default;

// We extend the PlaywrightEnvironment to be able to handle test events,
// such as automatically taking screenshots during test failures.
class CustomEnvironment extends PlaywrightEnvironment {
  async setup() {
    await super.setup();
    // Your setup
  }

  async teardown() {
    // Your teardown
    await super.teardown();
  }

  async handleTestEvent(event) {
    if (event.name === 'test_done' && event.test.errors.length > 0) {
      const parentName = event.test.parent.name.replace(/\W/g, '-');
      const specName = event.test.name.replace(/\W/g, '-');

      await this.global.page.screenshot({
        path: `screenshots/${parentName}_${specName}.png`,
      });
    }
  }
}

module.exports = CustomEnvironment;

// jest.config.js
testEnvironment: './CustomEnvironment.js',

It'd also be good to explicitly mention that handleTestEvent only works if the runner is set to use jest-circus i.e like this testRunner: 'jest-circus/runner',.