Jest 25's change in behaviour of collectCoverageFrom option

[EvHaus]

💥 Regression Report

After upgrading from Jest 24 to Jest 25 we are seeing more files in our coverage reports that we used to have. It seems like the behaviour of the collectCoverageFrom glob patterns has changed.

Last working version

Worked up to version: 24.9.0

Stopped working in version: 25.1.0

To Reproduce

Our folder structure is the following:

• client/
  • extension/
    • chrome/
      • js/
        • __tests__/
          • some_file.js
        • module.js

Our jest.config.js has the following option:

collectCoverageFrom: [
    '!**/__mocks__/**',
    '!**/__tests__/**',
    'extension/**'
  ],

Expected behavior

The expectation is that coverage is collected from module.js but NOT from some_file.js. In Jest 24 this is the case, but in Jest 25 coverage is being collected from some_file.js even though we've set '!**/__tests__/**',.

To work around this, we had to explicitly add '!extension/chrome/js/__tests__/**' to the list of directories and that made Jest 25 ignore it.

Link to repl or repo (highly encouraged)

Hopefully the details above are enough. If not, I can help setup a test repo to reproduce.

Run npx envinfo --preset jest

Paste the results here:

  System:
    OS: macOS 10.15.2
    CPU: (8) x64 Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz
  Binaries:
    Node: 10.17.0 - ~/.nvm/versions/node/v10.17.0/bin/node
    Yarn: 1.21.1 - /usr/local/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v10.17.0/bin/npm
  npmPackages:
    jest: 25.1.0 

[joelpalmer]

We ran into this as well. Jest 25 moved up to micromatch 4 which is where the problem seems to be: micromatch/micromatch#179.

This is the workaround we are using for now: micromatch/micromatch#179 (comment)

[SimenB]

Oh, this is very interesting! Thanks for that link @joelpalmer. Could you share which pattern you changed and to what?

[joelpalmer]

Sure @SimenB - that workaround comment is actually from us: micromatch/micromatch#179 (comment)

But, to add a little more context, in jest.config.js:

  collectCoverage: true,
  collectCoverageFrom: [
    '**/*.js',
-    '!(coverage|config|scripts|tests)/**'
+    '!**/(coverage|config|scripts|tests)/**'
  ],
  coverageReporters: ['html', 'lcovonly', 'text-summary'],
  testEnvironment: 'node',

[jonschlinkert]

@joelpalmer thanks for the detail. I maintain micromatch, I'm looking into this.

[juanlet]

I can confirm that joelpalmer solution works fine. I went through the same change of behavior but that did it.

[mkai]

Upgrading to Jest 25.2.4 has resolved this issue for us! We didn't have to make any changes to our collectCoverageFrom setting.

[EvHaus]

For me, the original issue is still occurring in 25.2.4. Even though I specified:

'!**/__tests__/**',

to be ignored, I'm still seeing directories from extension/chrome/js/__tests__/ inside my coverage report.

[SimenB]

I don't think we've changed anything to address this. I don't really think we can work around it either, without downgrading micromatch which we really don't wanna do (it'd be a breaking change as well)

[jonschlinkert]

Sorry for the radio silence on this. I’ll try to get this resolved in micromatch ASAP.

…

Sent from my iPhone

On Mar 31, 2020, at 5:44 PM, Simen Bekkhus ***@***.***> wrote: I don't think we've changed anything to address this. I don't really think we can work around it either, without downgrading micromatch which we really don't wanna do (it'd be a breaking change as well) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[csvan]

@jonschlinkert are there any news on this issue on the micromatch side?

[kibertoad]

@csvan This should have been fixed in micromatch 4.0.4, can someone confirm if issue goes away after the upgrade?

[atsikov]

micromatch/micromatch#179 (comment)
micromatch@4.0.4 was released today. According to changes, they bumped picomatch to fix negated globs issue, however it didn't work for me.

With such settings

    "collectCoverageFrom": [
      "src/**/*.{js,jsx,ts,tsx}",
      "!src/theme/**",
      "!src/**/*.generated.{ts,tsx}",
      "!src/gql/__generated__/**"
    ],

I still see src/theme/index.ts, src/gql/__generated__/generated.introspection-result.ts and some .generated.ts files in a coverage report. This workaround #9464 (comment) didn't help as well.
coveragePathIgnorePatterns works as expected.

Could anyone please check if new micromatch will work for them?

[SimenB]

For people seeing improvements from rolling back micromatch, please see #11293

[danez]

To sum it up.

• The issue @joelpalmer reported was fixed in fix(util): Replace micromatch with picomatch to fix issues with negated globs #11287 (not released yet). Basically negated patterns of the form !(<anything>)<something> were broken.
• I haven't yet looked at the issue that @atsikov posted.
• The changed behaviour for the example ['!**/__mocks__/**', '!**/__tests__/**', 'extension/**'] is now documented in docs: Add Note about order of glob patterns and fix link to micromatch #11341. The breaking change in jest 25 was that the order of globs now matters and has an effect.
  In the given example:

collectCoverageFrom: [
    '!**/__mocks__/**',
    '!**/__tests__/**',
    'extension/**'
  ],

What this basically means is

Ignore everything in __mocks__ and __test__ but include all files in extension

Because the order matters the last glob is simply overwriting the two negations, so in the end the config simply means

Include everything in extension

To fix the behaviour ensure that the order of globs does make sense for your case. In the example it would mean moving the negated patterns to the end of the array:

collectCoverageFrom: [
    'extension/**',
    '!**/__mocks__/**',
    '!**/__tests__/**',
  ],

[SimenB]

Thanks for clarifying here @danez! As you say, this change of behaviour is correct (albeit somewhat unexpected as it is not mentioned in Micromatch's changelog).

So I think we can close this 🙂

---

• fixed in fix(util): Replace micromatch with picomatch to fix issues with negated globs #11287 (not released yet)

It's available in jest@next FWIW

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.