Nyc processing files not covered by include when all is enabled.

[AGrzes]

Link to bug demonstration repository.

https://github.com/AGrzes/nyc-include-issue

Expected Behavior

When include is specified in nyc configuration it instrument only files matching include.

Observed Behavior

Nyc processes all files then discards those that not match include. In conjunction with ts-node it leads to compilation of unnecessary files and unexpected failures.

For example in demonstration repository

$ npm test

> nyc-issue@1.0.0 test /home/agrzes/git/nyc-issue
> nyc mocha test/**/*.{ts,tsx}


/home/agrzes/git/nyc-issue/node_modules/ts-node/src/index.ts:261
    return new TSError(diagnosticText, diagnosticCodes)
           ^
TSError: ⨯ Unable to compile TypeScript:
bad/bad.ts(1,6): error TS1005: ';' expected.
bad/bad.ts(1,9): error TS1005: ';' expected.
bad/bad.ts(1,13): error TS1005: ';' expected.
bad/bad.ts(1,15): error TS1005: ';' expected.
bad/bad.ts(1,23): error TS1005: ';' expected.
bad/bad.ts(1,26): error TS1005: ';' expected.
bad/bad.ts(1,1): error TS2304: Cannot find name 'This'.
bad/bad.ts(1,6): error TS2304: Cannot find name 'is'.
bad/bad.ts(1,9): error TS2304: Cannot find name 'not'.
bad/bad.ts(1,13): error TS2304: Cannot find name 'a'.
bad/bad.ts(1,15): error TS2304: Cannot find name 'correct'.
bad/bad.ts(1,23): error TS2304: Cannot find name 'ts'.
bad/bad.ts(1,26): error TS2552: Cannot find name 'file'. Did you mean 'File'?

    at createTSError (/home/agrzes/git/nyc-issue/node_modules/ts-node/src/index.ts:261:12)
    at getOutput (/home/agrzes/git/nyc-issue/node_modules/ts-node/src/index.ts:367:40)
    at Object.compile (/home/agrzes/git/nyc-issue/node_modules/ts-node/src/index.ts:558:11)
    at Object.m._compile (/home/agrzes/git/nyc-issue/node_modules/ts-node/src/index.ts:439:43)
    at Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Object.require.extensions.(anonymous function) [as .ts] (/home/agrzes/git/nyc-issue/node_modules/ts-node/src/index.ts:442:12)
    at NYC._readTranspiledSource (/home/agrzes/git/nyc-issue/node_modules/nyc/index.js:165:26)
    at NYC.addFile (/home/agrzes/git/nyc-issue/node_modules/nyc/index.js:149:21)
    at /home/agrzes/git/nyc-issue/node_modules/nyc/index.js:181:11
    at /home/agrzes/git/nyc-issue/node_modules/nyc/index.js:250:5
npm ERR! Test failed.  See above for more details.

Forensic Information

Operating System: Linux agrzes-desktop 4.15.0-33-generic #36~16.04.1-Ubuntu SMP Wed Aug 15 17:21:05 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Environment Information: https://gist.github.com/AGrzes/a3e87bbb58f592b499f005bdd2ad2a21

[coreyfarrell]

I have just confirmed this issue. From what I can tell this is an issue only when all is enabled. @AGrzes this is a bug on it's own but please let me know if you find a way to reproduce the issue with all disabled.

[AGrzes]

Looking at the source (of nyc) it appears that the code path causing error is only active when all flag is enabled.

[peterjgrainger]

I can take a look at this if thats OK?

Tested without all flag and everything works fine, seems to only occur on when the all flag is set.

The docs state Use the --all flag to include files that have not been required in your tests.

What is the expected outcome? That the files are included in the report but aren't compiled?

[AGrzes]

@peterjgrainger - My understanding is the --all flag is used to have files that not yet have any tests (and are not indirectly referenced from tested files) to show up in report. It gives more accurate total project coverage.

[peterjgrainger]

@AGrzes ah didn't realize you fixed this already, woops

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[robtarr]

I am having issues with this right now. I get inconsistent output, some files get completely dropped with the all flag enabled. And without it, my coverage report does not reflect the fact that some files are flat out missing tests.

[JaKXz]

@robtarr this fix will be released soon! Sorry about that

[robtarr]

Any chance there's a branch I could try out? Or a rough timeline?

[JaKXz]

You can try nyc#master of course

[robtarr]

After updating to 13.2.0, I'm still getting inconsistent data back on subsequent runs.

> nyc npm test

=============================== Coverage summary ===============================
Statements   : 74.01% ( 12122/16379 )
Branches     : 50.67% ( 3740/7381 )
Functions    : 60.91% ( 1736/2850 )
Lines        : 72.28% ( 10007/13844 )
================================================================================

followed immediately by

nyc npm test

=============================== Coverage summary ===============================
Statements   : 57.68% ( 3528/6116 )
Branches     : 41.61% ( 1441/3463 )
Functions    : 48.95% ( 723/1477 )
Lines        : 56.45% ( 3169/5614 )
================================================================================

my config:

  "nyc": {
    "include": [
      "src/**/*.js",
      "lib/**/*.js",
      "packages/**/*.js",
      "server/**/*.js",
      "tasks/**/*.js"
    ],
    "exclude": [
      "**/*-spec.js",
      "**/index-story.js"
    ],
    "reporter": [
      "lcov",
      "text-summary"
    ],
    "all": true,
    "report-dir": "reports/coverage"
  }

[coreyfarrell]

@robtarr please open a new issue including a link to a repository demonstrating this issue.

[robtarr]

#981