coverage isn’t working :(

[mikeal]

Coverage appears to be broken and I’m not sure when this happened or why, but I tried both the latest 12x an 13-rc20 versions and I’m getting the same issue.

----------|----------|----------|----------|----------|-------------------|
File      |  % Stmts | % Branch |  % Funcs |  % Lines | Uncovered Line #s |
----------|----------|----------|----------|----------|-------------------|
All files |        0 |        0 |        0 |        0 |                   |
----------|----------|----------|----------|----------|-------------------|

The .nyc-output directory has two files with empty JSON objects. Any pointers on how to debug this and figure what is going on?

[isaacs]

Well that's super weird. Can you share the project where this is happening? Are you using babel or any other source map type transpiling thing?

[mikeal]

Nope, nothing fancy, here’s the project https://github.com/mikeal/ipld-stack

I have other projects on the same machine where coverage is working fine.

Another odd thing, if I run with —nyc-arg=—all it shows the only two files that are not being imported to be tested.

[mikeal]

Ok, I figured it out, this was VERY STRANGE.

Basically, interface is a reserved word but no vm actually enforces it, but for some reason coverage parser was just choking on it silently never giving a decent error :(

I updated my code to not use interface as a variable name and it’s fine now.

[isaacs]

Whoaaaaa, that's so weird.

I'll raise this as a nyc issue. Might be a babel thing, but definitely something that should either be supported, or crash loudly, imo.

[mikeal]

Ya, it should definitely fail, but fail loudly ;) If I had to guess, I’d say that someone is assuming that if there is anything they consider a syntax error that the vm is gonna throw up anyway. Little do they know, JS has “reserved words” and “kinda-sorta-reserved words” 🤓

[isaacs]

Same as this issue, closed for staleness: istanbuljs/nyc#796

[coreyfarrell]

For what it's worth this would have failed loudly if you had 'use strict' in your source.

'use strict';
const interface = 'hello world';
console.log(interface);

Running this prints:

$ node bad-script.js 
/usr/src/npm/nyc/bad-script.js:2
const interface = 'hello world';
      ^^^^^^^^^

SyntaxError: Unexpected strict mode reserved word
    at new Script (vm.js:80:7)
    at createScript (vm.js:274:10)
    at Object.runInThisContext (vm.js:326:10)
    at Module._compile (internal/modules/cjs/loader.js:664:28)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:712:10)
    at Module.load (internal/modules/cjs/loader.js:600:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:539:12)
    at Function.Module._load (internal/modules/cjs/loader.js:531:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:754:12)
    at startup (internal/bootstrap/node.js:283:19)

I'm not against having nyc print an error when it fails to instrument a source. I'm not confident we can get a specific useful message (I don't want to spam the console, babel errors can be pretty verbose), but at minimum I can definitely say "Error instrumenting filename.js".

Another thought nyc instrument has a --exit-on-error option, maybe that should be supported for the main command as well. Maybe that option could cause a more complete error dump to stderr.