Coverage report does not include .vue files

[kouts]

Describe the bug

The coverage seems to only cover .vue files that are currently tested, giving wrong coverage reports since files without tests are not covered.
Possible related issues/PRs
#559
#518

Reproduction

https://github.com/kouts/vitest-coverage

running npm run coverage gives us the following

while it should include the uncovered src/Hello.vue component

System Info

System:
    OS: macOS 11.6.2
    CPU: (12) x64 Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
    Memory: 192.20 MB / 32.00 GB
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 16.13.2 - ~/.nvm/versions/node/v16.13.2/bin/node
    Yarn: 1.22.17 - ~/.nvm/versions/node/v16.13.2/bin/yarn
    npm: 8.1.2 - ~/.nvm/versions/node/v16.13.2/bin/npm
  Browsers:
    Chrome: 98.0.4758.80
    Firefox: 96.0.2
    Firefox Developer Edition: 97.0
    Safari: 15.2
    Safari Technology Preview: 15.4
  npmPackages:
    @vitejs/plugin-vue: ^2.1.0 => 2.1.0 
    vitest: latest => 0.2.7

Used Package Manager

npm

Validations

• Follow our Code of Conduct
• Read the Contributing Guidelines.
• Read the docs.
• Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
• The provided reproduction is a minimal reproducible example of the bug.

[Demivan]

https://github.com/bcoe/c8#checking-for-full-source-coverage-using---all

c8 by default only collects coverage for files that have been tested. You can pass all: true option to Vitest coverage config to report coverage for all files. You will probably need to adjust coverage.include option too, to not receive coverage for files that you don't want.

export default defineConfig({
  test: {
    coverage: {
      all: true
    }
  },
})

[kouts]

Thank you @Demivan I think I've already done that
https://github.com/kouts/vitest-coverage/blob/main/vitest.config.ts
is the issue reproducible on your side too?
https://github.com/kouts/vitest-coverage

[Demivan]

Your repository page returns 404. Is it private?

[kouts]

Sorry, my bad, should be public now.

[Demivan]

https://github.com/bcoe/c8/blob/ff01cd832a155494892b24c30c5a1c8f0169fd8e/lib/report.js#L197

Looks like all option in c8 only works with '.js', '.ts' and '.mjs' files. This is the issue on their side. They should be using include/exclude options to determine whether to include a file.
You can create issue in their repo or I can.

[kouts]

Thanks @Demivan, opened an issue here:
bcoe/c8#355

[brzezinskimarcin]

I experience the same issue. Does anyone have a workaround until it gets fixed?

[victorlmneves]

@kouts @brzezinskimarcin @Demivan for me it's working

This is what I have on vite.config

test: {
    /* for example, use global to avoid globals imports (describe, test, expect): */
    globals: true,
    environment: 'happy-dom',
    coverage: {
        all: true,
        exclude: [
            'node_modules/**',
            'src/utilities/**',
            '.storybook/**',
            'storybook-static/**',
            '_helpers/**',
            'dist/**',
            'public/**',
            'styleDictionary/**',
        ],
    },
},

[victorlmneves]

@kouts @brzezinskimarcin @Demivan
Sorry, but I didn't pay real attention to the results.
The **.vue files aren't included on the list 😞
Need to check Peeky base code, because it reports missing tests for vue files

[victorlmneves]

Looking to Peeky code it's using coverageOptions key instead of coverage

export default defineConfig({
  coverageOptions: {
    reportsDirectory: './coverage',
    excludeNodeModules: true,
    exclude: [
      'coverage/**',
      'packages/*/test{,s}/**',
      '**/*.d.ts',
      'cypress/**',
      'test{,s}/**',
      'test{,-*}.{js,cjs,mjs,ts,tsx,jsx}',
      '**/*{.,-}test.{js,cjs,mjs,ts,tsx,jsx}',
      '**/__tests__/**',
      '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc}.config.{js,cjs,mjs,ts}',
      '**/.{eslint,mocha}rc.{js,cjs}',
    ],
    reporter: ['text', 'html'],
    extension: ['.js', '.cjs', '.mjs', '.ts', '.tsx', '.jsx', '.vue', 'svelte'],
  },
})

I tried it but it also doesn't work as expected and he also uses C8

Unfortunately for me, Peeky it's not generating the HTML report
But on the console, I see a reference to App.vue

[Akryum]

Peeky had custom code before but it was too much work to implement istanbul and I just switched to c8 for the coverage report.

[jagoncalves14]

Looks like there is already an opened PR in c8's repository. Still yet to be approved and merged tho 🫤
bcoe/c8#357

[jagoncalves14]

I created a Stackblitz demo that reproduces this.

https://stackblitz.com/edit/vue-vitest-demo

[DannyFeliz]

C8 already merged the PR that will make this possible.

[DaveSanders]

I'm using vitest@0.12.6 and have c8@7.11.3 installed. It seems now that my coverage does show all of the .vue files that I don't have tests for in the coverage report, but almost all are showing 100%. Weirdly there are a couple files that are still at zero.

We made some major App changes yesterday so maybe that's affecting it, since I'm pretty sure yesterday we were seeing the files but they were listed at 0% covered. But... seems weird that today they would go to 100%, unless coverage just isn't being calculated correctly in .vue files?

I'm kind of lost here, so I'm curious what others are seeing.

[mahmost]

For me, the coverage of .vue files is also inaccurate. For some files it shows 100%, for others it shows low value while the file is just a static component with no script section or any conditionals/looping in template section, few cases show accurate coverage.

This is a repo that shows some success/failure cases of coverage :
https://github.com/mahmost/vitest-vue-coverage

Coverage html report on replit :
https://vitest-vue-coverage.mahmost.repl.co/src/components/HelloWorld.vue.html

[yy-dev7]

reproduces

Are you solved it? I have the same issue.

[jvanhees]

I'm having the same issue, a lot of my .vue files have 100% coverage while some have 0%, none of them have any tests associated with them, but they report a coverage of "1x". Maybe because it's being executed somewhere secretly? Or perhaps because I'm using Typescript and SFCs?

The Istanbul provider is doing a lot better and seems to not have the same issue.

[jvanhees]

So after a bit of researching, it seems that the template is not taken into account when calculating the coverage. This means that only the script is counted as coverage and if your component only has logic in the template (with v-if), the SFC would get 100% coverage although it's clearly not the case. The Istanbul coverage reporter does seem to take the templates into account.

[AriPerkkio]

@kouts is this issue still valid? The reproduction case seems to work as expected:

 ✓ test/async.test.ts (1)

 Test Files  1 passed (1)
      Tests  1 passed (1)
   Start at  14:30:29
   Duration  1.25s (transform 719ms, setup 0ms, collect 146ms, tests 9ms)

 % Coverage report from c8
------------------|---------|----------|---------|---------|-------------------
File              | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
------------------|---------|----------|---------|---------|-------------------
All files         |    57.5 |    66.66 |       0 |    57.5 |                   
 AsyncComp.vue    |     100 |      100 |     100 |     100 |                   
 AsyncWrapper.vue |     100 |      100 |     100 |     100 |                   
 Hello.vue        |       0 |        0 |       0 |       0 | 1-17              
------------------|---------|----------|---------|---------|-------------------

For me, the coverage of .vue files is also inaccurate. For some files it shows 100%,

@mahmost your reproduction case seems to work fine with changes of on-going PR #2591. It's having the same issue as #2539.

------------------------|---------|----------|---------|---------|----------------------
File                    | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s    
------------------------|---------|----------|---------|---------|----------------------
All files               |   29.29 |     6.66 |       0 |   29.29 |                      
 src                    |       0 |        0 |       0 |       0 |                      
  App.vue               |       0 |        0 |       0 |       0 | 1-81                 
  main.js               |       0 |        0 |       0 |       0 | 1-4                  
 src/components         |   41.96 |     12.5 |       0 |   41.96 |                      
  HelloWorld.vue        |   94.81 |    16.66 |       0 |   94.81 | 20,38,43-44,56-57,59 
  TheWelcome.vue        |       0 |        0 |       0 |       0 | 1-84                 
  WelcomeItem.vue       |       0 |        0 |       0 |       0 | 1-86                 
 src/components/icons   |       0 |        0 |       0 |       0 |                      
  IconCommunity.vue     |       0 |        0 |       0 |       0 | 1-7                  
  IconDocumentation.vue |       0 |        0 |       0 |       0 | 1-7                  
  IconEcosystem.vue     |       0 |        0 |       0 |       0 | 1-7                  
  IconSupport.vue       |       0 |        0 |       0 |       0 | 1-7                  
  IconTooling.vue       |       0 |        0 |       0 |       0 | 1-19                 
------------------------|---------|----------|---------|---------|----------------------

Maybe it's time to close this issue and open new ones if there are still existing problems.

[tw-eugeneoei]

Hi @AriPerkkio, unfortunately, i am still experiencing this issue and have opened a new issue here. i have tried using both c8 and istanbul and they both yield the same result.