Ability to configure updating coverage report rather than overriding it

[deser]

🚀 Feature Proposal

Here is ability to optimize running tests in CI utilizing https://knapsackpro.com/. It uses query mode to run series of test separately from in other ( literally running jest series of time instead one).
But here is issue with coverage in this case - it's being overridden, see KnapsackPro/knapsack-pro-jest#12

It would be nice if Jest could add a flag like coverageUpdateExisting which will update coverage file instead of replacing it.

Motivation

To run series of test suites via series of jest calls.

[SimenB]

You can use a separate coverageDirectory for each run and merge them yourself if you want to (using e.g. nyc merge).

An issue with just updating in place is that you can get stale coverage data (say you flip a if (true) to if (false) - coverage will say both branches are covered). We could clear out coverageDirectory before starting a run, but other people might oppose that for other reasons. I don't think we wanna do anything with this here, but maybe @jeysal and/or @thymikee disagrees?

[ArturT]

@SimenB Hey, an alternative solution would be to allow generate a unique report name whenever the JestCLI is called many times. This way you would have many reports generated instead of a single report being overridden.

A similar approach was already implemented in jest-junit by one of Knapsack Pro users. There is now flag JEST_JUNIT_UNIQUE_OUTPUT_NAME=true to enable unique report names.
jest-community/jest-junit#108

In Knapsack Pro for Jest I use many times jest.runCLI which leads to overriding the report on the disk. So if there would be a way to generate a report with UUID in the name then @knapsack-pro/jest users would be able to see preserved generated reports.

[SimenB]

Isn't that the same thing as setting coverageDirectory? You'd end up with a coverage report in myCovDir1, myCovDir2 etc. There's no single "code coverage result" file you can set a filename to

[andrcuns]

I think it would work if we could simply introduce additional option for Knapsack Pro to add a unique coverageDirectory every time it invokes jest.runCLI function.

[ArturT]

@SimenB @andrcuns I could implement generating unique coverageDirectory inside of @knapsack-pro/jest when I call jest.runCLI
https://github.com/KnapsackPro/knapsack-pro-jest/blob/a51eb0a60087ac358926500df22ce3d31491775f/src/knapsack-pro-jest.ts#L40

From a big picture perspective do you think this is the right way? I can imagine someone else would like to use jest.runCLI function and call it a few times and this person would have similar problems with overridden reports. It might be good to have a solution for that on Jest side similar to what jest-junit has (flag JEST_JUNIT_UNIQUE_OUTPUT_NAME). jest-community/jest-junit#108
What do you think?

[andrcuns]

I think it would be easier to do in @knapsack-pro/jest as technically it's not so much of an issue with jest or istanbul reporters but rather how and when they are called.

[ArturT]

I've released @knapsack-pro/jest 2.7.0 and here is PR that solves problem KnapsackPro/knapsack-pro-jest#13 I use a separate coverageDirectory option with uuid in directory name (coverageDirectory/uuid) for each run as @SimenB proposed in #9492 (comment)

@andrcuns @deser Here you can find an example of how to generate code coverage for Jest with Knapsack Pro Jest in Queue Mode:
https://knapsackpro.com/faq/question/how-to-generate-code-coverage-for-jest-with-knapsack-pro-jest-in-queue-mode

@SimenB I think this issue here can be closed assuming other people with a similar problem should also use custom coverageDirectory name for each run of jest.runCLI.

[github-actions]

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 14 days.

[github-actions]

This issue was closed because it has been stalled for 7 days with no activity. Please open a new issue if the issue is still relevant, linking to this one.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.