Collect coverage for the tests themselves

[orenbenkiki]

Currently, running kcov only generates coverage for files under the src/ directory. In my case, I would like to be able to also collect coverage for the files under the tests/ directory.

Looking at Makefile.stable.toml, the command line is: kcov --include-pattern=${CARGO_MAKE_WORKING_DIRECTORY}/src/ ...; There doesn't seem to be any environment variable one could set to request that the tests/ directory would also be included in the coverage.

How about something like CARGO_MAKE_KCOV_INCLUDE_TESTS or perhaps a more general CARGO_MAKE_KCOV_INCLUDE_PATTERN?

[sagiegurari]

will probably move that pattern to the config to enable overriding as I feel its not correct to show coverage value to include non library code (having unit test in is bad enough but hard to split out).

[orenbenkiki]

For me, the coverage is less about reaching a specific high percentage; it is about visibility of which lines are covered. Hence https://crates.io/crates/cargo-coverage-annotations which ensures that any lines that are not executed are explicitly marked with a // NOT TESTED comment (or variants).

I found this helps in figuring out what's happening in the code; on multiple occasions I was surprised that certain code paths were not taken, even though I expected them to. Yes, I know, pure TDD avoids this, but even then refactoring loses the tight connection between coverage and tests.

I would like the same certainty that all my code paths in the test code are actually reached. Admittedly, test code is typically trivial so this doesn't matter as much, but for the rare cases it does matter, it would be nice to have this ability. If nothing else, this would allow me to avoid having to place a // FILE NOT TESTED comment at the start of each test file. This comment is really jarring...

So absolutely no argument that the default should be only the src. I'm just asking for the ability explicitly to shoot myself in the foot ;-)

[sagiegurari]

@orenbenkiki I modified task coverage-kcov and setup the env var there, so it can be overridden on the task level (this is mainly because CARGO_MAKE_WORKING_DIRECTORY can dynamically change per task via cwd attribute).
this is now available in a development branch

[orenbenkiki]

Thanks!