Add cpu profiling

[Lakitna]

As discussed in #2417 (comment)

[nicojs]

Thanks. Could you give me a small introduction about what --cpu-prof does? 😅

[nicojs]

What is this doing here? n is never updated.

[Lakitna]

n is updated on 56 and used on 55. Basically, I use it to enumerate the child process output files to check how many of them are missing. Another naming scheme is fine with me.

[nicojs]

Should we not reuse this n? For example, with --concurrency 4, we might want to use 1, 2, 3, and 4, but never 5. Whenever a child process crashes, it can reuse the number.

[Lakitna]

No, that'll result in overwritten .cpuprofile files.

n here is not a good solution by any means, but it is a human-readable approach to unique file names. Doing it sequenced like this allows a human to see how many processes crashed, as crashed processes have no .cpuprofile file.

For example:

We can see that 1, 3, 6-10, 12-14, and 17 are missing here. These processes must have crashed for some reason.

This insight is the only reason why we need n.

Edit: The last file of the run is child-process-278.cpuprofile. I have a feeling that there are many more processes than you're expecting.

[nicojs]

I think adding environment variables should be a "last resort". (I literally have nightmares about the vue-cli and the zoo of env variables there 😪).

Can we simply check if the current process has the --cpu-prof argument and, if so, forward that to the child process?

[Lakitna]

Yes, we can, I initially had it set up with a flag. The downside here is that we will have to add the flag to the StrykerCli.ts, so the option would be exposed to the user.

[Lakitna]

See commit, I've changed things over to the --cpu-prof flag on stryker run.

[Lakitna]

In simple terms, --cpu-prof calls the V8 profiler where --prof uses a Node-specific profiler. The upside of using the V8 one is tooling support.

Running a script with --cpu-prof will output a .cpuprofile file. This file can be read by Chromes profiler (chrome://inspect -> Open dedicated DevTools for Node -> Profile tab -> load).

In the profiler tooling, you can see at what point of the scripts life cycle it was doing what. For example, in the image above you can see that the first part of the script is all requires, and then it starts running mutants. In fact, a simple ctrl+f tells me this particular child worker only executed tests for 3 mutants.

[nicojs]

I was thinking, maybe it might be a better idea to add --workerNodeArgs support. It would be a really low level and simple way to support both --cpu-prof, --prof and any other arguments, like --inspect-brk.

It would be a simple (dumb) sollution, so users should combine this with --concurrency.

It would allow for low maintenance and maximum flexibility.

@Lakitna , what do you think?

[nicojs]

This use case will be supported with #2609