Code coverage as a bundle metric

[theKashey]

Two things:

• while bundle size in most “visible” metric the main metric is the amount of code executed, not downloaded
• ‘size-limit’ is not just “size control” utility, but “size discovery” as well

wondering, would this extra metric be useful, including usecase for creation solutions with less “boot time footprint”. The “code usage percentage” falls into the same bucket as “code size” and “execution time”, so it would not be that alien for size-limit functionality, however there is an open question how to make it actually usable and useful.

based on https://twitter.com/patak_js/status/1451440832347877377?s=21

[ai]

Any idea how to get this data?

[theKashey]

According to Jest currently it can be done via:

• nyc Babel plugin
• V8 build in functionality - https://v8.dev/blog/javascript-code-coverage

probably offloading all this stuff to Istanbul is the best option.

[timocov]

I've implemented similar approach in tradingview/lightweight-charts#517 and I had the following idea in mind - the library provides isn't supposed to be working with tree-shaking (because we minified bundled code better than almost any consumer's minifier could do and because of library's API) so we wanted to make sure that the shipped code doesn't have any "dead" code in it. Thus I implemented a test to verify that:

• builds library
• opens chromium via puppeteer with predefined page (https://github.com/tradingview/lightweight-charts/pull/517/files#diff-18950a558770801ba92d5a694bca3332c310325fa72bd61d0e9da968c979407cR250-R270)
• starts collecting coverage
• does some actions on the page, in that case it was a file with calling different API functions (see https://github.com/tradingview/lightweight-charts/pull/517/files#diff-e9dcee00d9b721181d8ef977c7d5e7211a1ff19d7a71ca58809cdf7ed50573ecR1-R245)
• once everything is finished stop collecting coverage and report its value against the config value (in case of reducing coverage)

In other (library-agnostic) words the algorithm was the following:

• bundle a library (in case of testing tree-shaking the entry point for this bundle should be provided in the config)
• run puppeteer
• start coverage collecting by running await page.coverage.startJSCoverage()
• load the page with predefined script that does required actions with bundled library (if provided in the configuration)
• once actions are done and no errors on the page call const coverageEntries = await page.coverage.stopJSCoverage() to get coverage entries back
• in these entries find an entry for a library bundle file and report its coverage (e.g. https://github.com/tradingview/lightweight-charts/pull/517/files#diff-18950a558770801ba92d5a694bca3332c310325fa72bd61d0e9da968c979407cR185-R211)