Accurately measure code coverage during integration tests

[rsimha]

Today, the code coverage measured by the integration tests is ~31%.

This is inaccurate because we're not instrumenting the built runtime used in commonIntegrationTestPaths.

amphtml/build-system/config.js

Lines 36 to 62 in aafa151

	const builtRuntimePaths = [
	{
	pattern: 'dist/**/*.js',
	included: false,
	nocache: false,
	watched: true,
	},
	{
	pattern: 'dist.3p/**/*',
	included: false,
	nocache: false,
	watched: true,
	},
	{
	pattern: 'dist.tools/**/*.js',
	included: false,
	nocache: false,
	watched: true,
	},
	];
	const commonUnitTestPaths = initTestsPath.concat(fixturesExamplesPaths);
	const commonIntegrationTestPaths = initTestsPath.concat(
	fixturesExamplesPaths,
	builtRuntimePaths
	);

Adding istanbul instrumentation via browserify to gulp build slows down the build by an order of magnitude and doesn't change the coverage measured (#22567 (comment)). Not sure what else can be done, or if it's even feasible to measure code coverage for built JS files that are loaded by webpages during tests.

/cc @ampproject/wg-infra @aghassemi

[rsimha]

This has come up during recent conversations about improving release quality and enforcing a minimum code coverage bar. Bumping this to P2.

[rsimha]

[Infra triage] Some progress has been made towards accurately measuring coverage during integration / e2e tests since this issue was filed. See #30463, #30129, #31787 (abandoned). Putting this on the radar of @rileyajones in case there is other low-hanging fruit that will allow us to get coverage numbers during CI.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.