feat(build): enable incremental compilation

[bajtos]

Enable incremental compilation to speed up our builds. Note: this improvement will apply to projects consuming LoopBack too, once they upgrade to the latest version of @loopback/build.

BEFORE

$ time npm run build
  real  0m35.229s
  user  3m51.223s
  sys   0m14.384s

AFTER

The first build to populate tsbuildinfo file:

$ time npm run build
  real  0m35.692s
  user  3m55.728s
  sys   0m13.828s

Subsequent builds (in my case, there were no code changes from the last build):

$ time npm run build
  real  0m19.510s
  user  2m3.894s
  sys   0m9.581s

I am expecting the build times to improve even more when TypeScript 3.5 is released, see https://devblogs.microsoft.com/typescript/announcing-typescript-3-5-rc/

Checklist

👉 Read and sign the CLA (Contributor License Agreement) 👈

• npm test passes on your machine
• New tests added or existing tests modified to cover all changes
• Code conforms with the style guide
• API Documentation in code was updated
• Documentation in /docs/site was updated
• Affected artifact templates in packages/cli were updated
• Affected example projects in examples/* were updated

👉 Check out how to submit a PR 👈

[raymondfeng]

Is it extracted from #1636 (I enabled incremental build there too)?

[raymondfeng]

Let's use dist/.tsbuildinfo by configuring tsconfig compilerOptions with:

{
  "tsBuildInfoFile": ".tsbuildinfo"
}

This way, no changes is needed in clean script.

[raymondfeng]

Please add "tsBuildInfoFile": ".tsbuildinfo",.

[bajtos]

AFAIK, "tsBuildInfoFile": ".tsbuildinfo" setting in packages/build/config/tsconfig.common.json will be resolved as packages/build/config/.tsbuildinfo, which isn't what we want.

I think we will have to add tsBuildInfoFile setting to every per-package package.json file.

I'll take a look.

[raymondfeng]

IIRC, it's resolved against the outDir.

[bajtos]

Quoting from https://www.typescriptlang.org/docs/handbook/tsconfig-json.html#configuration-inheritance-with-extends:

All relative paths found in the configuration file will be resolved relative to the configuration file they originated in.

See also microsoft/TypeScript#29172

Quoting also from microsoft/TypeScript#29813

• When tsconfig specifies compiler options incremental file with .tsbuildinfo extension is emitted.
• Composite projects indirectly specify incremental as true and its error to specify incremental as false in the composite project
• If project specifies tsBuildInfoFile options, it uses the path specified by that to write the build information.
• If project specifies out or outFile option, the outFileWithoutExtension.tsbuildinfo file is written next to output js file.
• If project specified outDir, config files base file name with extension as .tsbuildinfo is written in outDir
• Otherswise config files base file name with extension as .tsbuildinfo is written next to the config file

AFAICT, we are not setting outDir in our tsconfig files and configure it via a CLI option instead. This is a legacy from the days where we used to have multiple dist directories (dist8 for Node.js 8, dist10 for Node.js 10, etc.)

[bajtos]

If project specified outDir, config files base file name with extension as .tsbuildinfo is written in outDir

This rule does not seem to be triggered in our current build setup, even when I change all tsconfig.build.json files to include "outDir": "dist" :(

[bajtos]

Is it extracted from #1636 (I enabled incremental build there too)?

Not at all, I was not aware of that effort.

In this pull request, I am unconditionally enabling incremental build for all mono-repo packages and all newly created projects. Existing projects created before my changes are not affected.

[bajtos]

In this pull request, I am unconditionally enabling incremental build for all mono-repo packages and all newly created projects. Existing projects created before my changes are not affected.

Actually, that's not correct. Because the changes are made in shared tsconfig.common.json, they will apply to existing projects too.

[bajtos]

@raymondfeng I have updated the pull request, PTAL again.

• added outDir and tsBuildInfoFile to per-package tsconfig.build.json files, including CLI templates
• enabled incremental flag in the shared config

The tests are failing because of a bug in lerna bootstrap, the problem will be fixed by #2944 and lerna/lerna#2104.

[bajtos]

added outDir and tsBuildInfoFile to per-package tsconfig.build.json files, including CLI templates

I think we should also modify list of files used by npm pack, so that tsbuildinfo is NOT published to npm.

[raymondfeng]

Maybe we should keep it simple - leave the defaults as is - .tsbuildinfo in the package root directory. We can add .tsbuildinfo to .gitignore.

[bajtos]

Maybe we should keep it simple - leave the defaults as is - .tsbuildinfo in the package root directory. We can add .tsbuildinfo to .gitignore.

Done.

@raymondfeng LGTY now?

[raymondfeng]

https://github.com/strongloop/loopback-next/pull/2928/files#r291249751

[bajtos]

@raymondfeng should we modify npm run clean to remove tsconfig.build.tsbuildinfo files? Should I modify clean script in every per-package package.json file, or should be perhaps improve lb-clean to automatically delete that file?