Avoiding to build hard-to-build packages, but still depending on them?

[fkleedorfer]

I am contemplating to set up a monorepo with a few packages that require quite an intense build setup ("scary packages"). Nobody really wants to go through that sort of thing except for the developers of that specific functionality. Other packages are standard typescript ("normal packages"). Many developers potentially work on normal packages.

In such a scenario, could I build a normal package that depends on the current version ('*') of a scary package without having to build the scary package? For example by pulling the scary artifact from the nightly builds on an npm repository (or something similar to that effect)?

If there is a solution for the above, here's a followup question: in such a repo, imagine a development branch in which a scary package changes, with breaking changes to a normal package. How could the developer of the normal package work on that branch - which would require using the changed scary package - without having to build the scary package themselves?

For context: The question really is about pulling a number of repos together into one monorepo. One of these repos is really hard to build. Many others depend on it and are easier to build. I think that putting everything into a monorepo would reduce turnaround times (for bugs and feature requests) and allow frequent releases - if we can get around everyone except some core devs having to setup those scary builds.

[fahslaj]

Hi @fkleedorfer , take a look at caching build results here: https://lerna.js.org/docs/features/cache-tasks. Lerna uses Nx under the hood, which supports caching results of tasks, including npm script runs. You could set up caching for the build step of the "scary packages" so that when lerna goes to build the scary package (because it's a dependency of a normal one), lerna will read from the cache instead of actually run the build script.

This can be further augmented by utilizing Nx Cloud to share these cached build artifacts between developers so that they wouldn't even need to build the scary packages once if CI or another developer has already done so.

[fkleedorfer]

Thanks for the explanation! However, I don't know if I am comfortable with locking us into the use of nx-cloud. I was hoping we could publish the artifacts of every PR on npmjs.com using a github action and set up the local build to fetch artifacts of unchanged packages from there. Would that be possible?

[fahslaj]

It is definitely possible, but Lerna will not automate all of this for you. You could do the publishing of each artifact of a merged PR with Lerna's canary releases. When used in conjunction with the dist-tag option, you could get close to what you are describing. From some quick research, this blog post might help.

[fkleedorfer]

Thanks for the response! I am not sure I will get a chance to test it, but I will report back if I do.

I'll mark this as answered for the time being so it does not clog the discussions.