Create pnpm webpack resolver plugin

[bjester]

Overview

In transitioning to pnpm from yarn, it has become apparent that a webpack plugin would be helpful to resolve the differences between the two package managers, and how webpack resolves dependencies. In essence, the hoisted nature of yarn v1 can cause dependency conflicts, but it can also create 'ghost dependencies' that are not explicitly defined in the project but end up being resolved anyway.

The benefits of this webpack plugin will be:

• easier transition to pnpm
• clearer indication of missing dependencies (raise warnings or errors)
• avoidance of global workarounds that can lead to ghost dependencies
• allows us to be more explicit with dependencies overall

Description and outcomes

A webpack resolver plugin should be created in kolibri-tools which:

• resolves first party dependencies (direct project dependencies) as normal: only if available in /node_modules/
• resolves third party dependencies by:
  • first resolving with that first party's dependencies: /node_modules/.pnpm/kolibri-design-system@XXX/node_modules/
  • if the module being resolved is a peer dependency of the first party dependency, allows: /node_modules/*
  • otherwise throws a warning / error about missing dependency

Once this plugin is created, we can use the plugin in Studio:

• add it to Studio's webpack config for resolving both modules and loaders
• remove the manual inclusion of /node_modules/.pnpm/node_modules/ in the config for resolving both modules and loaders

Resources

• Webpack resolvers API
• Plug'n'Play webpack plugin -- not too dissimilar but instead of using some external pnp api, we can implement the logic directly in the webpack plugin

Workarounds

The simplest workaround that maintains somewhat similar behavior to yarn is to add node_modules/.pnpm/node_modules to webpack's modules resolver configs. Although, this does not help to clarify 'ghost dependencies'.

I was not able to find something like this that exists. There is this plugin but it doesn't work because I believe it's using the wrong webpack API. pnpm does have support for Yarn's plug'n'play, but it requires a custom resolver to be injected into node commands (as far as I can tell) which is unappealing. yarn's PnP also gets rid of node_modules which feels weird and gives less visibility IMO.

Ghost dependencies

Kolibri Studio uses workbox, and there were some workbox dependencies missing. This wasn't apparent until transitioning to pnpm, likely because some other dependency had included it and that was hoisted. For workbox in particular, this wasn't a major issue because there hasn't been much change in the libraries, but it illustrates the ghost dependency problem.