@pnpmfile npm org

[vjpr]

Wanted to gather thoughts on this idea.

I created an org called @pnpmfile.

The plan is to publish packages there to use as readPackage hooks to fix broken packages.

E.g:

function readPackage(pkg) { 
  require('@pnpmfile/create-react-native-app').default(pkg) 
}

---

I also plan on having a package called pnpmfile-check which will inform you if there are incompatible packages in your tree.

function readPackage(pkg) { 
  require('@pnpmfile-check').default(pkg)
}

Or:

scripts: {
  postinstall: './node_modules/.bin/pnpmfile-check'
}

[vjpr]

I think this is a bad idea. A package manager requiring installing packages to run is not going to work.

Copy and paste feels like the only approach that could work.

---

Another solution would be to use webpack to compile any helpers to a minified file that you can copy-paste in your app.

But I think I would prefer to just keep the pnpmfile simple.

Having access to the semver library would be helpful though. But requiring it could slow down pnpm installs...

[vjpr]

@zkochan Do you think it would be possible to have a hook to install dependencies before pnpmfile.js is run? Or would you prefer to keep pnpmfile.js dependency-free?

Without this, it doesn't seem like its easy to extend pnpmfile.js. E.g. Say you want to use the semver package, you would have to include to source code in your project via copy-paste.

---

Or, we could use preinstall to install into a subdirectory called e.g. pnpmfile? But this might slow things down...

---

Or we could run twice pnpm install --ignore-pnpmfile, then pnpm install.

[vjpr]

Unfortunately a preinstall script adds ~1s to pnpm install.

Without preinstall:

pnpm install  1.99s user 0.36s system 108% cpu 2.172 total

With "preinstall": "pnpm install --prefix=pnpmfile --no-verify-store-integrity":

pnpm install  2.89s user 0.54s system 102% cpu 3.344 total

---

Checking for existence of pnpmfile/node_modules speeds things up but is not as safe.

"preinstall": "ls pnpmfile/node_modules || pnpm install --prefix=pnpmfile --no-verify-store-integrity"

pnpm i  2.02s user 0.38s system 108% cpu 2.220 total

pnpm i  1.93s user 0.34s system 106% cpu 2.127 total

---

Still very clunky. But the best I can think of at the moment for reusing pnpmfile hooks.

Say I have 5 apps that use CRNA, manually updating the pnpmfile in all 5 projects would be annoying. But also setting up preinstall scripts for all projects, plus seeing the preinstall message every time.

[zkochan]

It is possible to configure pnpm recursive install to do installation in project/pnpmfile/ then in project/. It should be faster than the preinstall script.

But project should depend on project/pnpmfile. Otherwise, pnpm will do installation in both projects concurrently. So you can create a package.json like this:

{
  "name": "pnpmfile",
  "version": "1.0.0"
  ...
}

and then in project/package.json, add pnpmfile as a dep:

{
  ...
  "pnpmfile": "^1.0.0",
  ...
}

[vjpr]

@zkochan Awesome. Didn't think of that!

But why do recursive installs need to rely on ordering? Shouldn't they all be done concurrently?

[zkochan]

A dependency can influence the installation of the dependent package if it is used in a lifecycle script.

I believe lerna installs this way as well

[vjpr]

I believe lerna installs this way as well

Hopefully. I will test this tomorrow. If so, its the perfect solution.

[zkochan]

oh, the right command will probably be pnpm recursive link because pnpm recursive install will try to install pnpmfile project/ from the npm registry

[vjpr]

I think it might even be a good idea to have this functionality built into pnpm, primarily because you don't want to have to rely on a preinstall script to install dependencies for a dependency manager to work.

We could have a central github repo with a json file that pnpm could optionally query.

Then when you install it could check if there are custom resolutions available for packages, and ask you if you want to merge them in to your existing resolutions.

Maybe we could have a separate file called pnpm-resolutions.json to save the resolutions. Related #1082.

Could also support a notes field that provides additional info about how to get it working with pnpm that could show as an info log, such as, "you must use --shamelessly-flatten".

It would be good too, to make a list of the major and most popular packages/frameworks people use and make sure we support them. E.g. create-react-app, create-react-native-app, gatsby, angular, react, etc.

[hulkish]

@vjpr I am not opposed to the basic idea described in OP. But, there should definitely be strong emphasis on the real packages having these issues corrected.

In the event that one of the real packages fixes their issue - i would hope that the matching @pnpmfile package would be deprecated.

We should not lose sight of the fact that these packages actually have legitimate issues that need to be fixed - regardless of the package manager.

We wont improve the outcome of this challenge by letting rogue duplicate packages live for long. Not to mention, it becomes a maitainance nightmare for publishing upstream updates.

While I understand why pnpm introduced --shamefully-flatten - I feel like we should eventually arrive at a place where its super rare to be needed at all. It is also a huge blocker to the whole idea of pnpm to have to use this option - just because of one package.

[hulkish]

@vjpr After thinking it through a bit more, and a continuation of my previous comment - I actually think we should just maintain a "broken package list".

This puts the burden where it belongs.

[vjpr]

Maybe a funny name and shame website of broken packages to raise awareness of this issue.

[hulkish]

@vjpr Agreed. I think that is a better way to resolve this. Basically we need to tell the truth as it is - "you are using node incorrectly"

[zkochan]

I think it is enough to have something like a json with the overrides

[zkochan]

Now Yarn 2 has the same issues as pnpm in strict mode.

They create a database of "replacement, seems like similar to what this issue suggests: https://github.com/yarnpkg/berry/blob/master/packages/plugin-compat/sources/extensions.ts

[vjpr]

I think it is enough to have something like a json with the overrides

@zkochan Is there any existing work on this?

[zkochan]

I did not work on this.

[zkochan]

We will use Yarn's compat database #4676