Make depedenciesMeta inject by default for workspace packages?

[deokseong-kim-toss]

• I'd be willing to implement this feature (contributing guide)

Describe the user story

We maintain many packages with monorepo, but we often make mistake "missing peers"

• packages/a
  • deps: workspace:b
• packages/b
  • peerDeps: something

expect: mssing peers 'something' in 'b'
result: nothing

dependenciesMeta.*.inject option helps, but it is quite annoying write inject when every deps. It also cause another human error.

Describe the solution you'd like

npm options like "use-inject-workspace = true"

Describe the drawbacks of your solution

I don't fully understand why inject should be in package.json instead a pnpm option, use-inject-workspace makes inject not explicit.

Describe alternatives you've considered

use-inject-workspace = dependencies
use-inject-workspace = dependencies,devDependencies

or

add inject when add workspace as dependency?

Question:
Sorry for question. I wonder why pnpm don't use inject for all dependencies. Does it have performance drawback?

[zkochan]

I wonder why pnpm don't use inject for all dependencies. Does it have performance drawback?

The main reason is that when you use inject and you rebuild the injected dependency, you somehow need to resync that dependency.

[blowsie]

I would agree with this decision, if a workspace package specifies a peerDependency, it is simply ignored when you install it from another workspace package, without any error or warning.

[NullVoxPopuli]

can the resync somehow happen automatically?

I'm using dependenciesMeta.injected right now to help out with a deep peer dependency, and my library that forwards that peer needs to be built, and those built assets aren't present in the injected version until I run install again.

Since this will be a common scenario for me (in a monorepo), I'd like this to somehow be more ergonomic

[zkochan]

Currently the packages will be automatically synced if they are built via a prepare script. But they are not resynced when someone rebuilds them. Probably this should be fixed.

[NullVoxPopuli]

anyone have ideas for how to re-sync dependencies when using turborepo? I could prefix all my scripts with pnpm install, but that'd only work once, because without changes detected, turbo would skip the prefixed pnpm install

I opened an issue here on turbo's repo: vercel/turbo#2306
just for cross-referencing

[NullVoxPopuli]

@zkochan I had an idea for how we can reasonably do this / implement this feature.
It's not 100% fool proof, so it may need to be opt-in via .npmrc option, but the gist is:

• analyze package.json#files (or package.json#exports)

• only symlink those files entries (+ package.json of course)
  -> this way, when the library updates its dist (or whatever folder), because the folders are symlinked, updates are automatic, and we don't need to re-run pnpm install

  thoughts?

[zkochan]

only symlink those files entries (+ package.json of course)

This will not work with symlinks.

This will only work with hard links (that we use now) or copies. Symlinks are ignored when node.js resolves dependencies. So even though the symlink to project-b will be inside project-a's node_modules, the dependencies of project-b will be resolved from the node_modules of project-b because that is where the symlink points.

[NullVoxPopuli]

would the same idea work with hard links? instead of file-based hard-links, hard-link folders, based on package.json#files / dirname(package.json#exports)?

Do hardlinks still work if say, the whole dist folder is deleted, and re-generated by rollup?

[zkochan]

No, hard links are only possible upon files.

[NullVoxPopuli]

Has node rejected the idea of resolving symlinks?
Maybe we can change node.

Edit: found this: symlink alternative and the rationale for node not resolving symlinks

But wait, this is all unrelated, we need the symlink in the repo-root .pnpm directory, not node_modules.
How does the .pnpm directory get used it resolution? Is that up to pnpm to configure? If so, i don't think the node issue 3402 would apply?

[zkochan]

Node.js should resolve symlinks to their real locations. Otherwise, node.js would not work with pnpm.

[NullVoxPopuli]

How do you mean?, you said:

Symlinks are ignored when node.js resolves dependencies.

[zkochan]

For example, let's consider these two projects:

a
  index.js
  node_modules
    react (v17)

b
  index.js
  node_modules
    a --> ../../a
    react (v16)

If b/index.js will require a, a will resolve react from a/node_modules/react, not from b/node_modules/react. Because the real location of a is not inside b/node_modules/a but inside a. Node.js is looking for dependencies in the real location.

[NullVoxPopuli]

does this behavior apply to all files or just the package root?
maybe if b's a is a real folder, with hard-link to a's package.json, and then symlinking a dist folder for the other contents?

I think I need to test this out in a monorepo. I'll report back with that

[NullVoxPopuli]

I got distracted, but in the mean time, @simonihmig made a util that would make re-syncing easier -- I commented about maybe adding it to pnpm here: #6088 (comment)

[NullVoxPopuli]

Added a README to this project: https://github.com/NullVoxPopuli/pnpm-sync-dependencies-meta-injected

it at least allows folks to regain some productivity 😅
and has a watch mode

[NullVoxPopuli]

@zkochan would it make sense to internalize this library as a daemon, or... how should we proceed?

[zkochan]

What do you mean by initializing it as a daemon? You mean adding it to pnpm as a new command?

Unfortunately, I cannot really dogfood this as I only used injected deps with Bit, which automatically writes to the "injected" dists when compiling. For now, if your package works, you can transfer it to the pnpm org and we can reference it from the docs.

[NullVoxPopuli]

What do you mean by initializing it as a daemon?

without build time integration, something needs to be running to watch output directories (package.json#files, for example), and if those change, re-sync them to the injected folders.

you can transfer it to the pnpm org and we can reference it from the docs.

sounds good, I'll make a demo tomorrow wednesday of with and without the tool to show how it works and add that to the readme before transfer

[NullVoxPopuli]

@zkochan -- how is transferring supposed to work? should I transfer it to your user account first?