[Turbopack] Yarn PnP installs

[arcanis]

Which project is this feature idea for?

Turbopack

(note: I saw #693, and specifically #693 (comment), but that issue was about Turborepo, not Turbopack. I checked with @sokra who confirmed that the goal is for Turbopack to eventually support everything Webpack supports, hence this issue specifically about Turbopack)

Describe the feature you'd like to request

Turbopack isn't compatible with Yarn PnP installs, unlike both Webpack and Esbuild. There are two reasons for this:

• Turbopack doesn't implement the PnP resolution (the formal spec is here, Esbuild followed it in their implementation)
• Turbopack doesn't support accessing files from zip archives (see the "Package Locations" section of the spec)

Reproduction

yarn create next-app my-app && cd my-app
yarn set version canary
yarn config set nodeLinker pnp
yarn install
yarn next dev

Describe the solution you'd like

Two options:

• Implement the zip fs logic within Turbopack, and make it take into account the .pnp.data.json data file as per spec

  • It'd be useful if someone familiar with the Turbopack codebase could share some pointers showing where we can find the relevant sections, and what would be the potential problems

• Allow Node plugins to handle specific file loading and resolution (like Webpack plugins; but then it'd be slower than necessary, and wouldn't be supported natively as it currently is in other bundlers)

Describe alternatives you've considered

In the meantime users can workaround this problem by reverting to the node-modules linker, but they don't get the PnP benefits anymore (fast installs, ghost dependency protection, semantic erroring, ...), which is often a blocker.

yarn config set nodeLinker node-modules

[IhsenBouallegue]

Any updates? This would be pretty awesome, but I assume it's a huge undertaking.

[arcanis]

Not directly related to Turbopack, but I implemented the PnP resolution algorithm (whose specs are here) as a Rust package, see here. It's still a bit experimental, but I'm more than open to merge PRs and make it compatible with Turbopack.

[robertwt7]

Yarn 4 is great, I don't know why vercel doesn't want to implement the PnP resolution. It'll help us a lot in the CI build time and the only package that's not supported right now in my repo is turbo

[ForsakenHarmony]

Closing as a duplicate of vercel/next.js#42651