[Bug?]: Portal doesn't find node binaries

[LekoArts]

Self-service

• I'd be willing to implement a fix

Describe the bug

Hi!

As a preface: I'm not 100% sure if this is a bug with yarn or Parcel but the current information presented to me leads me to believe that it's an issue with yarn. Hopefully we can figure it out!

So, let's get into it.

Description

I was trying to update Parcel to 2.5.0 in my PR gatsbyjs/gatsby#35446 that was based on renovate's PR gatsbyjs/gatsby#35307 to update Parcel to 2.4.1. Both PRs fail in our yarn PnP CI: https://app.circleci.com/pipelines/github/gatsbyjs/gatsby/82291/workflows/f57735e2-46c7-4a3d-ad8e-fecdd0580030/jobs/977309

So this happens both on the update from Parcel 2.3.2 -> 2.4.1 and 2.3.2 -> 2.5.0. We've gone through the changelog and couldn't find any related change (other than maybe parcel-bundler/parcel#7846) that could trigger this in Parcel?

Error:

error Error: /tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/gatsby-parcel-config-virtual-a1c7aa3859/4/home/circleci/project/node_modules/@parcel/source-map/parcel_sourcemap_node/artifacts/index.linux-x64-gnu.node: cannot open shared object file: No such file or directory
    at Object.Module._extensions..node (internal/modules/cjs/loader.js:1122:18)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at NodePackageManager.load (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/@parcel-package-manager-virtual-5853e6c739/0/cache/@parcel-package-manager-npm-2.5.0-451e73037f-d1b307d19c.zip/packages/core/package-manager/src/NodePackageManager.js:130:9)
    at NodePackageManager.requireSync (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/@parcel-package-manager-virtual-5853e6c739/0/cache/@parcel-package-manager-npm-2.5.0-451e73037f-d1b307d19c.zip/packages/core/package-manager/src/NodePackageManager.js:94:17)
    at Module.m.require (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/@parcel-package-manager-virtual-5853e6c739/0/cache/@parcel-package-manager-npm-2.5.0-451e73037f-d1b307d19c.zip/packages/core/package-manager/src/NodePackageManager.js:117:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/gatsby-parcel-config-virtual-a1c7aa3859/4/home/circleci/project/node_modules/@parcel/source-map/parcel_sourcemap_node/index.js:15:18)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Object.require$$0.Module._extensions..js (/tmp/e2e-tests/gatsby-pnp/.pnp.cjs:28713:33)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at NodePackageManager.load (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/@parcel-package-manager-virtual-5853e6c739/0/cache/@parcel-package-manager-npm-2.5.0-451e73037f-d1b307d19c.zip/packages/core/package-manager/src/NodePackageManager.js:130:9)
    at NodePackageManager.requireSync (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/@parcel-package-manager-virtual-5853e6c739/0/cache/@parcel-package-manager-npm-2.5.0-451e73037f-d1b307d19c.zip/packages/core/package-manager/src/NodePackageManager.js:94:17)
    at Module.m.require (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/@parcel-package-manager-virtual-5853e6c739/0/cache/@parcel-package-manager-npm-2.5.0-451e73037f-d1b307d19c.zip/packages/core/package-manager/src/NodePackageManager.js:117:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/tmp/e2e-tests/gatsby-pnp/.yarn/__virtual__/gatsby-parcel-config-virtual-a1c7aa3859/4/home/circleci/project/node_modules/@parcel/source-map/dist/node.js:14:18)

So we ssh'ed into the CircleCI instance, ran it again, and it still failed. We used yarn unplug and looked at the files that it says are missing, but they were there. Odd 🤔

Local Reproduction

Ok, so we went onto trying to reproduce this locally. And we saw two behaviors. We followed what our CI is doing first:

https://github.com/gatsbyjs/gatsby/blob/3beb047b05218eeef133f8a87e4019399fded34a/.circleci/config.yml#L320-L358

This is also what I'll write in the "To reproduce" field. Because our CI uses yarn link --all --private ~/project to create a portal in the example project to all deps of the monorepo. You can see the result of that here: https://github.com/LekoArts/pnp-bug-repro-01

Then as a second test we just cloned the starter project and used resolutions to force update the Parcel dependencies. Since essentially this is only what the PR is doing. And et voilà, it works! You can find a repository here: https://github.com/LekoArts/pnp-bug-repro-02

When not using Portals, but just resolutions this error doesn't appear.

Now the question is: Why is that happening?

To reproduce

throw new Error('see description')

1. Clone https://github.com/gatsbyjs/gatsby, checkout the branch parcel-move-plugin, and run yarn bootstrap. This should install all dependencies and build all packages
2. In a separate directory clone https://github.com/gatsbyjs/gatsby-starter-default and follow the steps: https://github.com/gatsbyjs/gatsby/blob/3beb047b05218eeef133f8a87e4019399fded34a/.circleci/config.yml#L332-L358. You'll want to change yarn link --all --private ~/project with ~/project being the absolute path to your local repository of Gatsby you created in step 1. You can also skip the step yarn add start-server-and-test@^1.11.0
3. Check that portals were created for all dependencies
4. Run yarn build and you should see the same error

Environment

Please note that this also happens on our CircleCI instance.

  System:
    OS: macOS 12.3.1
    CPU: (10) arm64 Apple M1 Pro
  Binaries:
    Node: 16.13.0 - /private/var/folders/h2/tzzmmt7s3bl2l2kbnvnhv5r40000gn/T/xfs-d4034098/node
    Yarn: 3.2.0 - /private/var/folders/h2/tzzmmt7s3bl2l2kbnvnhv5r40000gn/T/xfs-d4034098/yarn
    npm: 8.5.5 - ~/.nvm/versions/node/v16.13.0/bin/npm

Additional context

No response

[LekoArts]

I'm not sure how to reproduce this in Sherlock but I added a proper description on how to reproduce this reliably.

[yarnbot]

We couldn't reproduce your issue (all the assertions passed on master).