[Addon V2] Dependencies conflict between the addon and the test-app with the same package version for both (ember-source)

[Shishouille]

Using pnpm, when installing ember-source: ~5.2.0 in both addons and test-app, I got this error when launching tests:

test-app:test: Stack Trace and Error Report: /var/folders/jg/tyc48ncs3c99g10_vpqcckyr0000gn/T/error.dump.463e47596f7c3b8f29591ef187f10b92.log
test-app:test: Some V1 ember addons are resolving as incorrect peer dependencies. This makes it impossible for us to safely convert them to v2 format.
test-app:test: See https://github.com/embroider-build/embroider/blob/main/docs/peer-dependency-resolution-issues.md for an explanation of the problem and suggestions for fixing it.
test-app:test:
test-app:test: test-app@0.0.0 -> ember-formidable@0.0.0-beta.13 -> @ember/render-modifiers@2.1.0
test-app:test:     sees peerDep ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2/node_modules/ember-source
test-app:test:     but ember-formidable@0.0.0-beta.13 is using ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2_webpack@5.88.2/node_modules/ember-source
test-app:test:
test-app:test: test-app@0.0.0 -> ember-formidable@0.0.0-beta.13 -> @ember/render-modifiers@2.1.0
test-app:test:     sees peerDep ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2/node_modules/ember-source
test-app:test:     but test-app@0.0.0 is using ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2_webpack@5.88.2/node_modules/ember-source
test-app:test:
test-app:test: test-app@0.0.0 -> ember-formidable@0.0.0-beta.13 -> ember-concurrency@3.1.1
test-app:test:     sees peerDep ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2/node_modules/ember-source
test-app:test:     but ember-formidable@0.0.0-beta.13 is using ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2_webpack@5.88.2/node_modules/ember-source
test-app:test:
test-app:test: test-app@0.0.0 -> ember-formidable@0.0.0-beta.13 -> ember-concurrency@3.1.1
test-app:test:     sees peerDep ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2/node_modules/ember-source
test-app:test:     but test-app@0.0.0 is using ember-source@5.2.0
test-app:test:       at /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2_webpack@5.88.2/node_modules/ember-source

If it's helpful, I've logged the violations coming from validatePeerDependencies

{
  pkg: {
    root: '/Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-concurrency@3.1.1_@babel+core@7.22.10_ember-source@5.2.0/node_modules/ember-concurrency',
    packageCache: {
      appRoot: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
      rootCache: {},
      resolutionCache: {},
    },
    isApp: false,
    dependencyKeys: ['dependencies', 'peerDependencies'],
  },
  dep: {
    root: '/Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2/node_modules/ember-source',
    packageCache: {
      appRoot: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
      rootCache: {},
      resolutionCache: {},
    },
    isApp: false,
    dependencyKeys: ['dependencies', 'peerDependencies'],
  },
  ancestors: [
    {
      root: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
      packageCache: {
        appRoot: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
        rootCache: {},
        resolutionCache: {},
      },
      isApp: true,
      dependencyKeys: ['dependencies', 'devDependencies', 'peerDependencies'],
    },
    {
      root: '/Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/file+ember-formidable_@babel+core@7.22.10_@glimmer+component@1.1.2_@glimmer+tracking@1.1.2_@g_uulw5edgdwwlg6wyeb5fabft6q/node_modules/ember-formidable',
      packageCache: {
        appRoot: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
        rootCache: {},
        resolutionCache: {},
      },
      isApp: false,
      dependencyKeys: ['dependencies', 'peerDependencies'],
    },
  ],
  ancestor: {
    root: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
    packageCache: {
      appRoot: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
      rootCache: {},
      resolutionCache: {},
    },
    isApp: true,
    dependencyKeys: ['dependencies', 'devDependencies', 'peerDependencies'],
  },
  ancestorsDep: {
    root: '/Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2_webpack@5.88.2/node_modules/ember-source',
    packageCache: {
      appRoot: '/Users/home/DEV/BEVOLTA/ember-formidable/test-app',
      rootCache: {},
      resolutionCache: {},
    },
    isApp: false,
    dependencyKeys: ['dependencies', 'peerDependencies'],
  },
};

The only solution I've found is downgrading the addon ember-source version to ~5.1.2. Then it works.

I thought this was an isolated incident, but some people also had the issue:
https://discord.com/channels/480462759797063690/1151598660505833525

If it's an error on our side, is it possible to add a clearer error message?

Thank you very much!

[NullVoxPopuli]

I believe this is related to some of the info here: https://github.com/embroider-build/embroider/blob/main/docs/peer-dependency-resolution-issues.md
which brings in:

• https://pnpm.io/package_json#dependenciesmetainjected
• and support turborepo when using dependenciesMeta.*. injected (make injected deps re-sync) pnpm/pnpm#6088 (comment)
  (which is a broader JS ecosystem problem exposed by the error you ran in to, rather than anything with embroider itself, but the way I fix it personally is depmeta*injected)

It's not enough to have the version match for any particular dependency, but when using peers, it's important that only one version is resolved (the version defined by the consumer).

idk how many folks are familiar with peers, but I've found this document very helpful for myself: https://pnpm.io/how-peers-are-resolved

[Shishouille]

Thanks for your answer @NullVoxPopuli 😄 . To be honest, I've already read the embroider docs and the pnpm docs you've sent. I still don't see why matching peers could be an issue if the version we want to use is already defined everywhere? Doesn't peerDeps act like a condition (i.e doesn't allow the installation if the version is too old) then when both ember-source here are installed as devDependencies in the addon and test-app?

I already also use _syncPnpm in that project :( .

I can close an issue as it's probably not embroider related and we certainly need more understanding to peer dependencies. I'll dig dipper and try to gain some expertise on that!

[NullVoxPopuli]

well, I think this is still a problem that we, as a community, can make better -- we need to provide actionable next steps when encountering errors like this.

Is the repo you're working with open source? I'd love to poke around and see if I can figure out if there is a way to automate away the error and report back with next steps -- (and then maybe there is a way to suggest those next steps when future error-viewers encounter what you've encountered).

Doesn't peerDeps act like a condition (i.e doesn't allow the installation if the version is too old) then when both ember-source here are installed as devDependencies in the addon and test-app?

It's a bit goofy, because peerDeps is a communication tool, but only protects you from the error if the devDep version is omitted (which most folks can't do because they need the types from their peerDep).
We're constrained by the behavior of node's resolve() algo.

One tool I like to use is, in a particular project, find out where my dep is. For example, I'll cd to a directory, and

node -e "conosle.log(require.resolve('my peer dep name'));"

To see what path node is discovering.
And then I can compare the result in the consuming project.
If the paths are different, that reveals the issue that the embroider error is trying to communicate.

I'm re-opening the issue, because I think there are DX improvements to be had here, and the main issue I see is that the error isn't clear enough nor does it provide action items.
(it does link to the peer deps doc, but I think we can do more)

[Shishouille]

Alright 😄 Yes, it is open-source! You have to know that I've used your ember-primitives config hehe (thank you so so much for what you're doing by the way 👍 )

I've tried your command, here are the results:
test-app: /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2_webpack@5.88.2/node_modules/ember-source/lib/index.js

addon: /Users/home/DEV/BEVOLTA/ember-formidable/node_modules/.pnpm/ember-source@5.2.0_@babel+core@7.22.10_@glimmer+component@1.1.2_@glint+template@1.0.2/node_modules/ember-source/lib/index.js

So the only difference is that one is resolved with webpack and the other isn't. I don't really see what we can do about it though ☹️

[NullVoxPopuli]

I think in this particular case, there was a bug in pnpm. I wrote some more about it on the PR here: BEVOLTA/ember-formidable#29

So, a couple action items we/i can add to the error:

1. ensure package-manager version is up to date
2. wipe out all node_modules directories and re-install
3. dedupe dependencies

[ef4]

Thanks for reporting this and thanks @NullVoxPopuli for investigating it.

Everybody wants fast and reliable package management and the NPM ecosystem was... definitely not designed for that. So it's experiencing a lot of growing pains as both package authors and package managers start to actually get serious about correctness.

So it's not at all unusual for embroider to be handed nonsense situations. That's why I added this error. Without it, you'd most likely just get a weirder error later in the build or at runtime.

[Shishouille]

Thanks a lot @NullVoxPopuli, for enhancing the error and finding the issue! This is great 😄