🚀 Breaking changes for Yarn 4

[arcanis]

This issue is just there to keep track of which changes we might want to do once we'll get to the next major release. Doesn't mean it'll be anytime soon 🙂 Will be updated as time passes and we think about more things.

• Don't run the tests anymore on Node 12 #4196
• feat(patch)!: Improves patches #4218
• Removes the compatibility layer for Clipanion 2 #4262
• Improves boot time #4261 (Removes yup)
• Make the local cache opt-in (rather than opt-out) #4252
• Prefer using exclusively Corepack when possible #4254
• Implements --check-resolutions #4302
• Implements --refresh-lockfile #4299
• Normalizes dependencies #4305 (uses npm: everywhere)
• Enables all official plugins by default #4253
• chore(pnp)!: remove fallbacks to react-scripts and gatsby #3004
• feat: automatic ZipFS memory management #3691
• perf(pnp): hoist data string #4320
• refactor(libui)!: require renderForm streams to be specified #4589
• Add export fields to manifests to prevent deep imports #4834
• refactor!: update hooks to accept objects #4524
• feat!: make all settings mergeable #4982
• Make commands git-aware again
  • Make yarn init initialize a git repository (already the case in 3.x)
  • Make yarn version create tags for the relevant workspaces
• Installer['getCustomDataKey']: "Move this method into Linker so that linkers can use it to save some state useful to findPackageLocator (cf PnpmLinker)."
• Make ReportError lazily generate report messages when provided a configuration object by the streams; this would allow us to remove the configuration parameter from many helper functions where we only use it for display purposes.
• Update yarn workspaces foreach recursive configuration; it's not clear at the moment what should happen when -R and --since are used together. Perhaps we'd need a --recursive-dependents and --recursive-dependencies flag?
• feat(foreach)!: automatically enable --verbose in tty environments #4595
• Cleanup fix: revert breaking changes from conditional dependencies #3612
  • refactor!: remove FetchOptions.skipIntegrityCheck #4641
  • Discuss the fetchPackageFromCache options bag.
• Transfer arcanis.vscode-zipfs to the Yarnpkg org
• Make Yarn detect whether it's running inside a public repository (GitHub Actions) and, if it is AND the repository uses zero-installs, exit and recommend adding either --check-cache or --no-check-cache.
• Write a formal spec for the PnP data files, so that 3rd-party resolvers (in particular native resolvers) can implement their own (originally I pushed for it to be a JS API so that we had more freedom to extend/correct the file format as needed, but in 3.x we didn't need to change it so it might now be stable enough that we can relinquish some control).
  • Writes a PnP spec #4671
  • Consider only writing this data file by default, unless an option to generate an old-style .pnp.cjs file is set. (nope)
• fix(core)!: rename caFilePath to httpsCaFilePath #4503
• fix(audit)!: remove fallback to publish registries #4639
• refactor(manifest)!: remove deprecated compatibility check methods #4644
• refactor(core)!: remove MapConfigurationValue #4645
• refactor!: remove deprecated versionUtils #4648
• refactor(init)!: remove --assume-fresh-project #4649
• refactor(sdks)!: drop fallback on createRequireFromPath #4697
• refactor(init)!: disable zero-installs by default #4698
• refactor!: remove pnpDataPath #4773
• refactor!: stop removing old .pnp.js files when migrating #4774
• refactor(plugin-link)!: rename resolvers and fetchers #4865
• refactor(core)!: replace workspace.dependencies with worskpace.anchoredPackage #4898
• refactor!: cleanup lutimes #4899
• refactor!: remove lockfileFilename #5604
• perf!: lazy load packageExtensions #5608
• refactor(core)!: remove workspace.locator #5619

[brillout]

Drop Node 12 support (EOL will be 2022-04-30)

This is a no-go for OSS devs. E.g. I run vite-plugin-ssr's test suite against Node 12 to make sure vite-plugin-ssr works with Node 12.

[arcanis]

This is a no-go for OSS devs. E.g. I run vite-plugin-ssr's test suite against Node 12 to make sure vite-plugin-ssr works with Node 12.

We don't intend to support Node versions that aren't maintained anymore by Node themselves. Not only to make our life easier, but also because some of our dependencies do that too, and need to be kept updated (thus making such Node requirements a necessity).

[brillout]

Ok makes sense. Even if v4 releases a couple of months earlier than 2022-04-30 that would be ok. Sorry for the noise.

[evanw]

From this comment it sounds like Yarn 4 might also change the data format:

I'm considering switching to a pure JSON format in the next Yarn major (and we'd publish this JSON as a formal spec that third-party resolvers can rely upon)

I went looking for the corresponding issue for that and ended up here. Should that be on this list too?

[arcanis]

I added an item to the list 👍

[edvardchen]

We'll be exploring a new resolution algorithm that will prevent most of the attacks similar to the recent color.js hijacking.
from https://dev.to/arcanis/yarn-32-libc-yarn-explain-next-major--o22

Is there any way to follow, the implementation or design or discussion? @arcanis

[pcjmfranken]

The only installation instructions provided in the docs are for corepack. Since corepack is not always an option (security concerns, having no npm, etc.) it would be nice if the manual instructions could be added back.

[milesj]

@pcjmfranken Corepack doesn't require npm. It's built into Node.js as of v14.19, which everyone should be using since v12 is EOL in a few days.

[pcjmfranken]

@milesj Corepack is actually built into¹ the NPM that ships with the official Node distributions. Also, custom Node builds can opt out of Corepack's inclusion independently of NPM's.

You're right in that corepack should be available in the majority of environments, but it's certainly not a given.

Footnotes

1. You can see the files being added to the node_modules directory here: https://github.com/nodejs/node/blob/master/tools/install.py#L82-L121 ↩

[magnattic]

The default value change for enableGlobalCache should be well documented when yarn 4 is released. This tripped me up when switching since we sync our local cache folder into our docker dev environment and I was confused why docker didn't work anymore. I expect this might happen to a few other people.

[devinrhode2]

Hey @arcanis, could any of these changes be cherry picked into v3?

[merceyz]

No, breaking changes can't be backported.

[PowerKiKi]

Yarn 4.0 first rc release is now over 1 year old. And there's been 43 rc releases since then. And the 4.0.0 milestone has no other opened issue.

Is there anything that is blocking a final release of Yarn 4.0.0 ? Is there a rough ETA of when this might happen ?

I am aware that we can use rc releases in the meantime, but it does look a bit unusual to have such a long period of time for rc. If it is stable, then it should probably be released, but if it isn't, then we probably should populate the milestone to reflect what is missing, shouldn't we ?

[arcanis]

Is there anything that is blocking a final release of Yarn 4.0.0 ?

Time! It's always time. Most of the work is done, but there's always a couple of smallish improvements we want to make around some "details". Myself I have a couple of things around JS constraints, and @merceyz mentioned he'd like to close #4952 first to make the migration smoother. There's also the website revamp, but it'll likely be shipped separately.

Is there a rough ETA of when this might happen ?

Probably soon, but no ETA.

[PowerKiKi]

Myself I have a couple of things around JS constraints

Could you create issues assigned to 4.0.0 for those things ? then the community might be able to help, at least a bit ?

[landsman]

Where I can find waiting issues to be done? Milestone is the right place? Because there is only this issue present there. https://github.com/yarnpkg/berry/milestone/2

[lockieluke]

everyone's been talking about how good yarn 4 is but where on earth are the yarn 4 builds

[adrian-gierakowski]

yarn set version 4.0.0-rc.48

[danielpza]

everyone's been talking about how good yarn 4 is but where on earth are the yarn 4 builds

yarn set version 4.0.0-rc.48

also yarn set version canary https://yarnpkg.com/cli/set/version

[lockieluke]

where can i find the change logs

[coolbaluk]

Dependabot updates seem broken due to the cache key change, and each PR will include a mass replacement of zips with vendoring enabled.

I know it's technically unrelated, but thought it's useful to have it documented here.

Edit: When a release for v4 is created, they should pick it up as per here:

https://github.com/dependabot/dependabot-core/blob/main/npm_and_yarn/Dockerfile#L6-L7

[arcanis]

Will they? From the look of it they use Corepack, so if your package manager is pinned via either packageManager or yarnPath, you should only get updated PR after you migrate to v4, right?

[coolbaluk]

Will they? From the look of it they use Corepack, so if your package manager is pinned via either packageManager or yarnPath, you should only get updated PR after you migrate to v4, right?

@arcanis It would have been nice, but my guess is they are still using v3 even with the above configured. This is the behaviour I'm seeing:

With:

"packageManager": "yarn@4.0.0-rc.50" and yarnPath: .yarn/releases/yarn-4.0.0-rc.50.cjs

I will write an issue on their side.

[coolbaluk]

^ Just an update on the above, it was an oversight on my side that I eventually discovered - it was the difference between darwin & linux in the OS that was causing the replacement, not the key change.

I've set it in .yarnrc now, there's no issue with dependabot.

[trivikr]

Since yarn 4.0 stable is released in https://github.com/yarnpkg/berry/releases/tag/%40yarnpkg%2Fcli%2F4.0.0, this issue can be closed.

[SimenB]

getting an overview of what breaking changes are relevant to different use cases of yarn would be nice, and not just a list of the commits

[arcanis]

We will have a blog post tomorrow getting into the high level changes. I'll keep the issue open until then (we just wanted to tag the release beforehand to avoid having to rush a fix if something broke during the deploy process).

[herzaso]

This release made all of our builds to fail.

Here is an excerpt from one of our docker files:

FROM node:16.19.0-bullseye-slim

RUN yarn config set scripts-prepend-node-path true
RUN yarn set version 3.3.1

This image, seem to include the latest version of yarn (4 from several hours ago) and since we rely on 3.3.1 we try to set it.

This is the error that we see:

Step 7/18 : RUN yarn set version 3.3.1
 ---> Running in c2e50735b75c
Usage Error: This tool requires a Node version compatible with >=18.12.0 (got 16.19.0). Upgrade Node, or set `YARN_IGNORE_NODE=1` in your environment.
Yarn Package Manager - 4.0.0

Changing the line to RUN YARN_IGNORE_NODE=1 yarn set version 3.3.1 does fix the issue, just thought you should know about it

[wojtekmaj]

@herzaso I think the error messages are clear enough. This is an expected behavior.

And do yourself a favor and upgrade Node.js to a version that has not reached end of life yet.

[herzaso]

@wojtekmaj as I said, the error was clear and we managed to fix, but enterprises move slow. We are in the process of moving to Node 20

[arcanis]

Here we go: https://yarnpkg.com/blog/release/4.0 !