feat: prune support for lockfiles containing patches #2151

chris-olszewski · 2022-10-03T22:38:00Z

Add support for pruning lockfiles that contain patches created via yarn patch or pnpm patch.

This PR:

Adds a mechanism for package managers to prune the patched dependencies section of a package.json
Adds Berry implementation for pruning package.json
Adds pnpm implementation for pruning package.json
Fixes issue in pnpm lockfile implementation where peerDependencies could override the resolved version that appears in dependencies

Note: folding in #2120 to this PR since it will probably only be used by this feature

Original PR text for #2120:

In order to properly support patches created by yarn patch or pnpm patch for turbo prune we need to parse and marshal all fields in package.json. Since we only want to prune information in package.json related to patched dependencies we need to keep track of all fields.

This PR:

Changes parse behavior so we now keep any unrecognized fields in RawJSON
Adds support for marshalling PackageJSON so we can produce one with only the applicable patches

vercel · 2022-10-03T22:38:05Z

@chris-olszewski is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

vercel · 2022-10-03T22:38:07Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Updated
turborepo-docs-g413	❌ Failed (Inspect)		Oct 3, 2022 at 11:08PM (UTC)

chris-olszewski · 2022-10-03T23:06:37Z

cli/internal/lockfile/berry_lockfile.go

@@ -493,8 +493,13 @@ func (d *_Descriptor) primaryVersion() (string, bool) {
 	if patchFileIndex < 0 || versionRangeIndex < 0 {
 		panic("Patch reference is missing required markers")
 	}
+	// The ':' following npm protocol gets encoded as '%3A' in the patch string
+	version := strings.Replace(d.versionRange[versionRangeIndex+1:patchFileIndex], "%3A", ":", 1)


I ran into this issue when trying to get this working, but it also came up in #2146

gsoltis · 2022-10-04T20:27:56Z

cli/internal/packagemanager/pnpm.go

+	for dependency, patch := range patchedDependencies {
+		patch, ok := patch.(string)
+		if !ok {
+			return fmt.Errorf("Expected only strings in patchedDependencies. Got %v", patch)


Are we guaranteed to have a usable value of patch in this error? I think the failed cast is also going to reassign patch

gsoltis · 2022-10-04T20:28:41Z

cli/internal/packagemanager/berry.go

+			inPatches := false
+			patch, ok := patch.(string)
+			if !ok {
+				return fmt.Errorf("Expected value of %s in package.json to be a string, got %v", dependency, patch)


I think patch will be invalid here

cli/internal/fs/package_json.go

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [postcss](https://postcss.org/) ([source](https://togithub.com/postcss/postcss)) | [`8.4.16` -> `8.4.17`](https://renovatebot.com/diffs/npm/postcss/8.4.16/8.4.17) | [![age](https://badges.renovateapi.com/packages/npm/postcss/8.4.17/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/postcss/8.4.17/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/postcss/8.4.17/compatibility-slim/8.4.16)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/postcss/8.4.17/confidence-slim/8.4.16)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>postcss/postcss</summary> ### [`v8.4.17`](https://togithub.com/postcss/postcss/blob/HEAD/CHANGELOG.md#8417) [Compare Source](https://togithub.com/postcss/postcss/compare/8.4.16...8.4.17) - Fixed `Node.before()` unexpected behavior (by Romain Menke). - Added TOC to docs (by Mikhail Dedov). </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 10pm every weekday,before 5am every weekday,every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/vercel/turborepo).

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [typescript](https://www.typescriptlang.org/) ([source](https://togithub.com/Microsoft/TypeScript)) | [`4.8.3` -> `4.8.4`](https://renovatebot.com/diffs/npm/typescript/4.8.3/4.8.4) | [![age](https://badges.renovateapi.com/packages/npm/typescript/4.8.4/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/typescript/4.8.4/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/typescript/4.8.4/compatibility-slim/4.8.3)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/typescript/4.8.4/confidence-slim/4.8.3)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - "after 10pm every weekday,before 5am every weekday,every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/vercel/turborepo).

…el#2139) [![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [ts-json-schema-generator](https://togithub.com/vega/ts-json-schema-generator) | [`1.1.1` -> `1.1.2`](https://renovatebot.com/diffs/npm/ts-json-schema-generator/1.1.1/1.1.2) | [![age](https://badges.renovateapi.com/packages/npm/ts-json-schema-generator/1.1.2/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/ts-json-schema-generator/1.1.2/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/ts-json-schema-generator/1.1.2/compatibility-slim/1.1.1)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/ts-json-schema-generator/1.1.2/confidence-slim/1.1.1)](https://docs.renovatebot.com/merge-confidence/) | --- ### Configuration 📅 **Schedule**: Branch creation - "after 10pm every weekday,before 5am every weekday,every weekend" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Renovate will not automatically rebase this PR, because other commits have been found. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, click this checkbox. ⚠ **Warning**: custom changes will be lost. --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/vercel/turborepo).

chris-olszewski added 3 commits September 29, 2022 09:39

keep additional package.json fields

d5eb5b8

add package.json marshalling support

5ee6706

Merge branch 'main' into feat_extended_package_json

9440239

chris-olszewski mentioned this pull request Oct 3, 2022

feat: extended package json #2120

Closed

vercel bot had a problem deploying to Preview – turborepo-docs-g413 October 3, 2022 22:38 Failure

chris-olszewski marked this pull request as ready for review October 3, 2022 22:38

chris-olszewski requested a review from a team as a code owner October 3, 2022 22:38

chris-olszewski requested review from mehulkar and tknickman October 3, 2022 22:38

chris-olszewski commented Oct 3, 2022

View reviewed changes

vercel bot had a problem deploying to Preview – turborepo-docs-g413 October 3, 2022 23:08 Failure

chris-olszewski requested a review from gsoltis October 4, 2022 15:25

gsoltis added the pr: fixship label Oct 4, 2022

gsoltis approved these changes Oct 4, 2022

View reviewed changes

mehulkar and others added 8 commits October 5, 2022 11:06

Use Warn method for logWarning (vercel#2135)

5b338ec

Add support to prune berry patches

9b16d3b

Fixup prune support for pnpm lockfiles with patches

a3b268e

Fix empty slice check

4561174

use switch for type checks

8bd61e8

chris-olszewski force-pushed the feat_prune_patch_support branch from 0e0169e to 8bd61e8 Compare October 5, 2022 18:07

chris-olszewski added 3 commits October 5, 2022 11:12

fix bad type assertion

88655e5

Merge branch 'main' into feat_prune_patch_support

d4f4128

Merge branch 'main' into feat_prune_patch_support

0122446

chris-olszewski merged commit e77be8b into vercel:main Oct 5, 2022

chris-olszewski deleted the feat_prune_patch_support branch October 5, 2022 19:09

chris-olszewski mentioned this pull request Nov 17, 2022

fix: Yarn 2+ no longer include peer dependencies in AllDependencies #2745

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: prune support for lockfiles containing patches #2151

feat: prune support for lockfiles containing patches #2151

chris-olszewski commented Oct 3, 2022

vercel bot commented Oct 3, 2022

vercel bot commented Oct 3, 2022 •

edited

Loading

chris-olszewski Oct 3, 2022

gsoltis Oct 4, 2022

gsoltis Oct 4, 2022

feat: prune support for lockfiles containing patches #2151

feat: prune support for lockfiles containing patches #2151

Conversation

chris-olszewski commented Oct 3, 2022

Original PR text for #2120:

vercel bot commented Oct 3, 2022

vercel bot commented Oct 3, 2022 • edited Loading

chris-olszewski Oct 3, 2022

Choose a reason for hiding this comment

gsoltis Oct 4, 2022

Choose a reason for hiding this comment

gsoltis Oct 4, 2022

Choose a reason for hiding this comment

vercel bot commented Oct 3, 2022 •

edited

Loading