Publish does not update package-lock

[Robinfr]

Expected Behavior

When doing a publish, the version number should not only be bumped in the package.json but also in the package-lock.json when available.

Current Behavior

The package-lock version does not get updated. This causes the version to be updated the next time an install is run creating an extra (useless) commit.

Possible Solution

package-lock.json version number should be made equal to the package.json version number when publishing.

Context

I'm doing a Lerna publish automatically. This pulls the latest version and publishes all of the packages to NPM. However, now I constantly have to create extra commits for the package-lock.

Your Environment

Executable	Version
lerna --version	2.2.0
npm --version	5.8.0
node --version	8.11.1

[evocateur]

The root package-lock.json, or the leaf package-locks (packages/*/package-lock.json)?

[Robinfr]

The leaf package locks. Op 8 mei 2018 6:27 p.m. schreef Daniel Stockman <notifications@github.com>: The root package-lock.json, or the leaf package-locks (packages/*/package-lock.json)? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1415 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AD90FEoP99-cIWkubTwO5cxH-vACxoP-ks5twcdigaJpZM4T2Len>.

[evocateur]

We're not calling npm version, just modifying the package.json directly and then calling npm publish. I try to avoid leaf package lockfiles, much like lerna itself does (requires lerna 3.x).

[Robinfr]

I'd suggest just changing the version in the package lock files as well if possible. The same way it's done for package.json. Is there any reason not to? Op 8 mei 2018 9:14 p.m. schreef Daniel Stockman <notifications@github.com>: We're not calling npm version, just modifying the package.json directly and then calling npm publish. I try to avoid leaf package lockfiles, much like lerna itself does (requires lerna 3.x). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1415 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AD90FI6LSc9J1s981uHy0thl59duBMj5ks5twe5-gaJpZM4T2Len>.

[evocateur]

Is there any reason not to?

For lerna 2.x, it's because I don't want to deal with that old codebase anymore.

For lerna 3.x (current master), it's a bandwidth issue. I'm only one person, and I have a day-job.

[Robinfr]

I'm not asking whether there is a reason for you not to, I'm asking in general. One of the pros of open source is that you don't have to be solely responsible for developing the software. If you agree with the approach I could try taking a look at implementing it when I have some time to help you out. Op 8 mei 2018 9:29 p.m. schreef Daniel Stockman <notifications@github.com>: Is there any reason not to? For lerna 2.x, it's because I don't want to deal with that old codebase anymore. For lerna 3.x (current master), it's a bandwidth issue. I'm only one person, and I have a day-job. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1415 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AD90FBxX7aaoyqwSsHbhq9SokeBfeKtSks5twfIKgaJpZM4T2Len>.

[evocateur]

Sure, a PR would be great.

[tizmagik]

I'd be happy to assist with this as well if you could point me in the right direction 😄

[Robinfr]

I was reading through the code but I think I also need some directions, I understand how the system works but it's entirely centered around the package.json file so adding the code for package-lock.json anywhere feels hacky to me..

@evocateur how did you disable the package-lock.json in the Lerna packages anyway? I can't find any .npmrc file or .gitignore in the project that would do this.

[doug-wade]

@Robinfr we use npm's file: specifiers to manage our monorepo, which means that when you run bootstrap, npm i is only run in the root directory. Thus, the extraneous package-lock.jsons are never generated.

[Robinfr]

@doug-wade but then the additional dependencies in the sub-packages are not installed?

[garthk]

@Robinfr when Lerna writes its temporary package.json it pulls out everything it thinks it doesn't need — including version and name, both of which get materialised into package-lock.json in the way you need. I think the fix is to write both version and name to the temporary package.json. If you're doing version, could you also do name for #1454? Thanks.

[Robinfr]

@garthk this issue has nothing to do with the package.json, but rather with the package-lock.json.

[garthk]

@Robinfr npm derives package-lock.json from package.json, so by changing how Lerna produces package.json during bootstrap you can get the desired result observed in package-lock.json without having to write code to read, alter, and re-write the latter.

[evocateur]

publish and bootstrap are completely separate operations, and bootstrap should not elicit unstaged changes. thus it needs to be updated during lerna publish. but it is not, at present, because i personally never use nested lockfiles.

[tivac]

I'm currently working around this by running

> npx lerna exec -- npm i

after each lerna publish. It's not ideal but it at least gets things stabilized again.

@evocateur what's your argument against nested lockfiles? I've got all my devDependencies hoisted to the root, but I still want to have consistent installs of the regular dependencies for everything in my packages dir.

I'm currently using npm ci for installs and I definitely don't want to let it just randomly decide to upgrade a nested package.

[evocateur]

Relative file: specifiers, just like the lerna source. It’s a unified package-lock for the entire tree, no bootstrap required. (I really need to document this pattern, I feel like I’ve repeated myself about eleventy-billion times over the past six months)

…

On Jun 25, 2018, at 22:00, Pat Cavit ***@***.***> wrote: I'm currently working around this by running > npx lerna exec -- npm i after each lerna publish. It's not ideal but it at least gets things stabilized again. @evocateur what's your argument against nested lockfiles? I've got all my devDependencies hoisted to the root, but I still want to have consistent installs of the regular dependencies for everything in my packages dir. I'm currently using npm ci for installs and I definitely don't want to let it just randomly decide to upgrade a nested package. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[tivac]

I'm using relative file specifiers in modular-css, but I've still got dependencies in all my packages because I can't hoist those to the root. Lerna looks to be doing the same thing?

lerna/commands/diff/package.json

Line 38 in f674f35

"npmlog": "^4.1.2"

So I still don't get how you get around having leaf node package-lock.json files. I've got one in the root for all my devDependencies just like lerna, but that doesn't help w/ anything in packages/ does it?

Would definitely love to know how I could improve my setup or workflow for using lerna@3 in modular-css so I appreciate any wisdom you can share!

[lizardruss]

Relative file: specifiers, just like the lerna source. It’s a unified package-lock for the entire tree, no bootstrap required. (I really need to document this pattern, I feel like I’ve repeated myself about eleventy-billion times over the past six months)

This approach won't work when a leaf package is going to be built into a Docker image. The docker build process won't follow symlinks, and the file:/// specifiers won't resolve unless the local dependencies are copied into the build context.

I'm not requesting a fix, only giving this example if anyone runs into the same issue I'm having. I realize it's not within the scope of this project to support docker builds.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[trippingtarballs]

Edit 20/01/19

Fix

Add the following to the root package.json of your project:

{
  "scripts": {
    "version": "lerna clean --yes && lerna exec \"npm install --ignore-scripts --package-lock-only --no-audit\" && git add packages/*/package-lock.json",
  }
}

Explanation

• lerna exec \"npm install --ignore-scripts --package-lock-only --no-audit\"
  • The use of lerna exec "npm install" over lerna bootstrap is because the bootstrap does not do what I expected.. In my particular use-case I have a package in my project where the package.json contains peerDependencis, but no dependencies or devDependencies. And, as it so turns out, if there are no dependencies or devDependencies in a package's package.json then the lerna bootstrap does not trigger an npm install and the package-lock.json of that package does not have its version pumped (but its package.json) does.
  • In order to generate the rather empty looking package-lock.json for a package with an empty dependencies and devDependencies I had to manually perform an npm install in the package's root folder.
  • I wanted to add a package-lock.json for my package with an empty dependencies and devDependencies for consistency, and because I may well have dependencies / devDependencies in this package in future.

---

My work around ...

Fix

Add the following to the root package.json of your project:

{
  "scripts": {
    "version": "lerna clean --yes && lerna bootstrap --ignore-scripts -- --package-lock-only --no-audit && git add packages/*/package-lock.json"
  }
}

Explanation

When the lerna publish command is executed the version run-script is invoked by lerna "AFTER bumping the package version, but BEFORE commit". So, breaking down the above run-script:

• lerna clean --yes
  • This is necessary because if a node_modules folder is present during the installation of a package's dependencies then the version that package's package-lock.json will not be updated by npm. I claim as much based on trial and error.
  • Given my workflow (where I perform a lerna publish) I do not expect there to be a node_modules present for any of my project's packages. However, it does no harm to be explicit.
• lerna bootstrap --ignore-scripts -- --package-lock-only --no-audit
  • The use of lerna bootstrap over lerna exec "npm install" is because that is what the bootstrap command is for.
  • The use of the --ignore-scripts option is because I use preinstall & postinstall in my project's root package.json and, for my particular lerna publish workflow, I do not care to have those run-scripts get triggered.
  • The use of the --ignore-scripts option before the -- is to instruct both bootstrap and npm to not execute any run-scripts.
  • --package-lock-only "argument will only update the package-lock.json, instead of checking node_modules and downloading dependencies".
  • --no-audit again, another time saver. I really just want to bump the version in the package's package-lock.json.
• git add packages/*/package-lock.json
  • Specific to my use-case. I have all my project's packages as subfolders of packages/.
  • The use of the * in packages/*/package-lock.json means the git add will only match on the package-lock.json file that is in a folder one-level down. Again, something that makes sense for my project.

Possible PR?

The need for a git add at all is due to the following lines (from lerna source code, commands/version/index.js):

    // exec version lifecycle in root (after all updates)
    chain = chain.then(() => this.runRootLifecycle("version"));

    if (this.commitAndTag) {
      chain = chain.then(() => gitAdd(Array.from(changedFiles), this.execOpts));
    }

The logic of the version command for lerna will; 1. compile a list of files that are to be included with the tag commit, 2. execute whatever version run-script you might have in your project's root package.json, and 3. execute the git add part of the tag commit.

In other words, it's not enough that lerna bootstrap --ignore-scripts -- --package-lock-only --no-audit would have updated the package-lock.json because lerna would have already decided on what files to stage for the tag commit. Hence why I include git add packages/*/package-lock.json – to ensure the package-lock.json is added to the tag commit.

Conclusions

Apologies for the monologue. This issue has been raised more than once (#925). Hopefully my input is helpful to some.

I did think about a PR to have the git add part of the tag commit be moved to after the version run-script has been invoked. But I don't know enough about lerna's source code to know if such a change would have unwanted side-effects.

[evocateur]

@trippingtarballs The git add in your custom lifecycle script is perfectly appropriate. Just because you're using Lerna doesn't mean Lerna must do all the work.

[binomialstew]

@trippingtarballs, in your example, when/how does the commit happen for the lock files? If I understand correctly, this is supposed to happen in the same commit where versions are tagged, but if I have these changed lock files added during the version script, I see the fatal lerna ERR! EUNCOMMIT Working tree has uncommitted changes.

[huboneo]

This full on breaks NPM functionality for lerna packages. Since dependency changes are not reflected in lock files basic npm commands such as npm audit fail to execute. If lerna uses NPM (or yarn), I feel it only appropriate it ensures they work as expected?

[matthias-ccri]

If you're using hoist, the lerna exec "npm i" workaround is not a good solution, because it generates a bad lockfile. Normally when bootstrapping with hoisting, lerna populates the lockfile with only the non-hoisted packages. npm i populates it with all.

Then, when you run lerna bootstrap, the lockfile takes precedence. Npm installs from the lockfile and not from the package.json, which effectively means packages aren't hoisted properly. I may be wrong about that, but it's what I've seen.

[remjx]

This workaround ensures that leaf packages are included as dependencies in the package-locks of other leaf packages: #1462 (comment)