no name, version in package.json → broken Node Security Project

[garthk]

Due to lerna bootstrap running npm install with a thinned package.json, the package-lock.json produced by npm lacks the name field required by nsp, the Node Security Project. This is similar to #1415's lack of an updated version.

Expected Behavior

lerna bootstrap
cd package/xxx
node -e 'assert.equal(require("./package.json").name, require("./package-lock.json").name)'
echo $?
nsp check

→

0
(+) No known vulnerabilities found

… indicating success.

Current Behavior

AssertionError [ERR_ASSERTION]: 'xxx' == undefined
    at [eval]:1:8
    at ContextifyScript.Script.runInThisContext (vm.js:50:33)
    at Object.runInThisContext (vm.js:139:38)
1
(+) Debug output: {"statusCode":400,"error":"Bad Request","message":"Error: child \"packagelock\" fails because [child \"name\" fails because [\"name\" is required]]","details":[{"message":"\"name\" is required","path":"packagelock.name","type":"any.required","context":{"key":"name"}}]}
(+) Error: Got an invalid response from Node Security, please email the above debug output to support@nodesecurity.io

… because there's no name or version in package-lock.json.

Possible Solution

Write name and version to the temporary package.json

Steps to Reproduce (for bugs)

This isn't a buggy implementation. Rather, it's correct implementation of an insufficiently good definition of "done". Easily fixed, but I'll skip the repro if you don't mind.

lerna.json

{
  "lerna": "2.0.0",
  "hoist": false,
  "packages": [
    "pkg/*"
  ],
  "version": "independent",
  "commands": {
    "publish": {
      "ignore": [
        "*.md",
        "test/**"
      ]
    }
  }
}

Context

How has this issue affected you?

I can't check to see if any of my dependencies have security vulnerabilities.

What are you trying to accomplish?

I'm trying to use nsp to check to see if any of my dependencies have security vulnerabilities.

Your Environment

Executable	Version
lerna --version	2.11.0
npm --version	5.6.0
node --version	v8.11.2

OS	Version
macOS Sierra	10.13.4

[evocateur]

I think you’ll find lerna 3 doesn’t mangle as much as v2, if you insist on using the hacky bootstrap method.

[garthk]

I tried 3.0.0-beta.20, @evocateur, but ran into #1419 forced hoisting with incorrect paths. If someone is willing to fix the hoisting or give me some hints on where in the code to fix it myself, I'll give it another shot with my full repo and isolating the incorrect paths part if it's still happening in beta 21.

[evocateur]

I don't understand your reticence to "hoist" your devDependencies to the root. It's how lerna should have behaved in the first place, imo.

[garthk]

Yeah, let's unpack it. Why not.

• Some of the projects' devDependencies sets are huge
• One project's dependencies contains a module that sometimes has a 15min install time for reasons beyond our control
• npm install and, thus, lerna bootstrap, used to take a really long time
• So, that's why I'm so interested in lerna bootstrap --scope $(PACKAGE) --include-filtered-dependencies --ignore-scripts with no hoisting: speed, mainly through avoiding optional work

All that said:

• npm install has since got a lot faster
• If I'm reading your comment in lerna link convert → forced bad hoisting; bad file: refs #1419 right, withfile:packages/wut relationships we won't get the devDependencies for wut at all with an npm install in the dependent package, which might break the build phase
• That one annoying package hasn't taken 15min in months
• So, it might plausibly all turn out fine

Remaining concerns:

• I'm not sure how npm publish will work with those file: dependencies: what's your plan, there?
• I'm also not sure how those builds will work, which comes down to the TODO docs
• Hoisted devDependencies would force us to upgrade them all in sync, which we do most of the time but which we sometimes need to defer because of scheduling pressure

Zooming out, I'm getting the feel you're doubling down on support for monorepo management for packages which ship together as version-synchronised shards e.g. Turf and drifting away from support — which might have been accidental — for closed source internal teams using Lerna while shipping packages that ship separately with distinct versions. Which is fine, but let's call it out early so everyone in the latter category can make plans.

[tivac]

Lerna automatically rewrites local path dependencies to monorepo packages during publishing. It's how I've published the last 9-10 modular-css releases.

[evocateur]

Lerna itself has been dogfooding the relative file specifiers for almost all of the 3.0 betas so far. Nothing has changed (nor will change) to our support for “independent” versioning. I switched a few internal monorepos to it just last week. A dev dependency that takes 15 minutes to install sounds like a nightmarish urban legend. I don’t think it’s lerna’s responsibility to fix that, though you have my sympathy.

…

On Jun 14, 2018, at 19:36, Pat Cavit ***@***.***> wrote: Lerna automatically rewrites local path dependencies to monorepo packages during publishing. It's how I've published the last 9-10 modular-css releases. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[garthk]

Plain nightmare, I'm afraid. Try installing gdal whenever your version of Node hits a major upgrade before their build infrastructure catches up. One time, we ended up forking it just so we could store our own copy of the heavy artifact.

I'll double-check our workflow to see if we're up for lerna publish again. We stopped using it because of the obnoxious version bumping (#718 / #707).

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.