Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Notice: Node 10.16.x npm 6.9.0 upgrade npm ci regression #28987

Closed
apaprocki opened this issue Aug 6, 2019 · 5 comments
Closed

Notice: Node 10.16.x npm 6.9.0 upgrade npm ci regression #28987

apaprocki opened this issue Aug 6, 2019 · 5 comments
Labels
npm Issues and PRs related to the npm client dependency or the npm registry.

Comments

@apaprocki
Copy link
Contributor

npm ci no longer passes configuration properly to node-gyp in npm 6.9.0, so many Node.js use cases that require building native modules have completely broken in the 10.16.x upgrade.

While this is technically an npm issue, this will break almost anyone relying on passing custom options to node-gyp for it to even work properly. See the following bugs:
https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032
https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658
https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069

I found this out on my own because Node 10.16.x default npm is now incapable of building native modules in a container/chroot with no network access:
https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2

I'm only raising this issue to make the Node team aware (if they weren't already), since the decision was made here to incorporate 6.9.0 into the 10.16 minor semver bump. I have no idea if there is any possibility to revert the 6.9.0 upgrade at this point. At a minimum, perhaps there need to be integration tests for npm commands as part of the total upgrade process, testing node, node-gyp, and npm together.

@ChALkeR ChALkeR added the npm Issues and PRs related to the npm client dependency or the npm registry. label Aug 6, 2019
@MylesBorins
Copy link
Member

/cc @isaacs

any idea about this?

@apaprocki
Copy link
Contributor Author

FWIW it was mentioned in the first link:

Having spent some time on it, is seems that all keys starting with npm_config_ are missing from process.env when you use npm ci.

This looks to have been introduced between v6.5.0 and v6.6.0-next.0.

@isaacs
Copy link
Contributor

isaacs commented Aug 12, 2019

This will be fixed in the next npm release, most likely landing Tuesday 2019-06-20.

@mikePMG
Copy link

mikePMG commented Aug 15, 2019

This will be fixed in the next npm release, most likely landing Tuesday 2019-06-20.

This date is a in the past. Can you please check.

Trott pushed a commit to npm/node that referenced this issue Aug 20, 2019
Full release notes:

A few meaty bugfixes, and introducing `peerDependenciesMeta`.

FEATURES

* [`a12341088`](npm/cli@a123410)
  [nodejs#224](npm/cli#224) Implements
  peerDependenciesMeta ([@arcanis](https://github.com/arcanis))
* [`2f3b79bba`](npm/cli@2f3b79b)
  [nodejs#234](npm/cli#234) add new forbidden 403
  error code ([@claudiahdz](https://github.com/claudiahdz))

BUGFIXES

* [`24acc9fc8`](npm/cli@24acc9f)
  and
  [`45772af0d`](npm/cli@45772af)
  [nodejs#217](npm/cli#217)
  [npm.community#8863](https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863)
  [npm.community#9327](https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,)
  do not descend into directory deps' child modules, fix shrinkwrap
  files that inappropriately list child nodes of symlink packages
  ([@isaacs](https://github.com/isaacs) and
  [@salomvary](https://github.com/salomvary))
* [`50cfe113d`](npm/cli@50cfe11)
  [nodejs#229](npm/cli#229) fixed typo in semver doc
  ([@gall0ws](https://github.com/gall0ws))
* [`e8fb2a1bd`](npm/cli@e8fb2a1)
  [nodejs#231](npm/cli#231) Fix spelling mistakes in
  CHANGELOG-3.md ([@XhmikosR](https://github.com/XhmikosR))
* [`769d2e057`](npm/cli@769d2e0)
  [npm/uid-number#7](npm/uid-number#7) Better
  error on invalid `--user`/`--group` configs. This addresses the issue
  when people fail to install binary packages on Docker and other
  environments where there is no 'nobody' user.
  ([@isaacs](https://github.com/isaacs))
* [`8b43c9624`](npm/cli@8b43c96)
  [nodejs#28987](nodejs#28987)
  [npm.community#6032](https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032)
  [npm.community#6658](https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658)
  [npm.community#6069](https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069)
  [npm.community#9323](https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2)
  Fix the regression where random config values in a .npmrc file are not
  passed to lifecycle scripts, breaking build processes which rely on
  them.  ([@isaacs](https://github.com/isaacs))
* [`8b85eaa47`](npm/cli@8b85eaa)
  save files with inferred ownership rather than relying on `SUDO_UID`
  and `SUDO_GID`. ([@isaacs](https://github.com/isaacs))
* [`b7f6e5f02`](npm/cli@b7f6e5f)
  Infer ownership of shrinkwrap files
  ([@isaacs](https://github.com/isaacs))
* [`54b095d77`](npm/cli@54b095d)
  [nodejs#235](npm/cli#235) Add spec to dist-tag
  remove function ([@theberbie](https://github.com/theberbie))

DEPENDENCIES

* [`dc8f9e52f`](npm/cli@dc8f9e5)
  `pacote@9.5.7`: Infer the ownership of all unpacked files in
  `node_modules`, so that we never have user-owned files in root-owned
  folders, or root-owned files in user-owned folders.
  ([@isaacs](https://github.com/isaacs))
* [`bb33940c3`](npm/cli@bb33940)
  `cmd-shim@3.0.0`:
  * [`9c93ac3`](npm/cmd-shim@9c93ac3)
    [#2](npm/cmd-shim#2)
    [npm#3380](npm/npm#3380) Handle
    environment variables properly
    ([@basbossink](https://github.com/basbossink))
  * [`2d277f8`](npm/cmd-shim@2d277f8)
    [nodejs#25](npm/cmd-shim#25)
    [nodejs#36](npm/cmd-shim#36)
    [nodejs#35](npm/cmd-shim#35) Fix 'no shebang' case
    by always providing `$basedir` in shell script
    ([@igorklopov](https://github.com/igorklopov))
  * [`adaf20b`](npm/cmd-shim@adaf20b)
    [nodejs#26](npm/cmd-shim#26) Fix `$*` causing an
    error when arguments contain parentheses
    ([@satazor](https://github.com/satazor))
  * [`49f0c13`](npm/cmd-shim@49f0c13)
    [nodejs#30](npm/cmd-shim#30) Fix paths for
    MSYS/MINGW bash ([@dscho](https://github.com/dscho))
  * [`51a8af3`](npm/cmd-shim@51a8af3)
    [nodejs#34](npm/cmd-shim#34) Add proper support
    for PowerShell ([@ExE-Boss](https://github.com/ExE-Boss))
  * [`4c37e04`](npm/cmd-shim@4c37e04)
    [#10](npm/cmd-shim#10) Work around quoted
    batch file names ([@isaacs](https://github.com/isaacs))
* [`a4e279544`](npm/cli@a4e2795)
  `npm-lifecycle@3.1.3` ([@isaacs](https://github.com/isaacs)):
    * fail properly if `uid-number` raises an error
    * [`7086a1809`](npm/cli@7086a18)
  `libcipm@4.0.3` ([@isaacs](https://github.com/isaacs))
* [`8845141f9`](npm/cli@8845141)
  `read-package-json@2.1.0` ([@isaacs](https://github.com/isaacs))
* [`51c028215`](npm/cli@51c0282)
  `bin-links@1.1.3` ([@isaacs](https://github.com/isaacs))
* [`534a5548c`](npm/cli@534a554)
  `read-cmd-shim@1.0.3` ([@isaacs](https://github.com/isaacs))
* [`3038f2fd5`](npm/cli@3038f2f)
  `gentle-fs@2.2.1` ([@isaacs](https://github.com/isaacs))
* [`a609a1648`](npm/cli@a609a16)
  `graceful-fs@4.2.2` ([@isaacs](https://github.com/isaacs))
* [`f0346f754`](npm/cli@f0346f7)
  `cacache@12.0.3` ([@isaacs](https://github.com/isaacs))
* [`ca9c615c8`](npm/cli@ca9c615)
  `npm-pick-manifest@3.0.0` ([@isaacs](https://github.com/isaacs))
* [`b417affbf`](npm/cli@b417aff)
  `pacote@9.5.8` ([@isaacs](https://github.com/isaacs))

TESTS

* [`b6df0913c`](npm/cli@b6df091)
  [nodejs#228](npm/cli#228) Proper handing of
  /usr/bin/node lifecycle-path test
  ([@olivr70](https://github.com/olivr70))
* [`aaf98e88c`](npm/cli@aaf98e8)
  `npm-registry-mock@1.3.0` ([@isaacs](https://github.com/isaacs))
isaacs added a commit to npm/node that referenced this issue Aug 21, 2019
Full changelog:

6.11.1 (2019-08-20):

Fix a regression for windows command shim syntax.

* [`37db29647`](npm/cli@37db296)
  `cmd-shim@3.0.2` ([@isaacs](https://github.com/isaacs))

v6.11.0 (2019-08-20):

A few meaty bugfixes, and introducing `peerDependenciesMeta`.

FEATURES

* [`a12341088`](npm/cli@a123410)
  [nodejs#224](npm/cli#224) Implements
  peerDependenciesMeta ([@arcanis](https://github.com/arcanis))
* [`2f3b79bba`](npm/cli@2f3b79b)
  [nodejs#234](npm/cli#234) add new forbidden 403 error
  code ([@claudiahdz](https://github.com/claudiahdz))

BUGFIXES

* [`24acc9fc8`](npm/cli@24acc9f)
  and
  [`45772af0d`](npm/cli@45772af)
  [nodejs#217](npm/cli#217)
  [npm.community#8863](https://npm.community/t/installing-the-same-module-under-multiple-relative-paths-fails-on-linux/8863)
  [npm.community#9327](https://npm.community/t/reinstall-breaks-after-npm-update-to-6-10-2/9327,)
  do not descend into directory deps' child modules, fix shrinkwrap files
  that inappropriately list child nodes of symlink packages
  ([@isaacs](https://github.com/isaacs) and
  [@salomvary](https://github.com/salomvary))
* [`50cfe113d`](npm/cli@50cfe11)
  [nodejs#229](npm/cli#229) fixed typo in semver doc
  ([@gall0ws](https://github.com/gall0ws))
* [`e8fb2a1bd`](npm/cli@e8fb2a1)
  [nodejs#231](npm/cli#231) Fix spelling mistakes in
  CHANGELOG-3.md ([@XhmikosR](https://github.com/XhmikosR))
* [`769d2e057`](npm/cli@769d2e0)
  [npm/uid-number#7](npm/uid-number#7) Better
  error on invalid `--user`/`--group` configs. This addresses the issue
  when people fail to install binary packages on Docker and other
  environments where there is no 'nobody' user.
  ([@isaacs](https://github.com/isaacs))
* [`8b43c9624`](npm/cli@8b43c96)
  [nodejs#28987](nodejs#28987)
  [npm.community#6032](https://npm.community/t/npm-ci-doesnt-respect-npmrc-variables/6032)
  [npm.community#6658](https://npm.community/t/npm-ci-doesnt-fill-anymore-the-process-env-npm-config-cache-variable-on-post-install-scripts/6658)
  [npm.community#6069](https://npm.community/t/npm-ci-does-not-compile-native-dependencies-according-to-npmrc-configuration/6069)
  [npm.community#9323](https://npm.community/t/npm-6-9-x-not-passing-environment-to-node-gyp-regression-from-6-4-x/9323/2)
  Fix the regression where random config values in a .npmrc file are not
  passed to lifecycle scripts, breaking build processes which rely on them.
  ([@isaacs](https://github.com/isaacs))
* [`8b85eaa47`](npm/cli@8b85eaa)
  save files with inferred ownership rather than relying on `SUDO_UID` and
  `SUDO_GID`. ([@isaacs](https://github.com/isaacs))
* [`b7f6e5f02`](npm/cli@b7f6e5f)
  Infer ownership of shrinkwrap files
  ([@isaacs](https://github.com/isaacs))
* [`54b095d77`](npm/cli@54b095d)
  [nodejs#235](npm/cli#235) Add spec to dist-tag remove
  function ([@theberbie](https://github.com/theberbie))

DEPENDENCIES

* [`dc8f9e52f`](npm/cli@dc8f9e5)
  `pacote@9.5.7`: Infer the ownership of all unpacked files in
  `node_modules`, so that we never have user-owned files in root-owned
  folders, or root-owned files in user-owned folders.
  ([@isaacs](https://github.com/isaacs))
* [`bb33940c3`](npm/cli@bb33940)
  `cmd-shim@3.0.0`:
  * [`9c93ac3`](npm/cmd-shim@9c93ac3)
    [#2](npm/cmd-shim#2)
    [npm#3380](npm/npm#3380) Handle environment
    variables properly ([@basbossink](https://github.com/basbossink))
  * [`2d277f8`](npm/cmd-shim@2d277f8)
    [nodejs#25](npm/cmd-shim#25)
    [nodejs#36](npm/cmd-shim#36)
    [nodejs#35](npm/cmd-shim#35) Fix 'no shebang' case by
    always providing `$basedir` in shell script
    ([@igorklopov](https://github.com/igorklopov))
  * [`adaf20b`](npm/cmd-shim@adaf20b)
    [nodejs#26](npm/cmd-shim#26) Fix `$*` causing an
    error when arguments contain parentheses
    ([@satazor](https://github.com/satazor))
  * [`49f0c13`](npm/cmd-shim@49f0c13)
    [nodejs#30](npm/cmd-shim#30) Fix paths for MSYS/MINGW
    bash ([@dscho](https://github.com/dscho))
  * [`51a8af3`](npm/cmd-shim@51a8af3)
    [nodejs#34](npm/cmd-shim#34) Add proper support for
    PowerShell ([@ExE-Boss](https://github.com/ExE-Boss))
  * [`4c37e04`](npm/cmd-shim@4c37e04)
    [#10](npm/cmd-shim#10) Work around quoted
    batch file names ([@isaacs](https://github.com/isaacs))
* [`a4e279544`](npm/cli@a4e2795)
  `npm-lifecycle@3.1.3` ([@isaacs](https://github.com/isaacs)):
    * fail properly if `uid-number` raises an error
* [`7086a1809`](npm/cli@7086a18)
  `libcipm@4.0.3` ([@isaacs](https://github.com/isaacs))
* [`8845141f9`](npm/cli@8845141)
  `read-package-json@2.1.0` ([@isaacs](https://github.com/isaacs))
* [`51c028215`](npm/cli@51c0282)
  `bin-links@1.1.3` ([@isaacs](https://github.com/isaacs))
* [`534a5548c`](npm/cli@534a554)
  `read-cmd-shim@1.0.3` ([@isaacs](https://github.com/isaacs))
* [`3038f2fd5`](npm/cli@3038f2f)
  `gentle-fs@2.2.1` ([@isaacs](https://github.com/isaacs))
* [`a609a1648`](npm/cli@a609a16)
  `graceful-fs@4.2.2` ([@isaacs](https://github.com/isaacs))
* [`f0346f754`](npm/cli@f0346f7)
  `cacache@12.0.3` ([@isaacs](https://github.com/isaacs))
* [`ca9c615c8`](npm/cli@ca9c615)
  `npm-pick-manifest@3.0.0` ([@isaacs](https://github.com/isaacs))
* [`b417affbf`](npm/cli@b417aff)
  `pacote@9.5.8` ([@isaacs](https://github.com/isaacs))

TESTS

* [`b6df0913c`](npm/cli@b6df091)
  [nodejs#228](npm/cli#228) Proper handing of
  /usr/bin/node lifecycle-path test
  ([@olivr70](https://github.com/olivr70))
* [`aaf98e88c`](npm/cli@aaf98e8)
  `npm-registry-mock@1.3.0` ([@isaacs](https://github.com/isaacs))
muxator added a commit to ether/etherpad-lite that referenced this issue Oct 20, 2019
This upgrade should be backward compatible, but still suffers form major
vulnerabilities in its https-proxy-agent transitive dependency (see
https://www.npmjs.com/advisories/1184).

Changelog:
- https://github.com/npm/cli/releases

6.12.0 (2019-10-08):
    Now npm ci runs prepare scripts for git dependencies, and respects the
    --no-optional argument. Warnings for engine mismatches are printed again.
    Various other fixes and cleanups.

    BUG FIXES
    890b245dc #252 ci: add dirPacker to options (@claudiahdz)
    f3299acd0 #257 npm.community#4792 warn message on engine mismatch
                   (@ruyadorno)
    bbc92fb8f #259 npm.community#10288 Fix figgyPudding error in npm token
                   (@benblank)
    70f54dcb5 #241 doctor: Make OK more consistent (@gemal)

    FEATURES
    ed993a29c #249 Add CI environment variables to user-agent (@isaacs)
    f6b0459a4 #248 Add option to save package-lock without formatting Adds a new
                   config --format-package-lock, which defaults to true.
                   (@bl00mber)

DEPENDENCIES
    0ca063c5d npm-lifecycle@3.1.4:
        fix: filter functions and undefined out of makeEnv (@isaacs)
    5df6b0ea2 libcipm@4.0.4:
        fix: pack git directories properly (@claudiahdz)
        respect no-optional argument (@cruzdanilo)
    7e04f728c tar@4.4.12
    5c380e5a3 stringify-package@1.0.1 (@isaacs)
    62f2ca692 node-gyp@5.0.5 (@isaacs)
    0ff0ea47a npm-install-checks@3.0.2 (@isaacs)
    f46edae94 hosted-git-info@2.8.5 (@isaacs)

TESTING
    44a2b036b #262 fix root-ownership race conditions in meta-test (@isaacs)

6.11.3 (2019-09-03):
    Fix npm ci regressions and npm outdated depth.

    BUG FIXES
    235ed1d28 #239 Don't override user specified depth in outdated. Restores
                   ability to update packages using --depth as suggested by npm audit. (@G-Rath)
    1fafb5151 #242 npm.community#9586 Revert "install: do not descend into
                   directory deps' child modules" (@isaacs)
    cebf542e6 #243 npm.community#9720 ci: pass appropriate configs for file/dir
                   modes (@isaacs)

    DEPENDENCIES
    e5fbb7ed1 read-cmd-shim@1.0.4 (@claudiahdz)
    23ce65616 npm-pick-manifest@3.0.2 (@claudiahdz)

6.11.2 (2019-08-22):
    Fix a recent Windows regression, and two long-standing Windows bugs. Also,
    get CI running on Windows, so these things are less likely in the future.

    DEPENDENCIES
    9778a1b87 cmd-shim@3.0.3: Fix regression where shims fail to preserve exit
              code (@isaacs)
    bf93e91d8 npm-package-arg@6.1.1: Properly handle git+file: urls on Windows
              when a drive letter is included. (@isaacs)

    BUGFIXES
    6cc4cc66f escape args properly on Windows Bash Despite being bash, Node.js
              running on windows git mingw bash still executes child processes
              using cmd.exe. As a result, arguments in this environment need to
              be escaped in the style of cmd.exe, not bash. (@isaacs)

    TESTS
    291aba7b8 make tests pass on Windows (@isaacs)
    fea3a023a travis: run tests on Windows as well (@isaacs)

6.11.1 (2019-08-20):
    Fix a regression for windows command shim syntax.

    37db29647 cmd-shim@3.0.2 (@isaacs)

v6.11.0 (2019-08-20):
    A few meaty bugfixes, and introducing peerDependenciesMeta.

    FEATURES
    a12341088 #224 Implements peerDependenciesMeta (@arcanis)
    2f3b79bba #234 add new forbidden 403 error code (@claudiahdz)

    BUGFIXES
    24acc9fc8 and 45772af0d #217 npm.community#8863 npm.community#9327 do not
              descend into directory deps' child modules, fix shrinkwrap files
              that inappropriately list child nodes of symlink packages (@isaacs
              and @salomvary)
    50cfe113d #229 fixed typo in semver doc (@gall0ws)
    e8fb2a1bd #231 Fix spelling mistakes in CHANGELOG-3.md (@XhmikosR)
    769d2e057 npm/uid-number#7 Better error on invalid --user/--group configs.
              This addresses the issue when people fail to install binary
              packages on Docker and other environments where there is no
              'nobody' user. (@isaacs)
    8b43c9624 nodejs/node#28987 npm.community#6032 npm.community#6658
              npm.community#6069 npm.community#9323 Fix the regression where
              random config values in a .npmrc file are not passed to lifecycle
              scripts, breaking build processes which rely on them. (@isaacs)
    8b85eaa47 save files with inferred ownership rather than relying on SUDO_UID
              and SUDO_GID. (@isaacs)
    b7f6e5f02 Infer ownership of shrinkwrap files (@isaacs)
    54b095d77 #235 Add spec to dist-tag remove function (@theberbie)

    DEPENDENCIES
    dc8f9e52f pacote@9.5.7: Infer the ownership of all unpacked files in
              node_modules, so that we never have user-owned files in root-owned
              folders, or root-owned files in user-owned folders. (@isaacs)
    bb33940c3 cmd-shim@3.0.0:
        9c93ac3 #2 npm#3380 Handle environment variables properly (@basbossink)
        2d277f8 #25 #36 #35 Fix 'no shebang' case by always providing $basedir
                in shell script (@igorklopov)
        adaf20b #26 Fix $* causing an error when arguments contain parentheses
                (@satazor)
        49f0c13 #30 Fix paths for MSYS/MINGW bash (@dscho)
        51a8af3 #34 Add proper support for PowerShell (@ExE-Boss)
        4c37e04 #10 Work around quoted batch file names (@isaacs)
    a4e279544 npm-lifecycle@3.1.3 (@isaacs):
        fail properly if uid-number raises an error
    7086a1809 libcipm@4.0.3 (@isaacs)
    8845141f9 read-package-json@2.1.0 (@isaacs)
    51c028215 bin-links@1.1.3 (@isaacs)
    534a5548c read-cmd-shim@1.0.3 (@isaacs)
    3038f2fd5 gentle-fs@2.2.1 (@isaacs)
    a609a1648 graceful-fs@4.2.2 (@isaacs)
    f0346f754 cacache@12.0.3 (@isaacs)
    ca9c615c8 npm-pick-manifest@3.0.0 (@isaacs)
    b417affbf pacote@9.5.8 (@isaacs)

    TESTS
    b6df0913c #228 Proper handing of /usr/bin/node lifecycle-path test (@olivr70)
    aaf98e88c npm-registry-mock@1.3.0 (@isaacs)
@BethGriggs
Copy link
Member

I believe this issue was fixed as of Node.js v10.17.0 where npm was updated to v6.11.3. The specific npm fix was released in npm v6.11.0 (#29231).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
npm Issues and PRs related to the npm client dependency or the npm registry.
Projects
None yet
Development

No branches or pull requests

7 participants