Turbo installation via npm ci fails on Windows with package-lock.json created on another platform

[natesilva]

What version of Turborepo are you using?

1.4.3

What package manager are you using / does the bug impact?

npm

What operating system are you using?

Windows

Describe the Bug

This happens when running npm ci on a Windows system, on a repo that was originally created on a Mac. My guess is that the package-lock.json has turbo-darwin-arm64 instead of turbo-windows-64 (after all that’s an optional dependency) -- but the install script for turbo expects the platform-specific dependency to be there.

Workaround: Our developers were able to work around this by first running npm i turbo, before running npm ci.

I believe this may also be an issue if we run turbo in our CI since it’s yet another platform (Linux ARM64) that is different from our Mac & Windows dev systems. I have not tried that.

Similar to issue #947.

PS C:\code\apps-monorepo> npm ci
npm ERR! code 1
npm ERR! path C:\code\apps-monorepo\node_modules\turbo
npm ERR! command failed
npm ERR! command C:\Windows\system32\cmd.exe /d /s /c node install.js
npm ERR! [turbo] Failed to find package "turbo-windows-64" on the file system
npm ERR!
npm ERR! This can happen if you use the "--no-optional" flag. The "optionalDependencies"
npm ERR! package.json feature is used by turbo to install the correct binary executable
npm ERR! for your current platform. This install script will now attempt to work around
npm ERR! this. If that fails, you need to remove the "--no-optional" flag to use turbo.
npm ERR!
npm ERR! node:internal/modules/cjs/loader:956
npm ERR!   const err = new Error(message);
npm ERR!               ^
npm ERR!
npm ERR! Error: Cannot find module 'turbo'
npm ERR! Require stack:
npm ERR! - C:\code\apps-monorepo\node_modules\turbo\node-platform.js
npm ERR! - C:\code\apps-monorepo\node_modules\turbo\install.js
npm ERR!     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:956:15)
npm ERR!     at Function.resolve (node:internal/modules/cjs/helpers:108:19)
npm ERR!     at downloadedBinPath (C:\code\apps-monorepo\node_modules\turbo\node-platform.js:52:44)
npm ERR!     at checkAndPreparePackage (C:\code\apps-monorepo\node_modules\turbo\install.js:277:15)
npm ERR!     at Object.<anonymous> (C:\code\apps-monorepo\node_modules\turbo\install.js:302:1)
npm ERR!     at Module._compile (node:internal/modules/cjs/loader:1126:14)
npm ERR!     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1180:10)
npm ERR!     at Module.load (node:internal/modules/cjs/loader:1004:32)
npm ERR!     at Function.Module._load (node:internal/modules/cjs/loader:839:12)
npm ERR!     at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12) {
npm ERR!   code: 'MODULE_NOT_FOUND',
npm ERR!   requireStack: [
npm ERR!     'C:\\code\\apps-monorepo\\node_modules\\turbo\\node-platform.js',
npm ERR!     'C:\\code\\apps-monorepo\\node_modules\\turbo\\install.js'
npm ERR!   ]
npm ERR! }

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\nate\AppData\Local\npm-cache\_logs\2022-08-23T18_31_16_387Z-debug-0.log
PS C:\code\apps-monorepo>

Expected Behavior

npm ci should complete without any prerequisites

To Reproduce

If turbo-windows-64 is available (globally or in the local node_modules) then you don’t see this error. For example, after running npm i turbo, then package-lock.json will be updated with the platform-specific version of turbo.

You need to be in a clean repo that has a package-lock.json built on another platform and no node_modules, and turbo not installed globally.

[vugar005]

I had the same issue on circleci which was using Linux platform.
So as a temporary workaround I used npm install --no-package-lock.
But of course, don't use it in production since package-lock.json is not going to be respected.

[jeffreys-cat]

I had the same issue on circleci which was using Linux platform. So as a temporary workaround I used npm install --no-package-lock. But of course, don't use it in production since package-lock.json is not going to be respected.

It works for me

[alfergus0n]

I believe this is happening for mac with linux as well but on Github Actions

i get the infamous npm ERR! [turbo] Failed to find package "turbo-linux-64" on the file system

not the first time either, had been working fine recently until the lock was touched

Been experiencing this issue for months now. It'll creep up every now and then. Though the fix in github actions is to change the version of turborepo, which isn't a solution. just a mickey-mouse fix.

EDIT: for me apparently at some point or another inside package-lock.json, npm was removing critical fields. Why? I'm not entirely sure. But that seems to be the culprit on my end at least, in my case it was Linux: #1263 (comment)

[soulchild]

One thing I noticed is that after doing the following to recreate my package-lock.json, every Turbo binary dependency except turbo-darwin-arm64 (which is correct for my Mac) is wiped from the package-lock.json and my project stops building on our CI with the Failed to find package "turbo-linux-64" on the file system error.

rm package-lock.json
npm install

After running npm install turbo again, all other (possible) platform binaries, including turbo-linux-64 are added back to the package-lock.json and the build on the CI is fine again. It's as if running just npm install pins the optional binary dependency to the one necessary for the system turbo was installed on, whereas explicitly installing Turbo with npm install turbo includes all available binary deps.

[chris-olszewski]

@soulchild What version of node/npm are you using? I've been trying to repro this on my machine since it sounds like a promising lead, but I haven't gotten the behavior you're describing.

[soulchild]

@chris-olszewski I'm on node v16.17.1 and npm v8.15.0. Thanks for investigating!

[gabeidx]

I was able to mitigate the problem on Github Actions with the following:

- run: npm install
+ run: npm install --no-save turbo-linux-64 && npm install

node 18.12.1
npm 9.1.2

[jeresig]

For what it's worth, I saw a similar error to this using pnpm - I was able to fix it by upgrading my copy of pnpm from 7.0.0-rc.8 to 7.17.1. At which point running pnpm install ran without incident.

[chris-olszewski]

Hi everyone, sharing some discoveries. It seems this is a pretty well reported issue with npm (npm/cli#4828) with a PR that should fix it. Not a solution, but the bug report does provide a work around:

That is, it then generates a package-lock.json with only the platform-specific dependency that was installed on this machine, and not with the other optional dependencies that should also be listed. If you delete both node_modules AND package-lock.json, and then re-run npm install, it generates the correct lockfile with all of those optional dependencies listed.

---

@jeresig

For what it's worth, I saw a similar error to this using pnpm

I think you might've hit pnpm/pnpm#4961 which would've been fixed by upgrading to 7.5.0 or later.

[chuanqisun]

I have observed the same with turbo@1.7.0

My lock file is generated on linux, and the CI/CD agent is running Windows.

I can confirm that the turbo-linux-64 is in the lock file and turbo-windows-64 is not in the lock file, causing the CI/CD to throw [turbo] Failed to find package "turbo-windows-64" on the file system

[alexkrolick]

Note that you can add multiple architectures to the package-lock.json:

npm install --save-dev turbo --arch=x64 --platform=linux
npm install --save-dev turbo --arch=arm64 --platform=darwin

This should allow npm ci to work.

[nathanhammond]

There's a good discussion here with good workarounds for upstream issues. I'm going to close and lock this. If you encounter similar issues start by regenerating your lockfile with no node_modules on disk.

This is not intended to prevent reporting issues, it's so that, if a new issue is introduced, we can differentiate. If you still have issues, please feel encouraged to open another issue!