Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

yarn install during local development fails due to @vscode\sqlite3 #10194

Closed
shanthisagar-dhanya-kumar-db-zz opened this issue Mar 15, 2022 · 19 comments
Closed

yarn install during local development fails due to @vscode\sqlite3 #10194

shanthisagar-dhanya-kumar-db-zz opened this issue Mar 15, 2022 · 19 comments
Labels
bug Something isn't working

Comments

@shanthisagar-dhanya-kumar-db-zz
Copy link

Upgraded to the tag v0.70.1 and after which performed "yarn install" on the local development environment but fails with the below error
[5/5] Building fresh packages...
error C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@vscode\sqlite3: Command failed.
Exit code: 1
Command: node-gyp rebuild
Arguments:
Directory: C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@vscode\sqlite3
Output:
gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@16.13.0 | win32 | x64
gyp info find Python using Python version 3.8.3 found at "C:\Program Files\Anaconda3\python.exe"
gyp ERR! find VS
gyp ERR! find VS msvs_version not set from command line or npm config
gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt
gyp ERR! find VS could not use PowerShell to find Visual Studio 2017 or newer, try re-running with '--loglevel silly' for more details
gyp ERR! find VS looking for Visual Studio 2015
gyp ERR! find VS - not found
gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8
gyp ERR! find VS
gyp ERR! find VS **************************************************************
gyp ERR! find VS You need to install the latest version of Visual Studio
gyp ERR! find VS including the "Desktop development with C++" workload.
gyp ERR! find VS For more information consult the documentation at:
gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows
gyp ERR! find VS **************************************************************
gyp ERR! find VS
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Visual Studio installation to use
gyp ERR! stack at VisualStudioFinder.fail (C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:121:47)
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:74:16
gyp ERR! stack at VisualStudioFinder.findVisualStudio2013 (C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:351:14)
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:70:14
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:372:16
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\util.js:54:7
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\util.js:33:16
gyp ERR! stack at ChildProcess.exithandler (node:child_process:404:5)
gyp ERR! stack at ChildProcess.emit (node:events:390:28)
gyp ERR! stack at maybeClose (node:internal/child_process:1064:16)
gyp ERR! System Windows_NT 10.0.18363
gyp ERR! command "C:\Users\XXXXXX\Downloads\node-v16.13.0-win-x64\node.exe" "C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\@npmcli\run-script\node_modules\node-gyp\bin\node-gyp.js" "rebuild"
gyp ERR! cwd C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@vscode\sqlite3
gyp ERR! node -v v16.13.0
gyp ERR! node-gyp -v v7.1.2
gyp ERR! not ok
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
@shanthisagar-dhanya-kumar-db-zz shanthisagar-dhanya-kumar-db-zz added the bug Something isn't working label Mar 15, 2022
@shanthisagar-dhanya-kumar-db-zz shanthisagar-dhanya-kumar-db-zz changed the title yarn install @vscode\sqlite3 during local development fails yarn install during local development fails due to @vscode\sqlite3 Mar 15, 2022
@freben
Copy link
Member

freben commented Mar 15, 2022

Yeah you'll need build tooling present (visual studio in your case) to be able to use @vscode/sqlite3 as the sqlite driver.

You'll see that the main repo switched over to using better-sqlite3 instead because of these exact problems with the vscode version of the driver not having prebuilt bridges. #10132

Check out what we did in that PR. Your repo should also be able to do the same config change and see if that solves your problems.

 backend:
   database:
-    client: sqlite3
+    client: better-sqlite3

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

Now I see the below error.

[5/5] Building fresh packages...
error C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3: Command failed.
Exit code: 1
Command: prebuild-install || npm run build-release
Arguments:
Directory: C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3
Output:
prebuild-install WARN install tunneling socket could not be established, statusCode=407
node:internal/modules/cjs/loader:936
throw err;
^

Error: Cannot find module 'C:\Users\XXXXXX\Downloads\node-v16.13.0-win-x64\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
at Function.Module._load (node:internal/modules/cjs/loader:778:27)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
node:internal/modules/cjs/loader:936
throw err;
^

Error: Cannot find module 'C:\Users\XXXXXX\Downloads\node-v16.13.0-win-x64\node_modules\npm\bin\npm-cli.js'
at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
at Function.Module._load (node:internal/modules/cjs/loader:778:27)
at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
at node:internal/main/run_main_module:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

I switched to nodejs 16.14.0

[5/5] Building fresh packages...
error C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3: Command failed.
Exit code: 1
Command: prebuild-install || npm run build-release
Arguments:
Directory: C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3
Output:
prebuild-install WARN install tunneling socket could not be established, statusCode=407
(node:11772) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use node --trace-warnings ... to show where the warning was created)

better-sqlite3@7.5.0 build-release
node-gyp rebuild --release

gyp info it worked if it ends with ok
gyp info using node-gyp@7.1.2
gyp info using node@16.14.0 | win32 | x64
gyp info find Python using Python version 3.8.3 found at "C:\Program Files\Anaconda3\python.exe"
gyp http GET https://nodejs.org/download/release/v16.14.0/node-v16.14.0-headers.tar.gz
(node:7212) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
(Use node --trace-warnings ... to show where the warning was created)
gyp http 200 https://nodejs.org/download/release/v16.14.0/node-v16.14.0-headers.tar.gz
gyp http GET https://nodejs.org/download/release/v16.14.0/SHASUMS256.txt
gyp http GET https://nodejs.org/download/release/v16.14.0/win-x86/node.lib
gyp http GET https://nodejs.org/download/release/v16.14.0/win-x64/node.lib
gyp http GET https://nodejs.org/download/release/v16.14.0/win-arm64/node.lib
gyp http 404 https://nodejs.org/download/release/v16.14.0/win-arm64/node.lib
gyp http 200 https://nodejs.org/download/release/v16.14.0/win-x64/node.lib
gyp http 200 https://nodejs.org/download/release/v16.14.0/win-x86/node.lib
gyp http 200 https://nodejs.org/download/release/v16.14.0/SHASUMS256.txt
gyp ERR! find VS
gyp ERR! find VS msvs_version not set from command line or npm config
gyp ERR! find VS VCINSTALLDIR not set, not running in VS Command Prompt
gyp ERR! find VS could not use PowerShell to find Visual Studio 2017 or newer, try re-running with '--loglevel silly' for more details
gyp ERR! find VS looking for Visual Studio 2015
gyp ERR! find VS - not found
gyp ERR! find VS not looking for VS2013 as it is only supported up to Node.js 8
gyp ERR! find VS
gyp ERR! find VS **************************************************************
gyp ERR! find VS You need to install the latest version of Visual Studio
gyp ERR! find VS including the "Desktop development with C++" workload.
gyp ERR! find VS For more information consult the documentation at:
gyp ERR! find VS https://github.com/nodejs/node-gyp#on-windows
gyp ERR! find VS **************************************************************
gyp ERR! find VS
gyp ERR! configure error
gyp ERR! stack Error: Could not find any Visual Studio installation to use
gyp ERR! stack at VisualStudioFinder.fail (C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:121:47)
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:74:16
gyp ERR! stack at VisualStudioFinder.findVisualStudio2013 (C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:351:14)
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:70:14
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\find-visualstudio.js:372:16
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\util.js:54:7
gyp ERR! stack at C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules@npmcli\run-script\node_modules\node-gyp\lib\util.js:33:16
gyp ERR! stack at ChildProcess.exithandler (node:child_process:406:5)
gyp ERR! stack at ChildProcess.emit (node:events:520:28)
gyp ERR! stack at maybeClose (node:internal/child_process:1092:16)
gyp ERR! System Windows_NT 10.0.18363
gyp ERR! command "C:\Users\XXXXXX\Downloads\node-v16.14.0-win-x64\node.exe" "C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\@npmcli\run-script\node_modules\node-gyp\bin\node-gyp.js" "rebuild" "--release"
gyp ERR! cwd C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3
gyp ERR! node -v v16.14.0
gyp ERR! node-gyp -v v7.1.2
gyp ERR! not ok
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

@freben
Copy link
Member

freben commented Mar 15, 2022

Did you follow the instructions in there?

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

Looking at the commit efc73db,

did all the changes mentioned there, but still no luck

@freben
Copy link
Member

freben commented Mar 15, 2022

No I meant, did you follow the instructions in the error message you pasted?

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

It is asking to install visual studio. Is this really mandatory?
what if we are performing "yarn install" under an openshift pod?

@freben
Copy link
Member

freben commented Mar 15, 2022

In linux, it may ask for a different set of dependencies; usually build-essentials will do.

@freben
Copy link
Member

freben commented Mar 15, 2022

I found this guide too. https://github.com/JoshuaWise/better-sqlite3/blob/master/docs/troubleshooting.md

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

None of the solutions seem to be working either for windows or linux for us. Have raised issue here WiseLibs/better-sqlite3#777

@freben
Copy link
Member

freben commented Mar 15, 2022

Oh I see on the last one that the actual error was operation not permitted (EPERM) on the project file it tried to build. Are you using some form of odd mounted directories, readonly file systems, or antivirus software that interferes with stuff like this?

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

In our organization when we install softwares like MS studio, it goes under program files, and that is readonly. Else the other option is for us to put the MS studio where it has read-write on mount and use

npm config set msvs_version YYY
npm config set msbuild_path "XXX"

By doing this it may work in windows. But for linux it appears we need GCC compiler, which isn't in our OS build and not sure whether we can install the build-essentials as well. I will come back on this when I get more info

@freben
Copy link
Member

freben commented Mar 15, 2022

But this is within your own repo.

CUSTOMBUILD : error : EPERM: operation not permitted, symlink

'C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3\deps\sqlite3\sqlite3.c'
->
'C:\Users\XXXXXX\GIT_REPOSITORY_GA\backstage-app\node_modules\better-sqlite3\build\Release\obj\global_intermediate\sqlite3\sqlite3.c'

added some newlines for readability

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

Ah yes that is true. I am not sure why it is saying operation not permitted, as that is a read-write mount.

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

Looks like for windows symlink doesn't work.
WiseLibs/better-sqlite3#751.
It is quite tricky to get it working this way, as the yarn install will replace the file contents. We just need to be really quick to change it back before the step 5/5 of yarn install starts

@freben
Copy link
Member

freben commented Mar 15, 2022

Alright. Is this in WSL or not? Might be worth using that

@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

We are using gitbash. In our org using custom tools like WSL is not allowed.

@freben
Copy link
Member

freben commented Mar 21, 2022

Unfortunately I think you are out of luck in this instance, in that case. Both this package and the vscode one require compilation steps upon install, and they don't come with windows binaries so the build toolchains and a read-write filesystem is necessary while developing and building the project - IF using sqlite. The easiest path forward for you might be to use the postgres driver instead, both for local development and production. That might be a viable option, and worth trying.

I'll close this as inactionable for now, I hope that's OK. If you find any workaround or alternative driver for sqlite that works in your circumstances, we would passionately like to hear about it :/

@freben freben closed this as completed Mar 21, 2022
@shanthisagar-dhanya-kumar-db-zz
Copy link
Author

We use postgres and not sqlite in app-config.yaml as below and comment out the sqlite option.
database:
client: pg
connection: postgresql://${POSTGRES_USER}:${POSTGRES_PASSWORD}@${POSTGRES_HOST}:${POSTGRES_PORT}/postgres

But, how do we make sure that the entire sqlite engine isn't being used at all while doing "yarn install"?

@wychwitch wychwitch mentioned this issue Jul 20, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@freben @shanthisagar-dhanya-kumar-db-zz