node-gyp very slow on Windows arm64 (MSBuild)

[dennisameling]

Official Windows arm64 support is available as of Node 20.0.0, so usage on this platform will become more common soon.

• Node Version: Node: 20.0.0 / NPM: 9.6.4 / node-gyp: 9.3.1
• Platform:

OS Name:                   Microsoft Windows 11 Pro
OS Version:                10.0.22621 N/A Build 22621
System Type:               ARM64-based PC

• Compiler:

MSBuild version 17.5.0+6f08c67f3 for .NET Framework
17.5.0.10706

Microsoft (R) C/C++ Optimizing Compiler Version 19.35.32215 for ARM64
Copyright (C) Microsoft Corporation.  All rights reserved.

• Module: sharp (https://github.com/lovell/sharp)

Verbose output (from npm or node-gyp):

> node-gyp rebuild --verbose

gyp info it worked if it ends with ok
gyp verb cli [
gyp verb cli   'C:\\Program Files\\nodejs\\node.exe',
gyp verb cli   'C:\\repos\\sharp\\node_modules\\node-gyp\\bin\\node-gyp.js',
gyp verb cli   'rebuild',
gyp verb cli   '--verbose'
gyp verb cli ]
gyp info using node-gyp@9.3.1
gyp info using node@20.0.0 | win32 | arm64
gyp verb command rebuild []
gyp verb command clean []
gyp verb clean removing "build" directory
gyp verb command configure []
gyp verb find Python Python is not set from command line or npm configuration
gyp verb find Python Python is not set from environment variable PYTHON
gyp verb find Python checking if "python3" can be used
gyp verb find Python - executing "python3" to get executable path
gyp verb find Python - "python3" is not in PATH or produced an error
gyp verb find Python checking if "python" can be used
gyp verb find Python - executing "python" to get executable path
gyp verb find Python - executable path is "C:\Program Files\Python311-arm64\python.exe"
gyp verb find Python - executing "C:\Program Files\Python311-arm64\python.exe" to get version
gyp verb find Python - version is "3.11.2"
gyp info find Python using Python version 3.11.2 found at "C:\Program Files\Python311-arm64\python.exe"
gyp verb get node dir no --target version specified, falling back to host node version: 20.0.0
gyp verb command install [ '20.0.0' ]
gyp verb install input version string "20.0.0"
gyp verb install installing version: 20.0.0
gyp verb install --ensure was passed, so won't reinstall if already installed
gyp verb install version is already installed, need to check "installVersion"
gyp verb got "installVersion" 9
gyp verb needs "installVersion" 9
gyp verb install version is good
gyp verb get node dir target node version installed: 20.0.0
gyp verb build dir attempting to create "build" dir: C:\repos\sharp\build
gyp verb build dir "build" dir needed to be created? Yes
gyp verb find VS msvs_version not set from command line or npm config
gyp verb find VS VCINSTALLDIR not set, not running in VS Command Prompt
gyp verb find VS checking VS2022 (17.5.33424.131) found at:
gyp verb find VS "C:\Program Files\Microsoft Visual Studio\2022\Community"
gyp verb find VS - found "Visual Studio C++ core features"
gyp verb find VS - found VC++ toolset: v143
gyp verb find VS - found Windows SDK: 10.0.22621.0
gyp info find VS using VS2022 (17.5.33424.131) found at:
gyp info find VS "C:\Program Files\Microsoft Visual Studio\2022\Community"
gyp info find VS run with --verbose for detailed information
gyp verb build/config.gypi creating config file
gyp verb build/config.gypi writing out config file: C:\repos\sharp\build\config.gypi
gyp verb config.gypi checking for gypi file: C:\repos\sharp\config.gypi
gyp verb common.gypi checking for gypi file: C:\repos\sharp\common.gypi
gyp verb gyp gyp format was not specified; forcing "msvs"
gyp info spawn C:\Program Files\Python311-arm64\python.exe
gyp info spawn args [
gyp info spawn args   'C:\\repos\\sharp\\node_modules\\node-gyp\\gyp\\gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'msvs',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\repos\\sharp\\build\\config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\repos\\sharp\\node_modules\\node-gyp\\addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   'C:\\Users\\denni\\AppData\\Local\\node-gyp\\Cache\\20.0.0\\include\\node\\common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=C:\\Users\\denni\\AppData\\Local\\node-gyp\\Cache\\20.0.0',
gyp info spawn args   '-Dnode_gyp_dir=C:\\repos\\sharp\\node_modules\\node-gyp',
gyp info spawn args   '-Dnode_lib_file=C:\\\\Users\\\\denni\\\\AppData\\\\Local\\\\node-gyp\\\\Cache\\\\20.0.0\\\\<(target_arch)\\\\node.lib',
gyp info spawn args   '-Dmodule_root_dir=C:\\repos\\sharp',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'C:\\repos\\sharp\\build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp verb command build []
gyp verb build type Release
gyp verb architecture arm64
gyp verb node dev dir C:\Users\denni\AppData\Local\node-gyp\Cache\20.0.0
gyp verb found first Solution file build/binding.sln
gyp verb using MSBuild: C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\MSBuild.exe
gyp info spawn C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\MSBuild.exe
gyp info spawn args [
gyp info spawn args   'build/binding.sln',
gyp info spawn args   '/nologo',
gyp info spawn args   '/p:Configuration=Release;Platform=ARM64'
gyp info spawn args ]
Building the projects in this solution one at a time. To enable parallel build, please add the "-m" switch.
Build started 22-4-2023 13:43:44.

On ARM64 devices, MS Visual Studio has three MSBuild.exes:

• C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\MSBuild.exe (x86, the one node-gyp currently uses)
• C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\MSBuild.exe (x64)
• C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\arm64\MSBuild.exe (arm64)

ARM64 devices support x86 and x64 emulation, but it's slow. For comparison, building https://github.com/lovell/sharp is ~40% slower with the x86 MSBuild (and related tools) that node-gyp uses by default:

Building https://github.com/lovell/sharp:

x86: 1m02s
arm64: 0m37s (~40% faster)

#2650 fixes this, but I wanted to create a separate issue to raise visibility/awareness. It'd be great if that MR could be reviewed soon, because it'll take a while for the ecosystem to catch up on the newer node-gyp version when that fix is merged.

Thanks in advance! 🙏🏼

[StefanStojanovic]

Since #2650 landed, this can be closed now. Thanks @dennisameling for helping to make this happen!