[Medium] Fetch Native Apple Silicon Node builds for Node 16+

[charlespierce]

Closes #919

Info

• With the release of Node v16, there are now pre-built binaries available for Apple Silicon devices.
• Where possible, we should fetch and use those binaries, so that users get the best possible experience.
• However, for older versions of Node, we will need to continue to fetch the x64 binaries and rely on Rosetta.

Changes

• Updated the constants section in tool/node/mod.rs to include the constant values for Apple Silicon architecture
• Ensured that the parsing of the files metadata in the Node index will check for both the native and x64 versions of Node.
• Customized archive_filename on Apple Silicon builds to check the Node major version and fetch either a native or x64 build of Node depending on whether it is greater than or equal to 16.

Tested

• Our test suite currently doesn't execute on an Apple Silicon machine, so we don't have a way to automatically test these changes (opened [Blocked] Run CI test suite on Apple Silicon devices #973 to track that issue separately, currently blocked on GitHub Actions support).
• Could someone with access to an M1 machine (cc @chriskrycho ?) run a test to confirm this properly fetches Node 16+ and 15-?

Notes

• Per this comment, Node does not support Apple Silicon (even when building from source) prior to Node 15, and only provides pre-built binaries as of Node 16. So there shouldn't be any LTS releases of Node that have pre-built binaries but a major version less than 16.

[chriskrycho]

chris@Razorback:~/d/t/volta-test
❯ volta pin node@15
success: pinned node@15.14.0 (with npm@7.7.6) in package.json

chris@Razorback:~/d/t/volta-test
❯ file (volta which node)
/Users/chris/.volta/tools/image/node/15.14.0/bin/node: Mach-O 64-bit executable x86_64

chris@Razorback:~/d/t/volta-test
❯ volta pin node@16
success: pinned node@16.0.0 (with npm@7.10.0) in package.json

chris@Razorback:~/d/t/volta-test
❯ file (volta which node)
/Users/chris/.volta/tools/image/node/16.0.0/bin/node: Mach-O 64-bit executable arm64

🎉

[chriskrycho]

I have three comments:

1. This is awesome! And wow is Node running native on M1 fast.
2. Thank you for all the clear comments along the way!
3. The fact that this also passes around Version instead of &str is 🤩

!

[alcuadrado]

Should this be working now? I'm using volta 1.0.4 and volta install node@16 installs an x86_64 version.

[chriskrycho]

Hmm. It should, yes, and I’ve seen it fetch the right thing. 🤔 Can you provide further information on Volta itself, the Node installed, etc.?

[alcuadrado]

It may be because I fetched node@16 before upgrading volta.

More info:

pato@parm:~% volta --version
1.0.4
pato@parm:~% volta install node@16
success: installed and set node@16.0.0 as default
pato@parm:~% file `which node`
/Users/pato/.volta/bin/node: Mach-O 64-bit executable x86_64
pato@parm:~% uname -a
Darwin parm 20.3.0 Darwin Kernel Version 20.3.0: Thu Jan 21 00:06:51 PST 2021; root:xnu-7195.81.3~1/RELEASE_ARM64_T8101 x86_64

[alcuadrado]

Also, this didn't help:

pato@parm:~% rm -rf ~/.volta
pato@parm:~% curl https://get.volta.sh | bash
pato@parm:~% volta install node@16
pato@parm:~% file `which node`
/Users/pato/.volta/bin/node: Mach-O 64-bit executable x86_64

[chriskrycho]

HMMMM. A couple more investigatory questions:

• what does file `which volta` show?
• are you running in a terminal (emulator) which is itself running under Rosetta 2?

[alcuadrado]

Np, happy to help.

pato@parm:~% file `which volta`
/Users/pato/.volta/bin/volta: Mach-O 64-bit executable x86_64

I'm running these commands using Alacritty, which launches zsh, which launches tmux, which launches zsh 😅

pato@parm:~% file `which alacritty`
/opt/homebrew/bin/alacritty: Mach-O 64-bit executable x86_64
pato@parm:~% file `which zsh`
/opt/homebrew/bin/zsh: Mach-O 64-bit executable arm64
pato@parm:~% file `which tmux`
/opt/homebrew/bin/tmux: Mach-O 64-bit executable arm64

[charlespierce]

I'm not at all an expert in how Rosetta works, but I'm wondering if starting with Alacritty (which appears to be an x64 program) puts everything underneath it into "Compat mode", meaning even though zsh is a native binary, it runs it with some settings that emulate an x64 environment?

For some context, which node should always point to the node shim, which itself is a symlink to the volta-shim executable. Since that is installed by the Volta installer, it seems (from your previous testing) that doing curl https://get.volta.sh | bash is getting the x64 version instead of the native M1 version. The install script uses uname -m to detect the architecture here, so I'd be interested to see what the output of that is when you run it in the same terminal environment (alacritty).

Another thing worth checking, since the install involves piping into bash, would be the output of file `which bash` , to make sure there isn't some weird x64 version of Bash that's breaking the install script.

[alcuadrado]

I know almost nothing about Rosetta tbh.

I found some interesting things:

This is run under zsh, which according to activity monitor is running without rosetta:

pato@parm:hardhat-ignition (main ✔)% which bash
/bin/bash
pato@parm:hardhat-ignition (main ✔)% file /bin/bash
/bin/bash: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e]
pato@parm:hardhat-ignition (main ✔)% uname -m
x86_64

This seems to indicate that this comment is right

The install script uses uname -m to detect the architecture here, so I'd be interested to see what the output of that is when you run it in the same terminal environment (alacritty).

So I opened Terminal.app, running zsh without tmux (both are aarm64), and this worked:

pato@parm:~% rm -rf ~/.volta
pato@parm:~% curl https://get.volta.sh | bash
pato@parm:~% volta install node@16
success: installed and set node@16.0.0 (with npm@7.10.0) as default
pato@parm:~%  file `which node`
/Users/pato/.volta/bin/node: Mach-O 64-bit executable arm64

Now on Alacritty:

pato@parm:~% volta install node@15
success: installed and set node@15.14.0 (with npm@7.7.6) as default
pato@parm:~% file `which node`
/Users/pato/.volta/bin/node: Mach-O 64-bit executable arm64
pato@parm:~% file `which volta`
/Users/pato/.volta/bin/volta: Mach-O 64-bit executable arm64

[charlespierce]

Okay, great! Yeah, it seems like something in the setup (my guess is alacritty at the start, but I don't have enough expertise to know for sure) was setting things up in a way that misled the Volta installer, causing it to fetch the x64 Mac version, instead of the native one. Now that you have the proper native one, however, Volta should work properly even in the Alacritty environment (since it uses compile-time flags to determine the architecture, rather than detecting it at runtime). So I believe you should be good to go.

One last thing, as I mentioned which node should always be pointing at the Volta shim file, which will always be arm64, regardless of which Node version is actually selected. If you want to introspect the actual Node binary that will be used, you can take advantage of volta which (a command designed to "unwrap" the shims and point you to the actual file that will execute): file `volta which node` should give you the info about the actual binary. Since it looks like things are set up properly now, I would expect that one to also be arm64.

[alcuadrado]

Oh, I missed that part! Yeah, it's arm64 for node@16, and x86_64 for @15.