Add Windows ARM64 + Apple Silicon support

[dennisameling]

Closes #9034

Description

Requirements for cross-compilation from x64 to arm64:

• (✔️ merged) Let Dugite download 32-bit Dugite Native (including Git for Windows) as Git isn't available natively on Windows arm64 Support NPM cross compilation npm_config_arch + Windows arm64 support dugite#392
• (✔️ 7.1.0 released which contains the fix) Add prebuild Electron arm64 to Keytar Add prebuild Electron for arm64 atom/node-keytar#319 + fix native arm64 build on Windows Update binding.gyp to fix "napi.h not found" error atom/node-keytar#325
• (✔️ merged) Switch to using the dart version of sass instead of node-sass Switch to using the dart version of sass instead of node-sass #11019
• (✔️ merged) Update Electron-winstaller Squirrel version to v2+ for arm64 support feat!: update to Squirrel 2.0.1 electron/windows-installer#361

Building for Windows ARM64

Run the following in Windows Command Prompt (NOT PowerShell):
set npm_config_arch=arm64
set TARGET_ARCH=arm64
yarn
yarn build:prod

Running tests

In my test build, all tests have passed: yarn test:unit:cov, yarn test:script:cov, yarn test:integration.

Getting the installer to work

Squirrel has been updated to support Windows arm64, now we're only waiting for electron-winstaller to approve the PR that updates Squirrel: electron/windows-installer#361

Screenshots

No UI changes, but just to show that building + launching + login on Windows ARM works:

Release notes

Notes: Add Windows ARM build support

[dennisameling]

Azure Pipelines build + tests now succeed on my ARM64 device (Surface Pro X) 🎉 https://dev.azure.com/fits4all/github-desktop/_build/results?buildId=507&view=logs&j=7df51adc-23f7-5bdc-263d-a9d9817a26b1

Thing is, the GitHub Desktop team would either need to get their hands on a Windows ARM64 device (like the Surface Pro X, the Electron and VS Code teams are doing this) or we could choose not to run tests in CI for ARM64 devices (for the time being) and just cross-compile to create an ARM64 build.

I already created a feature request for the Azure DevOps team to start offering a hosted Windows ARM64 agent, but that might take a while, if they will consider doing that at all. https://developercommunity.visualstudio.com/idea/1015752/please-add-a-windows-arm64-hosted-agent.html

[marswe]

Nice job! If you want an ARM64 version of node to speed up the tests, you can find pre-built versions at (for example) https://unofficial-builds.nodejs.org/download/release/v12.15.0/

[dennisameling]

@marswe
Thanks for the suggestion! I tried that already, but ran into node-gyp errors unfortunately (it's CRAZY FAST 😍 until the point where it errors):

Details

gyp verb tarball done parsing tarball
gyp verb on Windows; need to download `node.lib`...
gyp verb 32-bit node.lib dir C:\Users\denni\.node-gyp\12.15.0\ia32
gyp verb 64-bit node.lib dir C:\Users\denni\.node-gyp\12.15.0\x64
gyp verb `node.lib` 32-bit url https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/win-x86/node.lib
gyp verb `node.lib` 64-bit url https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/win-x64/node.lib
gyp verb check download content checksum, need to download `SHASUMS256.txt`...
gyp verb checksum url https://unofficial-builds.nodejs.org/download/release/v12.15.0/SHASUMS256.txt
gyp http GET https://unofficial-builds.nodejs.org/download/release/v12.15.0/SHASUMS256.txt
gyp verb streaming 32-bit node.lib to: C:\Users\denni\.node-gyp\12.15.0\ia32\node.lib
gyp http GET https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/win-x86/node.lib
gyp verb streaming 64-bit node.lib to: C:\Users\denni\.node-gyp\12.15.0\x64\node.lib
gyp http GET https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/win-x64/node.lib
gyp http 404 https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/win-x86/node.lib
gyp WARN install got an error, rolling back install
gyp verb command remove [ '12.15.0' ]
gyp verb remove using node-gyp dir: C:\Users\denni\.node-gyp
gyp verb remove removing target version: 12.15.0
gyp verb remove removing development files for version: 12.15.0
gyp http 404 https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/win-x64/node.lib
gyp http 200 https://unofficial-builds.nodejs.org/download/release/v12.15.0/SHASUMS256.txt
gyp verb checksum data {"node-v12.15.0-headers.tar.gz":"48e8ba40339e6cb9edc820f320b690b3401643e2c6bd36a7f2267ebf84cf98f2","node-v12.15.0-headers.tar.xz":"44217fee8c0d003888783f65c36eec871d36b93d7347333b659633e30cbd85db","node-v12.15.0-linux-armv6l.tar.gz":"f85fab5434149dedc032e999df694d27a13e43ae08e81abc85d653d7ce5d9432","node-v12.15.0-linux-armv6l.tar.xz":"a4e32f110376b61bd181e5a886f5e63f99630e3bfa24aee96f487a4d2fec7f70","node-v12.15.0-linux-x64-musl.tar.gz":"8b366362c825a585d33e626bacfbe79bdb961390920930f8582dee17e01b5f4b","node-v12.15.0-linux-x64-musl.tar.xz":"150c5f3283cf07953cd2a55490c5e6eadefe40f80e38fd4f19dd516896bc5cf9","node-v12.15.0-linux-x86.tar.gz":"620d4ffb5a3cd6e2c69c96d427f4cf789c55b4a1c34639ab722ed1c0520e9de3","node-v12.15.0-linux-x86.tar.xz":"5206862d0c5798f5db5e2156974199b054eebd2a3414b0924a6d16b38755ea4a","node-v12.15.0-win-arm64.7z":"900308f470014f96b5b8c390aac6da8871648f6f9c478951deff9ce225585692","node-v12.15.0-win-arm64.zip":"e9c0ac5131839ddb0f933605c7ece7b4aa132a87ca9c3ed47851f436b5aab76d","win-arm64/node.exe":"0f32ee20246341d06a4f977a2bc61faeed465b51a81a18ddef5efde232377ca6","win-arm64/node.lib":"2b7fde83b1f59c08709a0af638ee59463d0099217cbc33d1b4e421416860d01b","win-arm64/node_pdb.7z":"d2a09669ecccff5ead52a8aab1f0a3cf6199f24b224c23ae8048cce70438ea04","win-arm64/node_pdb.zip":"b364c4a703406338d93e29261998fb994b6e1e099208e5b6e7e3e80344e5319b"}
gyp verb download contents checksum {"node-v12.15.0-headers.tar.gz":"48e8ba40339e6cb9edc820f320b690b3401643e2c6bd36a7f2267ebf84cf98f2"}       
gyp verb validating download checksum for node-v12.15.0-headers.tar.gz (48e8ba40339e6cb9edc820f320b690b3401643e2c6bd36a7f2267ebf84cf98f2 == 48e8ba40339e6cb9edc820f320b690b3401643e2c6bd36a7f2267ebf84cf98f2)
gyp ERR! configure error
gyp ERR! stack Error: 404 status code downloading 32-bit node.lib
gyp ERR! stack     at Request. (C:\repos\desktop\node_modules\node-gyp\lib\install.js:344:20)
gyp ERR! stack     at Request.emit (events.js:228:7)
gyp ERR! stack     at Request.onRequestResponse (C:\repos\desktop\node_modules\node-gyp\node_modules\request\request.js:1066:10)
gyp ERR! stack     at ClientRequest.emit (events.js:223:5)
gyp ERR! stack     at HTTPParser.parserOnIncomingClient [as onIncoming] (_http_client.js:592:27)
gyp ERR! stack     at HTTPParser.parserOnHeadersComplete (_http_common.js:116:17)
gyp ERR! stack     at TLSSocket.socketOnData (_http_client.js:465:22)
gyp ERR! stack     at TLSSocket.emit (events.js:223:5)
gyp ERR! stack     at addChunk (_stream_readable.js:309:12)
gyp ERR! stack     at readableAddChunk (_stream_readable.js:290:11)
gyp ERR! System Windows_NT 10.0.18362
gyp ERR! command "C:\\Program Files (x86)\\nodejs\\node.exe" "C:\\repos\\desktop\\node_modules\\node-gyp\\bin\\node-gyp.js" "rebuild" "--verbose" "--libsass_ext=" "--libsass_cflags=" "--libsass_ldflags=" "--libsass_library="
gyp ERR! cwd C:\repos\desktop\node_modules\node-sass
gyp ERR! node -v v12.15.0

I'll patiently wait until nodejs/node#32867 is approved & merged. I managed to do some performance tweaking on my Surface Pro X and got the build + test time down to a total of ~32 mins. I think that's fine for the time being.

[dennisameling]

To make the installer work on Windows ARM64, we'd need the PR attached in Squirrel/Squirrel.Windows#1616 to be merged, that would be the last major blocker to be able to build GitHub Desktop for ARM64 + an installer (not considering ARM64 tests in the CI pipeline, see #9691 (comment))

[marswe]

@dennisameling
Ah, right, that's node-sass, which has a dependency on an old version of node-gyp that doesn't work with ARM64 node. You may be able to use node-sass from their v5 branch, where they've bumped the node-gyp version, but I haven't tried it with Github Desktop.

[dennisameling]

Just a quick update that this PR is waiting for Squirrel 2.0 to be released (Squirrel/Squirrel.Windows#1623). When that's done, I'll update this PR accordingly, will also fix the merge conflicts then.

[niik]

Just a quick update that this PR is waiting for Squirrel 2.0 to be released (Squirrel/Squirrel.Windows#1623). When that's done, I'll update this PR accordingly, will also fix the merge conflicts then.

Awesome @dennisameling, thanks for letting us know. For context we've recently switched from appveyor and circle to GitHub Actions (#9680). Unfortunately it doesn't seem like GitHub Actions support self-hosted ARM64 runners on Windows (actions/runner-images#768) yet.

[dennisameling]

@niik That should be OK, an amd64 host can also build ARM64 packages. AFAIK there aren't any cloud providers that offer ARM64 hosts yet. Only thing that isn't possible then is to run automated tests on ARM64, as that would need an ARM64 device. Hope that's OK for the time being; tests can of course run on amd64 as they do now.

[say25]

Squirrel 2.0 seems to have been released which should unblock this effort 🎉

[dennisameling]

Now all that's left is for electron-winstaller to create a new version which has the updated Squirrel version in it. There was a PR for that, but it got closed due to recently discovered issues with Squirrel 2.0: electron/windows-installer#354

So we'll have to wait a little longer for this I'm afraid 😞

[dennisameling]

@niik update: just rebased this PR and updated CI to GitHub Actions.

I was able to build GitHub Desktop for Windows arm64, but had to do some hacking with NodeJS/Electron binaries to get it to work, I think mostly because keytar and registry-js don't have prebuilt arm64 binaries yet (see https://github.com/desktop/desktop/pull/9691/checks?check_run_id=1190245404). I'm not sure what has changed in the last few months, but back in May those prebuilt binaries weren't necessary.

Here's the binary for anyone interested: https://github.com/dennisameling/desktop/releases/tag/2.5.6-arm64

I'll do some more investigation later. We anyway need to wait for some dependencies like electron/windows-installer#361 to be updated first, so expect that to take another month or two. At least this PR is up-to-date with the latest changes to GitHub Desktop now 👍

We might be able to combine this effort with releasing GitHub Desktop for Apple Silicon. The NodeJS and Electron teams are currently working on this as well.

[niik]

@dennisameling Thank you so much for your continued work on this! ❤️

https://github.com/desktop/registry-js I'd be happy to review a PR to add prebuilt arm64 binaries there but I imagine that would be similarly blocked by the lack of node arm64 builds.

[dennisameling]

Yeah we'd run into the same issues there. Last NodeJS version that had arm64 binaries was 12.15.0 (https://unofficial-builds.nodejs.org/download/release/v12.15.0/win-arm64/). I've created an issue against nodejs/build to ask for new builds with arm64 binaries, since an issue regarding those was solved a few months ago.

For now, we might get away with downloading the arm64 binary for NodeJS 12.15.0 and putting it in C:\\Users\\runneradmin\\AppData\\Local\\node-gyp\\Cache\\12.15.0\\arm64\\node.lib in CI (similar to what's mentioned here), but I'd rather not do that as it will make things very hacky and not future-proof.

Let's wait a bit for a reply to my question in the nodejs/build repo, then happy to take next steps here. To be continued 😊

[dennisameling]

Update: I can't move forward with this until NodeJS publishes native ARM64 builds for Windows. They currently can't reliably run tests on ARM64 in their CI (they have two Surface Pro Xs in their office I believe), so they are waiting for Azure Pipelines/GitHub to release hosted ARM64 agents.

Thing is, there's currently not a single CI provider that offers hosted Windows ARM64 machines AFAIK, so it might take multiple months before we can proceed with this 😞

Please, for anyone reading this, upvote this feature request for hosted Windows ARM64 agents in Azure DevOps and Windows ARM64 VMs in Azure, so that the NodeJS team can move on and release official ARM64 builds for Windows.

I'll update this PR as soon as things are moving on the NodeJS side 👍

[trulyronak]

Great news! Is there a way for me to download this to use locally (while I wait for #11683 to be merged)

[sergiou87]

I think the only option is that you build the app yourself locally 😅

I'll let you know when we build and deploy our own arm64 builds 😄

[trulyronak]

Gotcha haha - are there steps on how to do that? (And would I just pull to get updates)

[sergiou87]

You have all the info you need here: https://github.com/desktop/desktop/blob/development/docs/contributing/setup.md

Not sure if anything changes for arm64, though 😅  Please, let me know if any step is missing for M1!

[say25]

@trulyronak a PR to enhance the docs if needed would be greatly appreciated 😄.

[sergiou87]

@dennisameling @trulyronak (or anyone else) if you want to give this a try:

• Windows arm64: https://s3.amazonaws.com/github-desktop/releases/2.7.3-test1-34646e93/GitHubDesktopSetup-arm64.exe
• macOS arm64: https://s3.amazonaws.com/github-desktop/releases/2.7.3-test1-34646e93/GitHubDesktop-arm64.zip

Please, let me know if it works (and which specific platform you tested 😅 )

[dennisameling]

@sergiou87 can confirm it works on Windows ARM64! 🚀

[dennisameling]

One thing I noticed while clicking around is that it doesn't recognize VS Code on Windows ARM64 (that app also has a native version which I have installed). Would you mind pointing me at the code that looks for the VS Code installation? Happy to have a look at it!

[sergiou87]

Yay!! Thank you for testing it 😄

What you're looking for should be in https://github.com/desktop/desktop/blob/development/app/src/lib/editors/win32.ts

You can read more about it in https://github.com/desktop/desktop/blob/development/docs/technical/editor-integration.md (which might need to be updated if Windows for arm64 requires different Registry keys to check 😅 )

[dennisameling]

Bingo - it's {D9E514E7-1A56-452D-9337-2990C0DC4310}_is1 on ARM64 for VS Code stable User installation. So the file does need to be edited. I'll also have a look at the system-wide install as well as Insiders build etc. - hope to have a look this weekend and provide a PR 😊

[sergiou87]

You're the best, thank you!!! 🙌 🙇‍♂️

[firat-can-slnk]

@dennisameling @trulyronak (or anyone else) if you want to give this a try:

• Windows arm64: https://s3.amazonaws.com/github-desktop/releases/2.7.3-test1-34646e93/GitHubDesktopSetup-arm64.exe
• macOS arm64: https://s3.amazonaws.com/github-desktop/releases/2.7.3-test1-34646e93/GitHubDesktop-arm64.zip

Please, let me know if it works (and which specific platform you tested 😅 )

I tried to open the macOS arm64 version (on my M1 MacBook Air) but the main window does not show up except when I click on something on the menu bar like File->New Repository but the window is just white and blank.

Toggling the developer options shows the following error:

The app is in the applications folder and runs natively

I don't really know how to test it though and the logs are empty.
Hope that helps in some way.

[KishanBagaria]

Looks like keytar was built for x64 instead of arm64. Running file $BUILD_DIR/**/*.node will list the architecture of all built native node modules.

[dennisameling]

@waffelcop good catch - I helped Keytar to release arm64 builds of their native plugin a while ago, but we had to revert it due to bugs in the build process: atom/node-keytar#346 - let me have a look at this in the coming days. GH Actions doesn't support MacOS 11 yet (actions/runner-images#2486) which has the option to build native binaries for arm64, but there's a trick I applied elsewhere to have it use the proper SDK for building arm64 as well. Will report back here in the coming weekend!

[KishanBagaria]

We have been building native deps (including keytar) for arm64 Macs on a x64 Mac with electron-rebuild --module-dir app --arch arm64

[firat-can-slnk]

Looks like keytar was built for x64 instead of arm64. Running file $BUILD_DIR/**/*.node will list the architecture of all built native node modules.

file /Applications/GitHub Desktop.app/Contents/Resources/app/fs_admin.node and file /Applications/GitHub Desktop.app/Contents/Resources/app/keytar.node return Mach-O 64-bit bundle x86_64.
It's the .app that @sergiou87 provided

[trulyronak]

@dennisameling @trulyronak (or anyone else) if you want to give this a try:

• Windows arm64: s3.amazonaws.com/github-desktop/releases/2.7.3-test1-34646e93/GitHubDesktopSetup-arm64.exe
• macOS arm64: s3.amazonaws.com/github-desktop/releases/2.7.3-test1-34646e93/GitHubDesktop-arm64.zip

Please, let me know if it works (and which specific platform you tested 😅 )

I tried to open the macOS arm64 version (on my M1 MacBook Air) but the main window does not show up except when I click on something on the menu bar like File->New Repository but the window is just white and blank.

Toggling the developer options shows the following error:

The app is in the applications folder and runs natively

I don't really know how to test it though and the logs are empty.
Hope that helps in some way.

yep same issue here — I ran into this when building and running locally as well.

[dennisameling]

Have created a PR with a temporary fix for the keytar issue, more details here: #11969

This should fix the keytar issue, so we only have the fs-admin issue left. For anyone who would like to verify if the keytar module indeed has the right architecture now, please test https://github.com/dennisameling/desktop/releases/tag/2.7.3-test

UPDATE: also working on a PR to migrate fs-admin to N-API: atom/fs-admin#105

[christian180]

@ #.github/workflows/ci.yml

[Roverclover]

Help

[Roverclover]

Help

[ishowshao]

finally, great!