ARM64 support

[marcellanz]

With a bit of hype lately as well as some real emerging demand, I'd like to open this issue to support ARM64. There are multiple tasks to do and also some challenges as some depending technology is not yet supporting the architecture fully.

• Travis CI build support.
  • https://docs.travis-ci.com/user/multi-cpu-architectures
  • https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support
  • https://docs.travis-ci.com/user/reference/overview/#virtualisation-environment-vs-operating-system
• Multi-Arch Docker images
  • for user support libraries with native languages and binaries for the TCK.
  • for TCK and proxy images.
  • https://docs.docker.com/docker-for-mac/multi-arch/
• GraalVM Support and native image support.
  • How to build it for armhf/aarch64? oracle/graal#1154
  • LLVM for ARM64 oracle/graal#1090
• define other areas that are relevant for support on ARM64 systems.

Contemporary Notes

Components with potential issues:

• JIB plugin, which is used to build Java and Kotlin samples: Push multi-platform images together in one go (i.e., push manifest list instead of manifest) GoogleContainerTools/jib#2523, via @vkorenev
• Distroless is distroless support mutil-arch image? GoogleContainerTools/distroless#406, via @vkorenev

---

WIP
Before opening this issue I did some explorative work and after some "travis fiddling" making some sbt:it with the TCK tests running. There is also a running Multi-Arch TCK docker image for the Go Support with an ARM64 build (gcr.io/mrcllnz/cloudstate-go-tck:latest).

The WIP branch can be found here:
https://github.com/marcellanz/cloudstate/tree/feature/arm64_support / https://github.com/marcellanz/cloudstate/issues/4

The most complicated part so far was the travis-ci arm64 image that runs as a "LXD container on Packet" and has an annoying habit not to resolve the localhosts IP address properly through its given hostname. It does not help that the LXD containers have limited permissions given to the travis runner. I fixed that with a bit of a hack I think, after fiddling unsuccessfully with /etc/hosts and HOSTALIASES.

Still, some TCKs are failing but 20 of 25 tests succeeded:
https://travis-ci.com/github/marcellanz/cloudstate/jobs/355043948#L2335

[info] Run completed in 4 minutes, 1 second.
2333[info] Total number of tests run: 25
2334[info] Suites: completed 6, aborted 0
2335[info] Tests: succeeded 20, failed 5, canceled 0, ignored 0, pending 0
2336[info] *** 5 TESTS FAILED ***

TCK failing: https://travis-ci.com/github/marcellanz/cloudstate/jobs/355043948#L1710

(Please ignore other failing jobs on the links above, other tests like npm tests with node do work are working without any changes. As it takes a good amount of time, some parts of the travis build are commented out at the moment for the PoC.)

[sleipnir]

Very good @marcellanz I think this comes at an appropriate time since almost all Cloud Providers are adding support for the ARM architecture. I am going to change the travis files that I support to compile on multiple architectures. I hope this helps. But I think the biggest challenge is in the proxy image

[sleipnir]

It would be nice if it came into our Roadmap in https://github.com/orgs/cloudstateio/projects/1.
I opened a topic related to Dart support for ARM architecture in travis https://travis-ci.community/t/dart-arm-build-errors/9175

[sleipnir]

The ARM build will also fail for Kotlin support protocolbuffers/protobuf#3318

I opened this issue to address the issue in the Kotlin repository

[vkorenev]

@marcellanz, great job! I think that multiarch support is important given that ARM is making it's way slowly but steadily to both cloud providers https://aws.amazon.com/ec2/graviton/ and developers' workstations https://en.wikipedia.org/wiki/Mac_transition_to_Apple_silicon.
Unfortunately, Google does not seem to be very excited about ARM on servers at least now. So some otherwise great tools maintained by Google do not have proper support for multiarch at the moment:

• JIB plugin, which is used to build Java and Kotlin samples: Push multi-platform images together in one go (i.e., push manifest list instead of manifest) GoogleContainerTools/jib#2523
• Distroless is distroless support mutil-arch image? GoogleContainerTools/distroless#406

FWIW, I have some ARM64 hardware at home (4 x SBCs with 4GB of RAM and fast NVMe SSDs) which I can use to set up a k8s cluster and try to run Cloudstate on it.

[marcellanz]

@vkorenev This is also what I saw on multiple occasions where requests to support arm64 where dismissed with that the project just does not support it and one has to build "your own version". It's also understandable, as it is a lot of work to maintain production grade quality for a separate or multiple architecture. Let's see if that changes :) perhaps customers demand and cost reductions will drive that too.

It would be great to have Cloudstate running on your ARM64 cluster. What type/brand of SBC do you have?

[vkorenev]

@marcellanz Radxa Rock Pi 4 Model A https://wiki.radxa.com/Rockpi4/getting_started#get_start_specs
It has a very nice set of features:

• M.2 M-Key connector with PCIe Gen2 x4 lanes which can be used for a NVMe SSD
• 4GB 64bit dual channel LPDDR4
• on-board power converter which accepts USB PD, Qualcomm QuickCharge, or just a dumb 12-20V power supply with enough wattage
• huge passive heatsink
• cheaper version without WiFi/BT is available
• good support in the upstream kernel/u-boot

[marcellanz]

@vkorenev very interesting thanks, especially the M.2 M-Key connector.
I see there are "U.2 - MiniSAS to M.2 M-Key" adapters; and I'm on the verge to replace a server of mine that uses too much energy for not doing that much most if the time. Replacing the server mandates to have a connection to a LTO-7 Tape drive that perhaps could work. Going full ARM could be very much an option I think :)

[chanseokoh]

• JIB plugin, which is used to build Java and Kotlin samples: GoogleContainerTools/jib#2523
• Distroless GoogleContainerTools/distroless#406

FYI, these issues have been closed.

[marcellanz]

Thanks, @chanseokoh, awesome :)