How can i run intel containers on ARM VM with microk8s

[mshlain]

Hi!
I'm stuck with the need to run MS SQL container on my Microk8s cluster. Unfortunately, MS only provides intel containers.
I've found the instructions on how to run individual intel containers on ARM VM on ARM host:
https://lima-vm.io/docs/config/multi-arch/

$ lima nerdctl run --platform=amd64 --rm alpine uname -m
x86_64

My question is how to configure containerd/microk8s to run intel containers ?
Thanks in advance

[afbjorklund]

I'm not sure if microk8s supports this, but you will have to ask Canonical on https://microk8s.io/ ?

As you mention, with Lima you have the option of running an x86 VM or running a x86 container

1. Slow mode (qemu-system, et al)

2. Fast mode (qemu-user, et al)

[afbjorklund]

Minikube has the same issue, mostly needed due to the outdated Kubernetes tutorial (only being available for amd64). But I don't know how you can do multiarch on a Kubernetes node, except for running the whole node as a different architecture... It sounds like something that upstream should be able to fix?

[jandubois]

I don't know how you can do multiarch on a Kubernetes node, except for running the whole node as a different architecture...

Yes, you have to run a node on a different architecture, but you can run a node inside a container, so only the container needs to run with the different architecture. And that really just means kubelet, plus the pods that will be scheduled to this node.

I never got around to trying this, but one day I will, unless somebody else figures it out first. 😄

[afbjorklund]

Right, with "the whole node" I meant either solution (VM / KIC). For the tutorial, the best is to just update the 7-year-old image...

But ideally, the server should be able to understand this (multiarch) similar to how it understands to run 32-bit images (multilib)

[jandubois]

I don't know; how would you specify the platform architecture in a Pod spec? The only idea I have would be to use a different RuntimeClass, but I have not checked if that could actually be used to pull a different image.

I suspect KIC would be an easier solution, but I may be missing some networking issues with stuff like NodePorts. But using e.g. a LoadBalancer or other Ingress should be straight-forward (famous last words alert, because I haven't actually tried it).

[afbjorklund]

It could be something simple, like adding a custom node taint or label.

https://kubernetes.io/docs/reference/labels-annotations-taints/#kubernetes-io-arch

But I think it is currently is a bit black-or-white, and none of this "grey" (or gray? sheesh)

I guess the typical cloud cluster would have two (hardware) nodes, one for each architecture.

[mshlain]

I have some partialy-working solution.
Environment:debian11 (default template) lima-vm on apple m1
First, i installed the cross-platform emulator collection inside my debian11 lima-vm, as described in documentation
Next, installed microk8s by installing apt snap packages and then installed microk8s from snap
Next, deployed my private helmcharts that use my private containers which are compiled for intel only
All containers are running correctly, no problems observed

The bad news: mssql does not work. Googling the error leads to solutions on top of rosetta or full virtualization, both are extremely slow in my case and thus not acceptable

[afbjorklund]

Probably needs real hardware and software, maybe a cloud machine or something like a NUC if you want it locally...