Knowing if we are running arm or amd64, and ability to select them.

[cdrage]

When I run for example podman-bootc run quay.io/centos-bootc/centos-bootc:stream9 which is a hosted manifest with multiple arch images, I was unsure if podman-bootc was going to run either arch.

I wasn't sure so I checked out podman-bootc --help and I could not find any options to pass in specifying arm64 or amd64.

[ckyrouac]

podman-bootc run with a multi arch image will select the host machine's architecture. Since the end result of podman-bootc run is to spin up a VM on the host, the image needs to share the host's architecture. There might be a way we could enable cross architecture VMs but I'm not sure we want to support that.

[cdrage]

@ckyrouac Sorry, bit confused. The requirements say: qemu-system-x86_64/qemu-system-aarch64 for some reason I thought podman-bootc ships with both / requires both?

IMHO it's a key feature to be able to run amd64 on arm64, especially for those developing bootc images for amd64 platforms (since it's the majority of servers).

[ckyrouac]

ah interesting, I didn't realize it is relatively simple to emulate cross architectures via qemu

[ckyrouac]

@germag will libkrun support running a cross-architecture VM, especially amd64 VM on an arm host?

[germag]

@germag will libkrun support running a cross-architecture VM, especially amd64 VM on an arm host?

On apple silicon not yet, basically because it will be slow (same with qemu).

To have a usable performance we need to use rosetta, which is forbidden by apple. But, since the total storage order is documented, in the (near?) future it will support some form of cross-arch virt, but I'm not sure if it will be full system or user mode virt.

Add support for cross-arch on podman-bootc is in my TODO list (only with qemu), but it's currently very low priority

[cfergeau]

To have a usable performance we need to use rosetta, which is forbidden by apple.

You can't use rosetta to run VMs. You can use rosetta inside a linux VM to run userland binaries, but I don't think this is useful for podman-bootc, it wants to start an aarch64 disk image on an amd64 machine, or vice-versa.

[cfergeau]

ah interesting, I didn't realize it is relatively simple to emulate cross architectures via qemu

One caveat is that this is not available on RHEL.

[gbraad]

IMHO it's a key feature to be able to run amd64 on arm64,

We also do not run ARM image on x86_64. The consensus has been in podman desktop to allow the user to see which image arch is available and allow pulling these for the arch that is able to run. There is a technical limitation here. Especially this needs to stand out:

To have a usable performance

as this means Rosetta is the only viable option. Full system virtualization with Qemu is merely a workaround and might lead to issues due to low performance.

[ckyrouac]

since podman-bootc is a tool to aid in developing/testing bootc images (not building the final image), it might make more sense to use image builder to do a cross-arch build of the disk image so the VM can be run on the host's architecture. This will avoid the performance issues.

[rhatdan]

BTW Qemu emulation is also very unreliable, we have seem many apps crash when run in emulation mode. Specifically Podman and other containers which reexec themselves. Many other apps can have issues as well, for example SETUID apps do not work. Have not tried any of this with Rosetta yet, but hopefully this will be in Podman Machine soon.

[cfergeau]

BTW Qemu emulation is also very unreliable, we have seem many apps crash when run in emulation mode.

Do you know if these issues are known upstream? Are there bugs filed in fedora bugzilla or upstream to track this?