Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "cross-emulation" support #63

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Add "cross-emulation" support #63

wants to merge 8 commits into from

Conversation

albertofaria
Copy link
Member

When combined with --emulated, this enables running VMs with an architecture different from the host's.

Closes #62.

TODO

Signed-off-by: Alberto Faria <afaria@redhat.com>
We attempt to detect if a container image is bootable. We can't easily
retrieve the image's labels, so we check if /usr/lib/bootc/install
exists and is a directory. If so, it is a bootable container. If it is a
bootable container but we're not running under Podman, we fail with an
error.

Once our container's entrypoint starts running, a background process on
the host (outside the container) queries Podman for the image's name and
ID, which the OCI runtime does not get but bootc-install needs. It then
saves the container image as an OCI archive.

It then runs the original container to generate the VM image. We do this
using krun [1] so that elevated privileges aren't necessary. Our
entrypoint blocks until this is done, and all subsequent logic remains
the same.

We could potentially avoid the OCI archive creation step by mounting the
host's container storage into the container running under krun. This
isn't trivial to achieve due to SELinux label and context mismatches
between the host and the krun environment, so we leave this optimization
for a future date.

Closes #26.

[1] https://github.com/containers/crun/blob/main/krun.1.md

Signed-off-by: Alberto Faria <afaria@redhat.com>
Store them as untagged containerdisks in the user's container storage.
They appear in the output of `podman images`, making users aware of
their existence and size on disk, and cleanup commands like `podman
image prune` remove them.

Signed-off-by: Alberto Faria <afaria@redhat.com>
Signed-off-by: Alberto Faria <afaria@redhat.com>
Signed-off-by: Alberto Faria <afaria@redhat.com>
It allows settings the disk size of the VM image that is generated from
a bootc container image.

Also improve the default disk size by basing it on the container image
size.

Signed-off-by: Alberto Faria <afaria@redhat.com>
Its value is a random identifier unique to the current run of the
current test.

Signed-off-by: Alberto Faria <afaria@redhat.com>
When combined with --emulated, this enables running VMs with an
architecture different from the host's.

Closes #62.

Signed-off-by: Alberto Faria <afaria@redhat.com>
@albertofaria
Copy link
Member Author

cc @ericcurtin

@albertofaria albertofaria changed the title Cross emulation Add cross-emul May 8, 2024
@albertofaria albertofaria changed the title Add cross-emul Add "cross-emulation" support May 8, 2024
@ericcurtin
Copy link

I don't know Rust well yet, just C, C++ of those kinds of languages.

But, I'll leave a few comments, regard all comments optional.

KVM will fail.

It's not currently possible to use this flag when the container image is a bootc
bootable container.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the bootable containers blocker at present for --emulated? Just curious...

I wonder could we apply --emulated flag automatically if we detect we are not on the same CPU architecture, I think podman qemu-user-static functionality does this and it makes sense.

There's another use-case emulated could be potentially useful, when EL2/KVM (or /dev/kvm) is not available, but it's probably not worth automatically applying --emulated in that case because it's so much slower than kvm or using containers with just plain-old crun.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the bootable containers blocker at present for --emulated? Just curious...

We currently use libkrun to run a micro VM that generates the VM disk image from the bootable container. Since libkrun relies on KVM, this only works if the bootable container's arch is the same as the host's. We could potentially use some qemu-based micro VM alternative instead of libkrun to lift this limitation.

I wonder could we apply --emulated flag automatically if we detect we are not on the same CPU architecture, I think podman qemu-user-static functionality does this and it makes sense.

This could make sense. It would make --emulated's meaning more complicated and less intuitive, though. Right now the user knows that --emulated = use emulation, no --emulated = use KVM, without having to think about what the host and VM arches are.

There's another use-case emulated could be potentially useful, when EL2/KVM (or /dev/kvm) is not available, but it's probably not worth automatically applying --emulated in that case because it's so much slower than kvm or using containers with just plain-old crun.

Yes, I think it's best to fail here to alert the user to the fact that KVM is not available, instead of silently using slow emulation.

@ericcurtin
Copy link

Looks great to me in general though...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support x86_64 and aarch64 emulation
2 participants