Best way to run on CentOS 9 host?

[fwilhe2]

Hi,

I was trying to run a lima VM (Debian guest) on a CentOS 9 Stream host.

I was facing a few issues where I was wondering if there was a way to make it more smooth.

Issue: Binary name

I've gotten this message:

{"level":"fatal","msg":"exec: \"qemu-system-x86_64\": executable file not found in $PATH","time":"2023-10-14T10:25:42+02:00"}

This is a workaround. The binary name seems always to be qemu-kvm on RHEL/CentOS, cf https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/8.0_release_notes/index#virtualization
bin_qemu="/usr/libexec/qemu-kvm"

export QEMU_SYSTEM_X86_64=/usr/libexec/qemu-kvm

Issue: Legacy Firmware

I've gotten this message:

{"level":"debug","msg":"QEMU version 8.1.0 detected","time":"2023-10-14T10:35:03+02:00"}
{"level":"debug","msg":"firmware candidates = [/home/florian/.local/share/qemu/edk2-x86_64-code.fd /usr/share/qemu/edk2-x86_64-code.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bin /usr/share/edk2-ovmf/x64/OVMF_CODE.fd]","time":"2023-10-14T10:35:03+02:00"}
{"level":"fatal","msg":"could not find firmware for \"/usr/libexec/qemu-kvm\" (hint: try setting `firmware.legacyBIOS` to `true`)","time":"2023-10-14T10:35:03+02:00"}

Workaround:

firmware:
  legacyBIOS: true

Improvement potential

• Is there any specific reason why lima does not look for the /usr/libexec/qemu-kvm binary on it's own? This seems to be always the case when people use EL as a host machine, I think it would make sense to not require configuring this manually.
• Why is the legacyBIOS setting needed? I'm not sure why my qemu version can't boot with EFI. This seems like lima does not set any of the available firmwares.

[afbjorklund]

Red Hat does not support using their qemu-kvm binary as a regular program, it is only intended for their libvirt software.
But we have been using it with a symlink for docker-machine, and you can use it with QEMU_SYSTEM_X86_64 for Lima...

https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9/html/configuring_and_managing_virtualization/assembly_feature-support-and-limitations-in-rhel-9-virtualization_configuring-and-managing-virtualization

QEMU is an essential component of the virtualization architecture in RHEL 9, but it is difficult to manage manually, and improper QEMU configurations may cause security vulnerabilities. Therefore, using qemu-* command-line utilities, such as qemu-kvm is not supported by Red Hat.

The legacy BIOS is needed for some images, such as CentOS Stream. But it is not needed for others, like Alma or Rocky.

But I think you just need to install OVMF_CODE.fd (or AAVMF), for the EFI support files: https://github.com/tianocore/edk2
It would have been included with a regular installation of QEMU... In the future, we might offer a "lima-qemu" bundle of it.

examples/centos-stream-8.yaml: # CentOS Stream 8 still requires legacyBIOS, while AlmaLinux 8 and Rocky Linux 8 do not.
examples/centos-stream-8.yaml: legacyBIOS: true
examples/centos-stream-9.yaml: # CentOS Stream 9 still requires legacyBIOS, while AlmaLinux 9 and Rocky Linux 9 do not.
examples/centos-stream-9.yaml: legacyBIOS: true

[mattbucci]

I guess I misunderstood your comment "only needs the package install"

It also needs a symlink following your above mapping

sudo ln -s /usr/share/edk2/ovmf/OVMF_CODE.fd /usr/share/OVMF/OVMF_CODE.fd

[mattbucci@localhost .config]$ dnf repoquery --list edk2-ovmf-20230524-4.el9_3.noarch
/usr/share/OVMF
/usr/share/OVMF/OVMF_CODE.secboot.fd
/usr/share/OVMF/OVMF_VARS.fd
/usr/share/OVMF/OVMF_VARS.secboot.fd
/usr/share/OVMF/UefiShell.iso
/usr/share/doc/edk2-ovmf
/usr/share/doc/edk2-ovmf/README
/usr/share/doc/edk2-ovmf/ovmf-whitepaper-c770f8c.txt
/usr/share/edk2
/usr/share/edk2/ovmf
/usr/share/edk2/ovmf/EnrollDefaultKeys.efi
/usr/share/edk2/ovmf/OVMF.amdsev.fd
/usr/share/edk2/ovmf/OVMF.inteltdx.fd
/usr/share/edk2/ovmf/OVMF_CODE.cc.fd
**/usr/share/edk2/ovmf/OVMF_CODE.fd**
/usr/share/edk2/ovmf/OVMF_CODE.secboot.fd
/usr/share/edk2/ovmf/OVMF_VARS.fd
/usr/share/edk2/ovmf/OVMF_VARS.secboot.fd
/usr/share/edk2/ovmf/Shell.efi
/usr/share/edk2/ovmf/UefiShell.iso
/usr/share/licenses/edk2-ovmf
/usr/share/licenses/edk2-ovmf/LICENSE.openssl
/usr/share/licenses/edk2-ovmf/License-History.txt
/usr/share/licenses/edk2-ovmf/License.OvmfPkg.txt
/usr/share/licenses/edk2-ovmf/License.txt
/usr/share/qemu
/usr/share/qemu/firmware
/usr/share/qemu/firmware/30-edk2-ovmf-x64-sb-enrolled.json
/usr/share/qemu/firmware/40-edk2-ovmf-x64-sb.json
/usr/share/qemu/firmware/50-edk2-ovmf-x64-nosb.json
/usr/share/qemu/firmware/60-edk2-ovmf-x64-amdsev.json
/usr/share/qemu/firmware/60-edk2-ovmf-x64-inteltdx.json

[afbjorklund]

Ah, maybe it needs another path in the default search list?

[AkihiroSuda]

Maybe just install vanilla QEMU from the source?

[DrDaveD]

On Alma9.3 and lima version 0.21.0 I also had to do

sudo ln -s ../edk2/ovmf/OVMF_CODE.fd /usr/share/OVMF/OVMF_CODE.fd

because although edk2-ovmf-20231122-6.el9.noarch was installed and already had other symlinks in that directory it did not have that one.

I also of course needed

export QEMU_SYSTEM_X86_64=/usr/libexec/qemu-kvm

[afbjorklund]

• Add firmware search paths for fedora rpms #2342