Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lima fails in a confusing way when the home directory is on NFS #2325

Closed
DrDaveD opened this issue May 6, 2024 · 2 comments · Fixed by #2329
Closed

lima fails in a confusing way when the home directory is on NFS #2325

DrDaveD opened this issue May 6, 2024 · 2 comments · Fixed by #2329
Labels
component/qemu QEMU

Comments

@DrDaveD
Copy link

DrDaveD commented May 6, 2024

Description

I stumbled across the fact that a home directory on NFS does not work with lima, because qemu can't handle it. I discussed it with @afbjorklund and he requested that I make an issue so something can be done to make it more clear why a failure is happening.

The error I got looked like this:

$ limactl start
? Creating an instance "default" Choose another template (docker, podman, archlinux, fedora, ...)
? Choose a template ubuntu-lts
? Creating an instance "default" Proceed with the current configuration
INFO[0014] Starting the instance "default" with VM driver "qemu"
INFO[0014] Attempting to download the image              arch=x86_64 digest="sha256:42dcf9757e75c3275486b397a752fb535c7cd8e5232ee5ee349554b7a55f1702" location="https://cloud-images.ubuntu.com/releases/22.04/release-20240308/ubuntu-22.04-server-cloudimg-amd64.img"
Downloading the image (ubuntu-22.04-server-cloudimg-amd64.img)
618.25 MiB / 618.25 MiB [----------------------------------] 100.00% 31.03 MiB/s
INFO[0038] Downloaded the image from "https://cloud-images.ubuntu.com/releases/22.04/release-20240308/ubuntu-22.04-server-cloudimg-amd64.img"
FATA[0038] failed to run [qemu-img create -f qcow2 -F qcow2 -b /nashome/d/dwd/.lima/default/basedisk /nashome/d/dwd/.lima/default/diffdisk 107374182400]: "Formatting '/nashome/d/dwd/.lima/default/diffdisk', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=107374182400 backing_file=/nashome/d/dwd/.lima/default/basedisk backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16\nqemu-img: /nashome/d/dwd/.lima/default/diffdisk: Image is not in qcow2 format\n": exit status 1

The workaround is to set LIMACTL_HOME to an empty directory on a local disk. If you point it to a directory that has other subdirectories in it, lima unexpectedly dies looking for lima.yaml files in them. I expected a variable name with HOME in it to look like $HOME where a .lima directory would be created; instead it behaves more like a CONFIG directory, so that name was also misleading.
@afbjorklund
Copy link
Contributor

afbjorklund commented May 6, 2024 •
edited

The issue is https://gitlab.com/qemu-project/qemu/-/issues/946, about file locks being broken on NFS (known fact)

The suggestion is to document the workaround, and to add a check on lima home and warn if it is mounted as nfs

It is OK to have $HOME be on a network disk, including ~/.cache.

We just want $LIMA_HOME to be a on a local disk, for performance.

This was referenced May 7, 2024
Merged
Merged
@afbjorklund
Copy link
Contributor

afbjorklund commented May 12, 2024 •
edited

The current error messages are a bit blunt, if you try to run in an unsupported fashion:

# limactl ls
FATA[0000] must not run as the root
$ limactl ls
FATA[0000] must not run on NFS dir

Unfortunately it is up to the administrator, to configure $LIMA_HOME on a local disk...

e.g. if "/local" is the mount point used:

export LIMA_HOME=/local/$USER/.lima

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
component/qemu QEMU
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Issue an error message when trying to run on NFS afbjorklund/lima
3 participants
@DrDaveD @AkihiroSuda @afbjorklund