Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use VZ by default for new instances on macOS >= 13.5 #1951

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

AkihiroSuda
Copy link
Member

VZ is now the default type for new instances >= 13.5, unless the config is incompatible with vz (e.g., firmware.legacyBIOS=true, mountType=9p).

Existing instances will continue to use QEMU by default, unless vz-identifier is present in the instance directory.

Run limactl start with --debug to see how the vmType is resolved.

@AkihiroSuda
Copy link
Member Author

@lima-vm/maintainers

I'll release v1.0 beta after merging this.

@hasan4791
Copy link
Contributor

hasan4791 commented Oct 27, 2023

Here is the logs

(vz-default) lima-vm-lima> uname -a
Darwin Ts-MacBook-Pro.local 23.1.0 Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:24 PDT 2023; root:xnu-10002.41.9~6/RELEASE_ARM64_T6000 arm64
(vz-default) lima-vm-lima>
(vz-default) lima-vm-lima> limactl --version
limactl version 0.5.0-2060-ged39486
(vz-default) lima-vm-lima> which limactl 
/Users/chasan/Work/go/src/github.com/temp/lima-vm-lima/_output/bin/limactl
(vz-default) lima-vm-lima> limactl --debug start
DEBU[0000] Evaluating yq expression: ""                 
? Creating an instance "default" Proceed with the current configuration
DEBU[0001] ResolveVMType: resolved VMType "vz" (default for macOS 13.5 and later) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (default for macOS 13.5 and later) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (default for macOS 13.5 and later) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (default for macOS 13.5 and later) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (explicitly specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (explicitly specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (existing instance, without "/Users/chasan/.lima/podman-rootful-x86/vz-identifier") 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (existing instance, without "/Users/chasan/.lima/test/vz-identifier") 
DEBU[0001] ResolveVMType: resolved VMType "vz" (explicitly specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] Make sure "host" network is stopped          
DEBU[0001] Make sure "user-v2" network is stopped       
DEBU[0001] Make sure usernet network is stopped         
DEBU[0001] Make sure "shared" network is stopped        
DEBU[0001] Make sure "bridged" network is stopped       
INFO[0001] Starting the instance "default" with VM driver "vz" 
DEBU[0001] ResolveVMType: resolved VMType "vz" (default for macOS 13.5 and later) 
INFO[0001] Attempting to download the image              arch=aarch64 digest="sha256:78d35a2f551d281912ee7e5202660d94d713aa1b5de86a17e261328cc2f093d4" location="https://cloud-images.ubuntu.com/releases/23.10/release-20231011/ubuntu-23.10-server-cloudimg-arm64.img"
DEBU[0001] downloading "https://cloud-images.ubuntu.com/releases/23.10/release-20231011/ubuntu-23.10-server-cloudimg-arm64.img" into "/Users/chasan/Library/Caches/lima/download/by-url-sha256/aaadf19ec428e9ce95eb976df0ab7dc6b76e4537c681aadf637892e49b0c6b95/data" 
Downloading the image (ubuntu-23.10-server-cloudimg-arm64.img)
714.88 MiB / 714.88 MiB [-----------------------------------] 100.00% 5.02 MiB/s
DEBU[0145] res.ValidatedDigest=true                     
INFO[0145] Downloaded the image from "https://cloud-images.ubuntu.com/releases/23.10/release-20231011/ubuntu-23.10-server-cloudimg-arm64.img" 
INFO[0145] Converting "/Users/chasan/.lima/default/basedisk" (qcow2) to a raw disk "/Users/chasan/.lima/default/diffdisk" 
3.50 GiB / 3.50 GiB [-------------------------------------] 100.00% 221.74 MiB/s
INFO[0161] Expanding to 100GiB                          
INFO[0161] Attempting to download the nerdctl archive    arch=aarch64 digest="sha256:ea30ab544c057e3a0457194ecd273ffbce58067de534bdfaffe4edf3a4da6357" location="https://github.com/containerd/nerdctl/releases/download/v1.6.2/nerdctl-full-1.6.2-linux-arm64.tar.gz"
DEBU[0161] file "" is cached as "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/data" 
DEBU[0161] Comparing digest "sha256:ea30ab544c057e3a0457194ecd273ffbce58067de534bdfaffe4edf3a4da6357" with the cached digest file "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/sha256.digest", not computing the actual digest of "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/data" 
DEBU[0161] res.ValidatedDigest=true                     
INFO[0161] Using cache "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/data" 
DEBU[0161] [hostagent] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/chasan/.lima/default/vz-identifier") 
DEBU[0161] [hostagent] ResolveVMType: resolved VMType "vz" (existing instance, with "/Users/chasan/.lima/default/vz-identifier") 
DEBU[0161] [hostagent] Creating iso file /Users/chasan/.lima/default/cidata.iso 
DEBU[0161] [hostagent] Using /var/folders/11/07zj9vzj4hq4j9y3kjk4dhg80000gn/T/diskfs_iso1003621884 as workspace 
DEBU[0162] [hostagent] Failed to detect CPU features. Assuming that AES acceleration is available on this Apple silicon. 
DEBU[0162] [hostagent] OpenSSH version 9.4.1 detected   
DEBU[0162] [hostagent] AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com 
INFO[0162] [hostagent] hostagent socket created at /Users/chasan/.lima/default/ha.sock 
DEBU[0162] [hostagent] Start udp DNS listening on: 127.0.0.1:60805 
INFO[0162] [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/chasan/.lima/default/serial*.log") 
DEBU[0162] [hostagent] Start tcp DNS listening on: 127.0.0.1:64395 
DEBU[0162] [hostagent] Using search domains: [softlayer.local] 
INFO[0162] [hostagent] new connection from  to          
DEBU[0162] received an event                             event="{2023-10-27 16:49:37.725529 +0530 IST {false false false [] 60022}}"
INFO[0162] SSH Local Port: 60022                        
DEBU[0162] [hostagent] [VZ] - vm state change: "VirtualMachineStateStarting" 
INFO[0162] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0162] [hostagent] executing script "ssh"           
INFO[0162] [hostagent] [VZ] - vm state change: running  
DEBU[0162] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0162] [hostagent] stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n", err=failed to execute script "ssh": stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n": exit status 255 
INFO[0172] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0172] [hostagent] executing script "ssh"           
DEBU[0172] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0173] [hostagent] stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n", err=failed to execute script "ssh": stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n": exit status 255 
INFO[0183] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0183] [hostagent] executing script "ssh"           
DEBU[0183] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0183] [hostagent] stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n", err=failed to execute script "ssh": stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n": exit status 255 
ERRO[0183] [hostagent] r.CreateEndpoint() = connection was refused 
ERRO[0185] [hostagent] r.CreateEndpoint() = connection was refused 
ERRO[0187] [hostagent] r.CreateEndpoint() = connection was refused 
INFO[0193] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0193] [hostagent] executing script "ssh"           
DEBU[0193] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0193] [hostagent] stdout="", stderr="", err=<nil>  
INFO[0193] [hostagent] The essential requirement 1 of 3 is satisfied 
INFO[0193] [hostagent] Waiting for the essential requirement 2 of 3: "user session is ready for ssh" 
DEBU[0193] [hostagent] executing script "user session is ready for ssh" 
DEBU[0193] [hostagent] executing ssh for script "user session is ready for ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0193] [hostagent] stdout="", stderr="+ timeout 30s bash -c 'until sudo diff -q /run/lima-ssh-ready /mnt/lima-cidata/meta-data 2>/dev/null; do sleep 3; done'\n", err=<nil> 
INFO[0193] [hostagent] The essential requirement 2 of 3 is satisfied 
INFO[0193] [hostagent] Waiting for the essential requirement 3 of 3: "the guest agent to be running" 
DEBU[0193] [hostagent] executing script "the guest agent to be running" 
DEBU[0193] [hostagent] executing ssh for script "the guest agent to be running": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0193] [hostagent] stdout="", stderr="+ sock=/run/lima-guestagent.sock\n+ timeout 30s bash -c 'until [ -S \"/run/lima-guestagent.sock\" ]; do sleep 3; done'\n", err=<nil> 
INFO[0193] [hostagent] The essential requirement 3 of 3 is satisfied 
INFO[0193] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available" 
DEBU[0193] [hostagent] executing script "systemd must be available" 
DEBU[0193] [hostagent] Forwarding unix sockets          
DEBU[0193] [hostagent] executing ssh for script "systemd must be available": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
INFO[0193] [hostagent] Forwarding "/run/lima-guestagent.sock" (guest) to "/Users/chasan/.lima/default/ga.sock" (host) 
DEBU[0193] [hostagent] stdout="", stderr="+ command -v systemctl\n", err=<nil> 
INFO[0193] [hostagent] The optional requirement 1 of 2 is satisfied 
DEBU[0193] [hostagent] guest agent info: &{LocalPorts:[{IP::: Port:22} {IP:127.0.0.54 Port:53} {IP:127.0.0.53 Port:53}]} 
INFO[0193] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed" 
DEBU[0193] [hostagent] executing script "containerd binaries to be installed" 
DEBU[0193] [hostagent] guest agent event: {Time:2023-10-27 11:20:08.872640988 +0000 UTC LocalPortsAdded:[{IP:127.0.0.54 Port:53} {IP:127.0.0.53 Port:53} {IP::: Port:22}] LocalPortsRemoved:[] Errors:[]} 
INFO[0193] [hostagent] Not forwarding TCP 127.0.0.54:53 
DEBU[0193] [hostagent] executing ssh for script "containerd binaries to be installed": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
INFO[0193] [hostagent] Not forwarding TCP 127.0.0.53:53 
INFO[0193] [hostagent] Not forwarding TCP [::]:22       
DEBU[0205] [hostagent] stdout="/usr/local/bin/nerdctl\n", stderr="+ timeout 30s bash -c 'until command -v nerdctl || test -x /usr/local/bin/nerdctl; do sleep 3; done'\n", err=<nil> 
INFO[0205] [hostagent] The optional requirement 2 of 2 is satisfied 
INFO[0205] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished" 
DEBU[0205] [hostagent] executing script "boot scripts must have finished" 
DEBU[0205] [hostagent] executing ssh for script "boot scripts must have finished": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0215] [hostagent] stdout="", stderr="+ timeout 30s bash -c 'until sudo diff -q /run/lima-boot-done /mnt/lima-cidata/meta-data 2>/dev/null; do sleep 3; done'\n", err=<nil> 
INFO[0215] [hostagent] The final requirement 1 of 1 is satisfied 
DEBU[0215] received an event                             event="{2023-10-27 16:50:30.072711 +0530 IST {true false false [] 60022}}"
INFO[0215] READY. Run `lima` to open the shell.         
(vz-default) lima-vm-lima> 
(vz-default) lima-vm-lima> 
(vz-default) lima-vm-lima> limactl ls --json default | jq -r .vmType
vz
(vz-default) lima-vm-lima>

@hasan4791
Copy link
Contributor

(vz-default) lima-vm-lima> limactl --debug start --mount-type 9p
DEBU[0000] Evaluating yq expression: ".mountType = \"9p\"" 
? Creating an instance "default" Proceed with the current configuration
DEBU[0001] ResolveVMType: resolved VMType "qemu" (mountType="9p" is specified in []*LimaYAML{o,y,d}[1]) 
WARN[0001] `mountType: 9p` is experimental              
DEBU[0001] ResolveVMType: resolved VMType "qemu" (mountType="9p" is specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (mountType="9p" is specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (mountType="9p" is specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (explicitly specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "vz" (explicitly specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (existing instance, without "/Users/chasan/.lima/podman-rootful-x86/vz-identifier") 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (existing instance, without "/Users/chasan/.lima/test/vz-identifier") 
DEBU[0001] ResolveVMType: resolved VMType "vz" (explicitly specified in []*LimaYAML{o,y,d}[1]) 
DEBU[0001] Make sure "user-v2" network is stopped       
DEBU[0001] Make sure usernet network is stopped         
DEBU[0001] Make sure "shared" network is stopped        
DEBU[0001] Make sure "bridged" network is stopped       
DEBU[0001] Make sure "host" network is stopped          
INFO[0001] Starting the instance "default" with VM driver "qemu" 
DEBU[0001] Executed [codesign --verify /opt/homebrew/bin/qemu-system-aarch64]: out=""  error="<nil>"
DEBU[0001] Executed [codesign --display --entitlements - --xml /opt/homebrew/bin/qemu-system-aarch64]: out="Executable=/opt/homebrew/Cellar/qemu/8.1.1/bin/qemu-system-aarch64\n<?xml version=\"1.0\" encoding=\"UTF-8\"?><!DOCTYPE plist PUBLIC \"-//Apple//DTD PLIST 1.0//EN\" \"https://www.apple.com/DTDs/PropertyList-1.0.dtd\"><plist version=\"1.0\"><dict><key>com.apple.security.hypervisor</key><true/></dict></plist>\n"  error="<nil>"
INFO[0001] QEMU binary "/opt/homebrew/bin/qemu-system-aarch64" seems properly signed with the "com.apple.security.hypervisor" entitlement 
DEBU[0001] ResolveVMType: resolved VMType "qemu" (mountType="9p" is specified in []*LimaYAML{o,y,d}[1]) 
INFO[0001] Attempting to download the image              arch=aarch64 digest="sha256:78d35a2f551d281912ee7e5202660d94d713aa1b5de86a17e261328cc2f093d4" location="https://cloud-images.ubuntu.com/releases/23.10/release-20231011/ubuntu-23.10-server-cloudimg-arm64.img"
DEBU[0001] file "/Users/chasan/.lima/default/basedisk" is cached as "/Users/chasan/Library/Caches/lima/download/by-url-sha256/aaadf19ec428e9ce95eb976df0ab7dc6b76e4537c681aadf637892e49b0c6b95/data" 
DEBU[0001] Comparing digest "sha256:78d35a2f551d281912ee7e5202660d94d713aa1b5de86a17e261328cc2f093d4" with the cached digest file "/Users/chasan/Library/Caches/lima/download/by-url-sha256/aaadf19ec428e9ce95eb976df0ab7dc6b76e4537c681aadf637892e49b0c6b95/sha256.digest", not computing the actual digest of "/Users/chasan/Library/Caches/lima/download/by-url-sha256/aaadf19ec428e9ce95eb976df0ab7dc6b76e4537c681aadf637892e49b0c6b95/data" 
DEBU[0001] res.ValidatedDigest=true                     
INFO[0001] Using cache "/Users/chasan/Library/Caches/lima/download/by-url-sha256/aaadf19ec428e9ce95eb976df0ab7dc6b76e4537c681aadf637892e49b0c6b95/data" 
INFO[0001] Attempting to download the nerdctl archive    arch=aarch64 digest="sha256:ea30ab544c057e3a0457194ecd273ffbce58067de534bdfaffe4edf3a4da6357" location="https://github.com/containerd/nerdctl/releases/download/v1.6.2/nerdctl-full-1.6.2-linux-arm64.tar.gz"
DEBU[0001] file "" is cached as "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/data" 
DEBU[0001] Comparing digest "sha256:ea30ab544c057e3a0457194ecd273ffbce58067de534bdfaffe4edf3a4da6357" with the cached digest file "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/sha256.digest", not computing the actual digest of "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/data" 
DEBU[0001] res.ValidatedDigest=true                     
INFO[0001] Using cache "/Users/chasan/Library/Caches/lima/download/by-url-sha256/21921213c0426e38e2f26d2844446767e2a19c0b6655c774f9e4adbb5c368d8f/data" 
DEBU[0001] [hostagent] ResolveVMType: resolved VMType "qemu" (existing instance, without "/Users/chasan/.lima/default/vz-identifier") 
DEBU[0001] [hostagent] ResolveVMType: resolved VMType "qemu" (existing instance, without "/Users/chasan/.lima/default/vz-identifier") 
DEBU[0002] [hostagent] Creating iso file /Users/chasan/.lima/default/cidata.iso 
DEBU[0002] [hostagent] Using /var/folders/11/07zj9vzj4hq4j9y3kjk4dhg80000gn/T/diskfs_iso4038845048 as workspace 
DEBU[0002] [hostagent] Failed to detect CPU features. Assuming that AES acceleration is available on this Apple silicon. 
DEBU[0002] [hostagent] OpenSSH version 9.4.1 detected   
DEBU[0002] [hostagent] AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com 
INFO[0002] [hostagent] hostagent socket created at /Users/chasan/.lima/default/ha.sock 
DEBU[0002] [hostagent] Start udp DNS listening on: 127.0.0.1:49224 
DEBU[0002] [hostagent] Start tcp DNS listening on: 127.0.0.1:64661 
DEBU[0002] [hostagent] QEMU version 8.1.1 detected      
DEBU[0002] [hostagent] firmware candidates = [/Users/chasan/.local/share/qemu/edk2-aarch64-code.fd /opt/homebrew/share/qemu/edk2-aarch64-code.fd /usr/share/AAVMF/AAVMF_CODE.fd /usr/share/qemu-efi-aarch64/QEMU_EFI.fd] 
INFO[0002] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/chasan/.lima/default/serial*.log") 
DEBU[0002] [hostagent] qCmd.Args: [/opt/homebrew/bin/qemu-system-aarch64 -m 4096 -cpu host -machine virt,accel=hvf -smp 4,sockets=1,cores=4,threads=1 -drive if=pflash,format=raw,readonly=on,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/chasan/.lima/default/diffdisk,if=virtio,discard=on -drive id=cdrom0,if=none,format=raw,readonly=on,file=/Users/chasan/.lima/default/cidata.iso -device virtio-scsi-pci,id=scsi0 -device scsi-cd,bus=scsi0.0,drive=cdrom0 -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:ca:83:9e -device virtio-rng-pci -display none -device virtio-gpu -device virtio-keyboard-pci -device virtio-mouse-pci -device qemu-xhci,id=usb-bus -parallel none -chardev socket,id=char-serial,path=/Users/chasan/.lima/default/serial.sock,server=on,wait=off,logfile=/Users/chasan/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-serial-pci,path=/Users/chasan/.lima/default/serialp.sock,server=on,wait=off,logfile=/Users/chasan/.lima/default/serialp.log -device pci-serial,chardev=char-serial-pci -chardev socket,id=char-serial-virtio,path=/Users/chasan/.lima/default/serialv.sock,server=on,wait=off,logfile=/Users/chasan/.lima/default/serialv.log -device virtio-serial-pci,id=virtio-serial0,max_ports=1 -device virtconsole,chardev=char-serial-virtio,id=console0 -virtfs local,mount_tag=mount0,path=/Users/chasan,security_model=none,readonly -virtfs local,mount_tag=mount1,path=/tmp/lima,security_model=none -chardev socket,id=char-qmp,path=/Users/chasan/.lima/default/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-default -pidfile /Users/chasan/.lima/default/qemu.pid] 
DEBU[0002] received an event                             event="{2023-10-27 17:00:36.442189 +0530 IST {false false false [] 60022}}"
INFO[0002] SSH Local Port: 60022                        
INFO[0002] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0002] [hostagent] executing script "ssh"           
DEBU[0002] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0002] [hostagent] qemu[stderr]: qemu-system-aarch64: -virtfs local,mount_tag=mount0,path=/Users/chasan,security_model=none,readonly: warning: short-form boolean option 'readonly' deprecated 
DEBU[0002] [hostagent] qemu[stderr]: Please use readonly=on instead 
DEBU[0002] [hostagent] stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n", err=failed to execute script "ssh": stdout="", stderr="ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n": exit status 255 
INFO[0012] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0012] [hostagent] executing script "ssh"           
DEBU[0012] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0013] [hostagent] stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n", err=failed to execute script "ssh": stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n": exit status 255 
INFO[0023] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0023] [hostagent] executing script "ssh"           
DEBU[0023] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0023] [hostagent] stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n", err=failed to execute script "ssh": stdout="", stderr="\"System is booting up. Unprivileged users are not permitted to log in yet. Please come back later. For technical details, see pam_nologin(8).\"\nConnection closed by 127.0.0.1 port 60022\r\n": exit status 255 
INFO[0033] [hostagent] Waiting for the essential requirement 1 of 3: "ssh" 
DEBU[0033] [hostagent] executing script "ssh"           
DEBU[0033] [hostagent] executing ssh for script "ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0033] [hostagent] handleQuery lookup SRV failed    
DEBU[0034] [hostagent] stdout="", stderr="", err=<nil>  
INFO[0034] [hostagent] The essential requirement 1 of 3 is satisfied 
INFO[0034] [hostagent] Waiting for the essential requirement 2 of 3: "user session is ready for ssh" 
DEBU[0034] [hostagent] executing script "user session is ready for ssh" 
DEBU[0034] [hostagent] executing ssh for script "user session is ready for ssh": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0034] [hostagent] stdout="", stderr="+ timeout 30s bash -c 'until sudo diff -q /run/lima-ssh-ready /mnt/lima-cidata/meta-data 2>/dev/null; do sleep 3; done'\n", err=<nil> 
INFO[0034] [hostagent] The essential requirement 2 of 3 is satisfied 
INFO[0034] [hostagent] Waiting for the essential requirement 3 of 3: "the guest agent to be running" 
DEBU[0034] [hostagent] executing script "the guest agent to be running" 
DEBU[0034] [hostagent] executing ssh for script "the guest agent to be running": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0034] [hostagent] stdout="", stderr="+ sock=/run/lima-guestagent.sock\n+ timeout 30s bash -c 'until [ -S \"/run/lima-guestagent.sock\" ]; do sleep 3; done'\n", err=<nil> 
INFO[0034] [hostagent] The essential requirement 3 of 3 is satisfied 
INFO[0034] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available" 
DEBU[0034] [hostagent] executing script "systemd must be available" 
DEBU[0034] [hostagent] Forwarding unix sockets          
DEBU[0034] [hostagent] executing ssh for script "systemd must be available": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
INFO[0034] [hostagent] Forwarding "/run/lima-guestagent.sock" (guest) to "/Users/chasan/.lima/default/ga.sock" (host) 
DEBU[0034] [hostagent] stdout="", stderr="+ command -v systemctl\n", err=<nil> 
INFO[0034] [hostagent] The optional requirement 1 of 2 is satisfied 
DEBU[0034] [hostagent] guest agent info: &{LocalPorts:[{IP::: Port:22} {IP:127.0.0.54 Port:53} {IP:127.0.0.53 Port:53}]} 
INFO[0034] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed" 
DEBU[0034] [hostagent] executing script "containerd binaries to be installed" 
DEBU[0034] [hostagent] guest agent event: {Time:2023-10-27 11:31:07.934438501 +0000 UTC LocalPortsAdded:[{IP:127.0.0.54 Port:53} {IP:127.0.0.53 Port:53} {IP::: Port:22}] LocalPortsRemoved:[] Errors:[]} 
INFO[0034] [hostagent] Not forwarding TCP 127.0.0.54:53 
DEBU[0034] [hostagent] executing ssh for script "containerd binaries to be installed": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
INFO[0034] [hostagent] Not forwarding TCP 127.0.0.53:53 
INFO[0034] [hostagent] Not forwarding TCP [::]:22       
DEBU[0036] [hostagent] handleQuery lookup SRV failed    
DEBU[0046] [hostagent] stdout="/usr/local/bin/nerdctl\n", stderr="+ timeout 30s bash -c 'until command -v nerdctl || test -x /usr/local/bin/nerdctl; do sleep 3; done'\n", err=<nil> 
INFO[0046] [hostagent] The optional requirement 2 of 2 is satisfied 
INFO[0046] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished" 
DEBU[0046] [hostagent] executing script "boot scripts must have finished" 
DEBU[0046] [hostagent] executing ssh for script "boot scripts must have finished": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile="/Users/chasan/.lima/_config/user" -o IdentityFile="/Users/chasan/.ssh/id_rsa" -o IdentityFile="/Users/chasan/.ssh/id_rsa_git" -o IdentityFile="/Users/chasan/.ssh/podman-machine-default" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers="^aes128-gcm@openssh.com,aes256-gcm@openssh.com" -o User=chasan -o ControlMaster=auto -o ControlPath="/Users/chasan/.lima/default/ssh.sock" -o ControlPersist=yes -p 60022 127.0.0.1 -- /bin/bash] 
DEBU[0058] [hostagent] stdout="", stderr="+ timeout 30s bash -c 'until sudo diff -q /run/lima-boot-done /mnt/lima-cidata/meta-data 2>/dev/null; do sleep 3; done'\n", err=<nil> 
INFO[0058] [hostagent] The final requirement 1 of 1 is satisfied 
DEBU[0058] received an event                             event="{2023-10-27 17:01:32.011059 +0530 IST {true false false [] 60022}}"
INFO[0058] READY. Run `lima` to open the shell.         
(vz-default) lima-vm-lima>

@AkihiroSuda
Copy link
Member Author

Thanks @hasan4791 , so, is it working as expected?

@hasan4791
Copy link
Contributor

Thanks @hasan4791 , so, is it working as expected?

Yes.

@AkihiroSuda AkihiroSuda force-pushed the vz-default branch 2 times, most recently from a5f6936 to e1a34f8 Compare October 27, 2023 16:37
@pmalek
Copy link

pmalek commented Oct 30, 2023

While this only flips what's the default, @AkihiroSuda can we first solve #1940?

@AkihiroSuda
Copy link
Member Author

While this only flips what's the default, @AkihiroSuda can we first solve #1940?

Being fixed in:

@AkihiroSuda
Copy link
Member Author

Split trivial parts into:

@AkihiroSuda AkihiroSuda marked this pull request as draft October 30, 2023 11:28
examples/experimental/vz.yaml is retained until v1.0 RC, to avoid
breaking existing bookmarks.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
VZ is now the default type for new instances >= 13.5, unless the config
is incompatible with vz (e.g., `firmware.legacyBIOS=true`, `mountType=9p`).

Existing instances will continue to use QEMU by default,
unless `vz-identifier` is present in the instance directory.

Run `limactl start` with `--debug` to see how the vmType is resolved.

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>
@chancez
Copy link
Contributor

chancez commented Nov 20, 2023

Might also be worth waiting for #1957 to be mitigated or resolved

@jandubois
Copy link
Member

Might also be worth waiting for #1957 to be mitigated or resolved

I think that is the plan to delay the 1.0 release until VZ can safely become the default.

Unfortunately this has become even more complex with QEMU seemingly broken on Apple M3 chips #1996. So people with a brand new machine currently have no choice than to switch to VZ.

@EdwardMoyse
Copy link

Might also be worth waiting for #1957 to be mitigated or resolved

Just to clarify that despite the title (which I now changed) this was not specific to vz, and we also got corruption with qemu

@AkihiroSuda
Copy link
Member Author

@lima-vm/maintainers Are we in a good state to merge this? (after rebasing)

@AkihiroSuda
Copy link
Member Author

Finch is going to switch the default to vz soon

@afbjorklund
Copy link
Contributor

Podman Machine VM changed from qemu to vz (through their own vfkit wrapper), in 5.0:

https://blog.podman.io/2024/03/podman-5-0-is-around-the-corner/ (v5.0.0 tagged Mar 19)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants