Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime-rs: It fails to run kata with dragonball with error message StartMicroVm(CreateVfioDevice(NoResource)) #9509

Closed
Apokleos opened this issue Apr 18, 2024 · 2 comments · Fixed by #9596 · May be fixed by #9595
Closed

runtime-rs: It fails to run kata with dragonball with error message StartMicroVm(CreateVfioDevice(NoResource)) #9509

Apokleos opened this issue Apr 18, 2024 · 2 comments · Fixed by #9596 · May be fixed by #9595
Assignees
@studychao
Labels
bug Incorrect behaviour
Projects
Issue backlog

Comments

@Apokleos
Copy link
Contributor

Get your issue reviewed faster

When running a kata on such host, which it has cpu info

$ cat /proc/cpuinfo |grep "address sizes"
address sizes	: 39 bits physical, 48 bits virtual
address sizes	: 39 bits physical, 48 bits virtual
address sizes	: 39 bits physical, 48 bits virtual
address sizes	: 39 bits physical, 48 bits virtual
address sizes	: 39 bits physical, 48 bits virtual
address sizes	: 39 bits physical, 48 bits virtual

And it fails with error as below:

ctr: failed to create shim task: Others("failed to handle message try init runtime instance\n\nCaused by:\n    0: init runtime handler\n    1: start sandbox\n    2: start vm\n    3: start vmm instance\n    4: Failed to start vmm\n    5: Failed to start MicroVm\n    6: vmm action error: StartMicroVm(CreateVfioDevice(NoResource))"): unknown
@Apokleos Apokleos added bug Incorrect behaviour needs-review Needs to be assessed by the team. labels Apr 18, 2024
@ananos
Copy link
Member

ananos commented May 3, 2024

Same thing happened to us, on aarch64 as well. With the following, the issue disappears but I guess we need input from @Apokleos (your commit enabled that: 014e0f4) and maybe @zvonkok (all PCI-related stuff will get you tagged!)?

diff --git a/src/runtime-rs/crates/hypervisor/src/dragonball/inner.rs b/src/runtime-rs/crates/hypervisor/src/dragonball/inner.rs
index e0bd30e8e..e77e8719d 100644
--- a/src/runtime-rs/crates/hypervisor/src/dragonball/inner.rs
+++ b/src/runtime-rs/crates/hypervisor/src/dragonball/inner.rs
@@ -220,7 +220,7 @@ impl DragonballInner {
             max_vcpu_count: self.config.cpu_info.default_maxvcpus as u8,
             mem_type,
             mem_file_path,
-            pci_hotplug_enabled: true,
+            pci_hotplug_enabled: false,
             ..Default::default()
         };
         info!(sl!(), "vm config: {:?}", vm_config);

@ananos ananos removed the needs-review Needs to be assessed by the team. label May 3, 2024
@zvonkok
Copy link
Contributor

zvonkok commented May 3, 2024

Is this not configurable via config? It should since for CoCo we need cold-plug.

ananos added a commit to nubificus/kata-containers that referenced this issue May 4, 2024
@ananos
runtime-rs: Add pci_hotplug as a config option
592966c 
There are cases where enabling PCI hotplug by default, breaks
Dragonball's initial setup, resulting in the following error message:

```
FATA[0000] failed to create shim task: Others("failed to handle message try
init runtime instance\n\nCaused by:\n    0: init runtime handler\n
1: start sandbox\n    2: start vm\n  3: start vmm instance\n
4: Failed to start vmm\n    5: Failed to start MicroVm\n
6: vmm action error: StartMicroVm(CreateVfioDevice(NoResource))"): unknown
```

Adding this as a config option enables normal booting even on nodes that
lack the default functionality.

Fixes: kata-containers#9509

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
@ananos ananos mentioned this issue May 4, 2024
Open
@katacontainersbot katacontainersbot moved this from To do to In progress in Issue backlog May 4, 2024
lifupan pushed a commit to lifupan/kata-containers that referenced this issue May 6, 2024
db: fix the issue of failed to init pci root bus
26bee78 
dragonball reserves 2048G of mmio space for the pci root bus by default
on physical addresses greater than 4G. However, for some machines with
smaller physical address widths, such as 39-bit wide physical addresses,
dragonball reserves the mmio space when initializing the memory. It is
less than 2048G, so this commit dynamically calculates and allocates the
mmio size of each pci root bus.

Fixes: kata-containers#9509

Signed-off-by: Fupan Li <fupan.lfp@antgroup.com>
@lifupan lifupan mentioned this issue May 6, 2024
Merged
ananos added a commit to nubificus/kata-containers that referenced this issue May 6, 2024
@ananos
runtime-rs: Add pci_hotplug as a config option
17e3391 
There are cases where enabling PCI hotplug by default, breaks
Dragonball's initial setup [see kata-containers#9596], resulting in the following
error message:

```
FATA[0000] failed to create shim task: Others("failed to handle message try
init runtime instance\n\nCaused by:\n    0: init runtime handler\n
1: start sandbox\n    2: start vm\n  3: start vmm instance\n
4: Failed to start vmm\n    5: Failed to start MicroVm\n
6: vmm action error: StartMicroVm(CreateVfioDevice(NoResource))"): unknown
```

Adding this as a config option enables normal booting even on nodes that
lack the default functionality.

Fixes: kata-containers#9509

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
ananos added a commit to nubificus/kata-containers that referenced this issue May 6, 2024
@ananos
runtime-rs: Add pci_hotplug as a config option
9781a44 
There are cases where enabling PCI hotplug by default, breaks
Dragonball's initial setup [see kata-containers#9596], resulting in the following
error message:

```
FATA[0000] failed to create shim task: Others("failed to handle message try
init runtime instance\n\nCaused by:\n    0: init runtime handler\n
1: start sandbox\n    2: start vm\n  3: start vmm instance\n
4: Failed to start vmm\n    5: Failed to start MicroVm\n
6: vmm action error: StartMicroVm(CreateVfioDevice(NoResource))"): unknown
```

Adding this as a config option enables normal booting even on nodes that
lack the default functionality.

Fixes: kata-containers#9509

Signed-off-by: Anastassios Nanos <ananos@nubificus.co.uk>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@studychao studychao

Labels
bug Incorrect behaviour
Projects
Issue backlog
  
In progress
Milestone
No milestone
4 participants
@ananos @zvonkok @studychao @Apokleos