Updating from 4.2.0 to 4.2.1 breaks sys-* vms #9217
Comments
minegameYTB
added
P: default
|
Check /var/log/xen/console/hypervisor.log, /var/log/xen/console/guest-sys-net.log and also exact version of xen-hypervisor package ( |
|
For the kernel: 6.6.25-1.fc37 For the xen version: xen-hypervisor-4.17.4-2.fc37.x86-64 As well as the hypervisor and sys-net logs attached to my OneDrive: https://1drv.ms/f/s!AjdQsaJ6acqhg4M38rstjCQ_cS0JEA |
|
the sys-net doesn't seem to reach to the point to log anything... maybe there is some info in guest-sys-net-dm.log in the same dir? do you see any entries with recent timestamps? |
|
Here are the logs for guest-sys-net-dm.log attached here: https://1drv.ms/f/s!AjdQsaJ6acqhg4M8AHaGfn0Wi_04fw |
|
This one looks normal... lets try with sys-net itself a bit more - add |
|
No other information appears after log file opened |
|
I don't know if the info will be useful, but the log located in /var/log/qubes/qrexec.sys-net.log indicates this: |
|
I think this just means the VM has crashed, without much hint why... |
|
This is qubes-core-qrexec version 4.2.18-1 |
|
Lets try something else then. Try increasing Xen log level with You can also set the increased log level by adding |
|
Here is the content of hypervisor.log after adding the option in the xen command line at startup as attachment: https://1drv.ms/f/s!AjdQsaJ6acqhg4NBZ3Hq5p1jiKZqaw |
|
Finally some info is logged: Details
As a temporary workaround, you can try switching to a different kernel version in sys-net settings. |
|
Or better switch default kernel in global settings |
|
Does a qubes without netvm work? In default setup that would be |
|
I just followed the indication that you proposed to me (so the kernel delivered before the update to qubes 4.2.1, In this case, 6.1.62-1.qubes.fc37.x86_64), I notice when qubes os is completely restarted that the error continues, looking in the log file of the vm concerned, That it is a kernel panic |
Yes, vault works correctly |
|
Can you try switching virtualization mode to HVM in vault settings and see if it still works? I'm trying to figure out whether issue is with any HVM, or only with those with PCI devices (sys-net, sys-usb). |
|
I just changed to HVM, and I have the same qrexec error as on sys-net and the other vms having HVM mode |
andrewdavidwong
added
C: Xen
needs diagnosis
|
@minegameYTB There's a key log-line missing from every single one of your hypervisor logs. The one that gives the exact Family/Model/Stepping of the CPU. Please can you collect |
Knowing that there are two cores with this processor |
|
Thanks. There's a known bug in the microcode for this CPU, specifically revision 0x40 which was released in March. You've got 0x2e here, but other logs you've provided show which is the bad microcode. Revert the microcode package back to the previous version for now, ensuring it's revision 0x38 or earlier. |
|
Ah, uh I don't think that the microcode will change depending on the OS I run on, I was on an OS to save the current state of the SSD, I'm going to redo the measurement on dom0, I apologize for that |
|
Here is the measurement on qubes os (dom0) |
But how can I go back to revision 0x38? |
|
Revert the The bad microcode came in: https://www.qubes-os.org/news/2024/03/18/qsb-101-update/ which was package version |
|
I tried the command But the command starts by dependency sys-net, but the command fails (because the vm crashes) |
|
Temporarily remove ucode=scan from xen options
|
|
The networking part seems to be working fine, however, now sys-usb is giving me the error "Start failed: internal error: Unable to reset PCI device 0000:00:15.0: no FLR, PM reset or bus reset available, see /var/log/libvirt/libxl/libxl- driver.log for details" |
|
I just tried again (still having the ucode=scan option to remove in the cmdline in grub) using specify the "no strict reset" option on the USB ports in question and it works again |
|
I just went back to microcode_ctl-2.1-56.qubes1 (instead of microcode_ctl-2.1-57.qubes1) tells me that I am on microcode 0x3e (I didn't realize that I had written the firmware revision number incorrectly, hence the modification of the message at the last minute.. sorry), everything is working correctly again |
|
Besides, I was wondering if there will be a correction for this type of processor ? And (if this is not yet the case) how can we temporarily exclude the update of microcode_ctl on dom0 until the bug is fixed by Intel ? |
Good to hear. Regarding a fix, I'm working on it with Intel. Stay tuned |
|
@minegameYTB I've got a present for you. This microcode should fix your issue. Drop the file in |
|
Thank you so much ^^ I just checked the xl dmesg command, it tells me that firmware 0x42 is loaded. and the virtual machines have to boot correctly too |
andrewdavidwong
added
diagnosed
Qubes OS release
Qubes os 4.2.1 (R4.2)
Xen 4.17.4
Brief summary
When restarting after updating to qubes os 4.2.1, the vms sys-* are no longer functional, with a qrexec error "qrexec-daemon.c:144:sigchld_parent_handler: Connection to the VM failed"
Steps to reproduce
Have an HP 240 g7 and install qubes os 4.2.0
Update qubes os to the latest version
Expected behavior
That the vms start correctly after the update
Actual behavior
The vms does not start and gives the error
"qrexec-daemon.c:144:sigchld_parent_handler: Connection to the VM failed"
The text was updated successfully, but these errors were encountered: