Bug #3310
open
NVMM+QEMU fail to boot with UEFI: Mem Assist Failed [gpa=0xfffffff0]
Added by liweitianux almost 3 years ago. Updated over 2 years ago.
0%
Description
NVMM+QEMU fail to boot with UEFI, for example:
% qemu-system-x86_64 \
-boot menu=on -display sdl -accel nvmm \
-drive file=OVMF_CODE.fd,if=pflash,format=raw,readonly=on \
-drive file=OVMF_VARS.fd,if=pflash,format=raw
NetBSD Virtual Machine Monitor accelerator is operational
qemu-system-x86_64: NVMM: Mem Assist Failed [gpa=0xfffff000]
qemu-system-x86_64: NVMM: Failed to execute a VCPU.
The UEFI firmware can be obtained by installing the uefi-edk2-qemu-x86_64 package
or by downloading from: https://leaf.dragonflybsd.org/~aly/uefi/
First reported by Mario Marietto and confirmed by me, see:
https://lists.dragonflybsd.org/pipermail/users/2022-January/404898.html
Files
| qemu-system-x86.7z (3.5 MB) qemu-system-x86.7z | marietto, 07/15/2022 07:34 AM |
Updated by liweitianux almost 3 years ago
- Category set to nvmm
- Status changed from New to In Progress
A temporary workaround to boot with UEFI in NVMM+QEMU is:
specify the UEFI code with the -bios option instead of the -device if=pflash (or -pflash) option.
However, this is not recommended because the UEFI variables are partially emulated and aren't persistent.
See: https://lists.gnu.org/archive/html/qemu-discuss/2018-04/msg00045.html
Updated by liweitianux almost 3 years ago
After investigation, the issue is caused by the missing memory maps of the UEFI firmware for the guest.
The UEFI firmware are mapped by QEMU as ROM devices in the ROMD mode. And this is excluded for guest memory mappings in the QEMU NVMM code.
I figured out the following patch that makes NVMM+QEMU to boot with UEFI:
diff --git target/i386/nvmm/nvmm-all.c target/i386/nvmm/nvmm-all.c
index 290077f62..e3c948b31 100644
--- target/i386/nvmm/nvmm-all.c
+++ target/i386/nvmm/nvmm-all.c
@@ -1082,7 +1082,11 @@ nvmm_process_section(MemoryRegionSection *section, int add)
unsigned int delta;
uintptr_t hva;
- if (!memory_region_is_ram(mr)) {
+ /*
+ * Don't exclude ROMD memory; for example, it's used to map UEFI firmware
+ * (if=pflash) and should be mapped for guest.
+ */
+ if (!memory_region_is_ram(mr) && !memory_region_is_romd(mr)) {
return;
}
However, NVMM+QEMU is extremely slow and uses 100% CPU. It's even much slower than TCG (i.e., without -accel nvmm).
For example: for QEMU to boot into the UEFI shell on my desktop, it takes ~12 seconds with TCG, but it takes ~170 seconds with NVMM !
Test command:
qemu-system-x86_64 -boot menu=on \ -drive file=OVMF_CODE.fd,if=pflash,format=raw,readonly=on \ -drive file=OVMF_VARS.fd,if=pflash,format=raw \ -display sdl [-accel nvmm]
Updated by marietto almost 3 years ago
Hello. I tried using the -bios parameter to add the efi code,like this :
qemu-system-x86_64 \
-machine type=q35,accel=nvmm \
-smp cpus=1 -m 8G \
-bios /home/marietto/Desktop/Files/Virt/OVMF/OVMF.fd \
-drive file=/mnt/dk26/bhyve/impish-cuda-11-4-nvidia-470.img,if=none,id=disk0 \
-device virtio-blk-pci,drive=disk0 \
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:6022-:22 \
-device virtio-net-pci,netdev=net0 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0 \
-display curses \
-vga qxl \
-spice addr=127.0.0.1,port=5900,ipv4=on,disable-ticketing=on,seamless-migration=on
but it didn't work :
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x3a, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x3a [val=0x1], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x140, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0xce, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x140 [val=0x0], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x64e, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x34, ignored
and this :
qemu-system-x86_64 \
-machine type=q35,accel=nvmm \
-smp cpus=1 -m 8G \
-bios /home/marietto/Desktop/Files/Virt/OVMF/QEMU_UEFI_CODE-x86_64.fd \
-drive file=/mnt/dk26/bhyve/impish-cuda-11-4-nvidia-470.img,if=none,id=disk0 \
-device virtio-blk-pci,drive=disk0 \
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:6022-:22 \
-device virtio-net-pci,netdev=net0 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0 \
-display curses \
-vga qxl \
-spice addr=127.0.0.1,port=5900,ipv4=on,disable-ticketing=on,seamless-migration=on
qemu: could not load PC BIOS '/home/marietto/Desktop/Files/Virt/OVMF/QEMU_UEFI_CODE-x86_64.fd'
with this : OVMF_CODE.fd =
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x3a, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x3a [val=0x1], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x140, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0xce, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x140 [val=0x0], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x64e, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x34, ignored
with this :
Updated by marietto almost 3 years ago
with this : -bios /usr/pkg/share/qemu/edk2-x86_64-code.fd \
qemu: could not load PC BIOS '/usr/pkg/share/qemu/edk2-x86_64-code.fd'
Updated by tuxillo over 2 years ago
marietto wrote in #note-3:
Hello. I tried using the -bios parameter to add the efi code,like this :
qemu-system-x86_64 \
machine type=q35,accel=nvmm \:22 \
-smp cpus=1 -m 8G \
-bios /home/marietto/Desktop/Files/Virt/OVMF/OVMF.fd \
-drive file=/mnt/dk26/bhyve/impish-cuda-11-4-nvidia-470.img,if=none,id=disk0 \
-device virtio-blk-pci,drive=disk0 \
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:6022
-device virtio-net-pci,netdev=net0 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0 \
-display curses \
-vga qxl \
-spice addr=127.0.0.1,port=5900,ipv4=on,disable-ticketing=on,seamless-migration=onbut it didn't work :
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x3a, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x3a [val=0x1], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x140, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0xce, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x140 [val=0x0], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x64e, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x34, ignoredand this :
qemu-system-x86_64 \
machine type=q35,accel=nvmm \:22 \
-smp cpus=1 -m 8G \
-bios /home/marietto/Desktop/Files/Virt/OVMF/QEMU_UEFI_CODE-x86_64.fd \
-drive file=/mnt/dk26/bhyve/impish-cuda-11-4-nvidia-470.img,if=none,id=disk0 \
-device virtio-blk-pci,drive=disk0 \
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:6022
-device virtio-net-pci,netdev=net0 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0 \
-display curses \
-vga qxl \
-spice addr=127.0.0.1,port=5900,ipv4=on,disable-ticketing=on,seamless-migration=onqemu: could not load PC BIOS '/home/marietto/Desktop/Files/Virt/OVMF/QEMU_UEFI_CODE-x86_64.fd'
with this : OVMF_CODE.fd =
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x3a, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x3a [val=0x1], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x140, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0xce, ignored
qemu-system-x86_64: NVMM: Unexpected WRMSR 0x140 [val=0x0], ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x64e, ignored
qemu-system-x86_64: NVMM: Unexpected RDMSR 0x34, ignoredwith this :
Is there any specific reason why you need UEFI?
Also, what's that "impish cuda" image? Some custom linux? Please remember we do not have hardware devices passthrough.
Updated by marietto over 2 years ago
- File qemu-system-x86.7z qemu-system-x86.7z added
1) Is there any specific reason why you need UEFI?
Yes,because today every modern hypervisor uses UEFI and not BIOS anymore. In addition,I'm trying to start a collaboration to implement the passthrough. I'm not interested in using a hypervisor that uses the old BIOS bootloader.
2) Also, what's that "impish cuda" image? Some custom linux?
it is only a linux VM (ubuntu impish + cuda 470) that I've previously created for bhyve. I've also tried with windows 11 (also in this case it is a vm created for bhyve) and it gave the same error message.
4) I tried another experiment. I've created a fresh new img file with the command :
qemu-img create -f raw jammy.img 200G
and then I've launched the vm with these parameters :
qemu-system-x86_64 \
-machine type=q35,accel=nvmm \
-smp cpus=4 -m 8G \
-drive if=pflash,format=raw,readonly=on,file=/usr/local/share/uefi-edk2-qemu/QEMU_UEFI_CODE-x86_64.fd \
-drive if=pflash,format=raw,file=/usr/local/share/uefi-edk2-qemu/QEMU_UEFI_VARS-x86_64.fd \
-drive id=cdrom,if=none,media=cdrom,file="ubuntu-22.04-desktop-amd64.iso" \
#-drive file=ubuntu-22.04-desktop-amd64.iso,media=cdrom,id=cdrom \
-drive file=/mnt/da16s1d/home/marietto/Desktop/VMS/jammy.img,if=none,id=disk0 \
-device virtio-blk-pci,drive=disk0 \
-netdev user,id=net0,hostfwd=tcp:127.0.0.1:6022-:22 \
-device virtio-net-pci,netdev=net0 \
-object rng-random,id=rng0,filename=/dev/urandom \
-device virtio-rng-pci,rng=rng0 \
-display curses \
-vga qxl \
-spice addr=127.0.0.1,port=5900,ipv4=on,disable-ticketing=on,seamless-migration=on
the error messages are :
root@marietto:/mnt/da16s1d/home/marietto/Desktop/VMS # ./vm2.sh
NetBSD Virtual Machine Monitor accelerator is operational
libGL error: MESA-LOADER: failed to open iris: Cannot open "/usr/local/lib/dri/iris_dri.so" (search paths /usr/local/lib/dri, suffix _dri)
libGL error: failed to load driver: iris
qemu-system-x86_64: NVMM: Mem Assist Failed [gpa=0xfffffff0]
qemu-system-x86_64: NVMM: Failed to execute a VCPU.
Someone wants to debug the core file produced by the bug ? I've attached it.