DragonFlyBSD

I think we tracked this down this morning, a fix is being tested.

I think the problem boils down to:
http://nxr.netbsd.org/xref/src-dragonflybsd/sys/platform/pc64/x86_64/vm_machdep.c#143
pcb2->pcb_cr3 |= PG_RW | PG_U | PG_V;

On x86 and x86-64, the CR3 register points to the root of the page tables for an address space. The bottom three bits of the address in CR3 are supposed to be zero, (AMD64 documentation states 'should be zero'), but DFly has been setting them with the corresponding flags from page table entries. This has worked out okay on real hardware, as the AMD docs only say 'should be zero' (Intel says 'ignored').

When (qemu-)KVM is used on hardware without two-dimensional paging, it has to handle writes to CR3 itself. The code to do so is in Linux's arch/x86/kvm/emulate.c, check_cr_write(). If any of the reserved bits in CR3 are set on a store, we see a #GP. This is perhaps arguably a bug.

To fix this, just remove that line in vm_machdep.c and try it.

Am 16.05.2013 04:49, schrieb Chris Turner via Redmine:

Issue #2561 has been reported by c.turner1.

----------------------------------------
Bug #2561: Fwd: Re: DragonFly x86_64 won't boot on qemu-kvm
http://bugs.dragonflybsd.org/issues/2561

Author: c.turner1
Status: New
Priority: Normal
Assignee:
Category:
Target version:

Forwarding to bugs@ - relavent threads contain
further details.

-------- Original Message --------
Subject: Re: DragonFly x86_64 won't boot on qemu-kvm
Date: Wed, 15 May 2013 21:48:22 -0500
From: Chris Turner <c.turner@199technologies.com>
To: users@dragonflybsd.org

On 05/15/13 11:41, Michael Neumann wrote:

Hi,

I am having exactly the same problems as described in:

http://leaf.dragonflybsd.org/mailarchive/users/2012-10/msg00027.html

Indeed I am also having this problem as well

(RHEL6-stock host/kernel/etc/virt-manager on a core2-duo)

Have not yet reported since I know I haven't had time to get a debug
env properly setup.

I've tested x86-64 from ~1-2wks back, and also had tested around
september last year - have been using 32bit as a workaround since
it suits my purposes in this situation.

It seem like FreeBSD has a similar problem:

https://bugzilla.redhat.com/show_bug.cgi?id=881579
http://forums.freebsd.org/showthread.php?t=36761

I'm not clear from your email - did any of the workarounds work?

setting cpu flags, the kvm modprobe option, etc?

No, nothing worked. It took me half a day until I realized that :)

I was able to boot up when using 'pure emulation' QEMU,
(as outlined in original thread IIRC) but this is hardly ideal :D

I did not try pure emulation, but I am very sure that it would work for
me as well,
but as the system is a server I did not take this "solution" into account.

For now, the only solution seems to be to use i386 on that particual host (or wait that this is fixed in kvm).
I am wondering why it fails on DF but not for example on Linux. Are they doing things differently?

It does appear that the panic happens consistently for me after 'low level' init
and right before mounting the root filesystem - IIRC this is when the actual
buffer cache/VM etc is setup, and so presumably lots would be going on w/r/t
memory pages/interrupts/etc in a fairly OS-dependent manner.

My thought was to add some spurious printfs here and maybe throw in some ASM
noops in the right places to see if that allows any narrowing down of the
problem, but again, I haven't had a chance to get a proper dev env setup
in this environment.

Unfortunately my time will likely be short until at least mid summer.
GRR! need more unix time!

I think Venkatesh has found the bug, so there is no need for that. In
this moment I am compiling
an ISO image with his suggested change. It takes some time to compile
DragonFly inside a virtual
machine on this low-end laptop :). Stay tuned.

Regards,

Michael

aggelos@ found a second place where there was an offending %cr3 register, in the x86-64 pmap:

http://leaf.dragonflybsd.org/~aggelos/0002-kernel-x86_64-Do-not-set-reserved-bits-in-CR3.patch

If you apply that and the original:
http://acm.jhu.edu/~me/0001-kernel-x86_64-Do-not-set-reserved-bits-in-CR3.patch

it works on qemu-kvm 1.1 on a Linux 3.2 host.

If folks could test these two patches together on a few physical x86-64 systems too, to make sure that it doesn't break them, that would be really helpful.

Am 19.05.2013 08:31, schrieb Venkatesh Srinivas via Redmine:

Issue #2561 has been updated by vsrinivas.

aggelos@ found a second place where there was an offending %cr3 register, in the x86-64 pmap:

http://leaf.dragonflybsd.org/~aggelos/0002-kernel-x86_64-Do-not-set-reserved-bits-in-CR3.patch

If you apply that and the original:
http://acm.jhu.edu/~me/0001-kernel-x86_64-Do-not-set-reserved-bits-in-CR3.patch

it works on qemu-kvm 1.1 on a Linux 3.2 host.

If folks could test these two patches together on a few physical x86-64 systems too, to make sure that it doesn't break them, that would be really helpful.

Hi Vekatesh,

I can confirm that these two patches work on qemu-kvm. Awesome work!
Thank you so much!

Regards,

Michael

I cherry-picked commit 5e73d and tested, and it definitely fixes the problem I reported in http://leaf.dragonflybsd.org/mailarchive/users/2012-10/msg00027.html

Excellent work! Thank you!