DragonFlyBSD

In an attempt to diagnose issues with the re(4) driver, I connected my dragonflybsd workstation via crossover cable to my (Slackware) laptop and assigned IP addresses to both on the same subnet. They could ping each other. I then attempted to ssh from the laptop to the workstation. The ssh client on the laptop displayed:

Bad packet length 2852084625
Disconnecting: Packet corrupt

The dragonflybsd machine immediately panicked. The machine in question:

DragonFly sorrow 2.13-DEVELOPMENT DragonFly v2.13.0.332.g9855a4-DEVELOPMENT #0: Sun Nov 20 09:45:34 UTC 2011 root@pkgbox64.dragonflybsd.org:/usr/obj/usr/src/sys/X86_64_GENERIC x86_64

The backtrace, after rebooting and having the core saved:

sorrow# kgdb /var/crash/kern.0  /var/crash/vmcore.0 
GNU gdb (GDB) 7.3
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying" 
and "show warranty" for details.
This GDB was configured as "x86_64-dragonfly".
For bug reporting instructions, please see:
<http://bugs.dragonflybsd.org/>...
Reading symbols from /var/crash/kern.0...done.

Unread portion of the kernel message buffer:
panic: vm_fault: fault on nofault entry, addr: 0xffffffe011390000
cpuid = 1
Trace beginning at frame 0xffffffe0a6c86258
panic() at panic+0x1fb
panic() at panic+0x1fb
vm_fault() at vm_fault+0x1fb
trap_pfault() at trap_pfault+0x183
trap() at trap+0x4cb
calltrap() at calltrap+0x8
--- trap 000000000000000c, rip = ffffffff8065e286, rsp = ffffffe0a6c865d0, rbp = ffffffe0a6c86648 ---
hammer_init_cursor() at hammer_init_cursor+0x1c9
hammer_vop_nresolve() at hammer_vop_nresolve+0x2b6
vop_nresolve_ap() at vop_nresolve_ap+0xe
null_nresolve() at null_nresolve+0x26
vop_nresolve() at vop_nresolve+0x88
cache_resolve() at cache_resolve+0x750
nlookup() at nlookup+0x35c
kern_stat() at kern_stat+0x1c
sys_lstat() at sys_lstat+0x3e
syscall2() at syscall2+0x370
Xfast_syscall() at Xfast_syscall+0xcb
Debugger("panic")

CPU1 stopping CPUs: 0x0000000d
stopped

CPU1 restarting CPUs: 0x000000000000000d
restarted
panic: vm_fault: fault on nofault entry, addr: 0xffffffe011390000
cpuid = 1
Debugger("panic")

CPU1 stopping CPUs: 0x0000000d
stopped

CPU1 restarting CPUs: 0x000000000000000d
restarted
boot() called on cpu#1
Uptime: 58m52s
Physical memory: 4063 MB
Dumping 1234 MB: 1219 1203 1187 1171 1155 1139 1123 1107 1091 1075 1059 1043 1027SECONDARY PANIC ON CPU 2 THREAD 0xffffffe0332c5870
1011 995 979 963 947 931 915 899 883 867 851 835 819 803 787 771 755 739 723 707 691 675 659 643 627 611 595 579 563 547 531 515 499 483 467 451 435 419 403 387 371 355 339 323 307 291 275 259 243 227 211 195 179 163 147 131 115 99 83 67 51 35 19 3

[New pid 23216/0, ftp]
[New pid 20690/0, bash]
[New pid 687/0, sshd]
[New pid 777/0, getty]
[New pid 705/0, sendmail]
[New kernel bufdaemon]
[New kernel pagezero]
[New kernel swapcached]
[New kernel hammer-S1]
[New kernel rtable_cpu 3]
[New kernel ithread 15]
[New kernel usb6]
[New kernel ithread 23]
[New kernel usb2]
[New kernel ithread 27]
[New kernel usb0]
[New kernel ithread 195]
[New kernel crypto 3]
[New kernel firmware taskq]
[New kernel taskq_cpu 3]
[New kernel ifnet 3]
[New kernel netisr_cpu 3]
[New kernel dsched 3]
[New kernel usched 3]
[New kernel ithread emerg]
[New kernel softclock 3]
[New kernel idle_3]
[New pid 365/0, rpcbind]
[New pid 789/0, su]
[New pid 788/0, bash]
[New pid 782/0, getty]
[New pid 780/0, getty]
[New pid 778/0, getty]
[New pid 701/0, sendmail]
[New pid 492/0, nfsd]
[New pid 491/0, nfsd]
[New pid 490/0, nfsd]
[New pid 487/0, nfsd]
[New kernel consttyd]
[New kernel bufdaemon_hw]
[New kernel pagedaemon]
[New kernel hammer-S0]
[New kernel rtable_cpu 2]
[New kernel ithread 14]
[New kernel usb5]
[New kernel ithread 18]
[New kernel ithread 22]
[New kernel ithread 30]
[New kernel usb1]
[New kernel ithread 26]
[New kernel acpi_task]
[New kernel crypto 2]
[New kernel taskq_cpu 2]
[New kernel ifnet 2]
[New kernel disk_msg_core]
[New kernel netisr_cpu 2]
[New kernel dsched 2]
[New kernel usched 2]
[New kernel softclock 2]
[New kernel idle_2]
[New pid 720/0, cron]
[New pid 27754/0, rm]
[New pid 27753/0, sh]
[New pid 27719/0, sh]
[New pid 27682/0, sh]
[New pid 27185/0, bmake]
[New pid 27181/0, sh]
[New pid 27179/0, bmake]
[New pid 27177/0, bmake]
[New pid 19857/0, login]
[New pid 18157/0, bmake]
[New pid 18156/0, sh]
[New pid 17752/0, bmake]
[New pid 90650/0, sh]
[New pid 90648/0, sh]
[New pid 90308/0, bmake]
[New pid 855/0, sh]
[New pid 786/0, screen-4.0.3]
[New pid 784/0, bash]
[New pid 783/0, getty]
[New pid 781/0, getty]
[New pid 775/0, login]
[New pid 657/0, usbd]
[New pid 656/0, moused]
[New kernel ithread 17]
[New pid 488/0, nfsd]
[New pid 334/0, syslogd]
[New kernel syncer]
[New kernel hammer-S3]
[New kernel hammer-M]
[New kernel rtable_cpu 1]
[New kernel ithread 1]
[New kernel usb4]
[New kernel ithread 21]
[New kernel usbtask-dr]
[New kernel xpt_thrd]
[New kernel ithread 197]
[New kernel crypto 1]
[New kernel taskq_cpu 1]
[New kernel ifnet 1]
[New kernel netisr_cpu 1]
[New kernel devfs_msg_core]
[New kernel dsched 1]
[New kernel usched 1]
[New kernel softclock 1]
[New kernel idle_1]
[New pid 1/0, init]
[New pid 787/0, screen-4.0.3]
[New pid 853/0, sh]
[New pid 795/0, bmake]
[New pid 794/0, bash]
[New pid 790/0, csh]
[New pid 779/0, getty]
[New kernel vnlru]
[New kernel vmdaemon]
[New kernel hammer-S2]
[New kernel rtable_cpu 0]
[New kernel random]
[New kernel ithread 4]
[New kernel ithread 192]
[New pid 0/0, swapper]
[New kernel ithread 20]
[New kernel usb7]
[New kernel usb3]
[New kernel ithread 16]
[New kernel ithread 28]
[New kernel usbtask-hc]
[New kernel ithread 9]
[New kernel crypto returns]
[New kernel crypto 0]
[New kernel taskq_cpu 0]
[New kernel ifnet 0]
[New kernel netisr_cpu 0]
[New kernel dsched 0]
[New kernel usched 0]
[New kernel ithread 196]
[New kernel softclock 0]
[New kernel idle_0]
Reading symbols from /boot/kernel/acpi.ko...done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/ahci.ko...done.
Loaded symbols for /boot/kernel/ahci.ko
Reading symbols from /boot/kernel/ehci.ko...done.
Loaded symbols for /boot/kernel/ehci.ko
Reading symbols from /boot/kernel/snd_hda.ko...done.
Loaded symbols for /boot/kernel/snd_hda.ko
Reading symbols from /boot/kernel/sound.ko...done.
Loaded symbols for /boot/kernel/sound.ko
_get_mycpu () at ./machine/thread.h:69
69 ./machine/thread.h: No such file or directory.
in ./machine/thread.h
(kgdb) backtrace
#0 _get_mycpu () at ./machine/thread.h:69
#1 md_dumpsys (di=<optimized out>) at /usr/src/sys/platform/pc64/x86_64/dump_machdep.c:263
#2 0xffffffff80488b65 in dumpsys () at /usr/src/sys/kern/kern_shutdown.c:925
#3 0xffffffff804891cb in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:387
#4 0xffffffff80489482 in panic (fmt=0xffffffff80814b60 "vm_fault: fault on nofault entry, addr: p")
at /usr/src/sys/kern/kern_shutdown.c:831
#5 0xffffffff806935f4 in vm_fault (map=0xffffffff80d1c3e0, vaddr=18446743936559546368, fault_type=<optimized out>,
fault_flags=<optimized out>) at /usr/src/sys/vm/vm_fault.c:358
#6 0xffffffff80736b95 in trap_pfault (frame=0xffffffe0a647f8b8, usermode=0) at /usr/src/sys/platform/pc64/x86_64/trap.c:916
#7 0xffffffff807371b0 in trap (frame=0xffffffe0a647f8b8) at /usr/src/sys/platform/pc64/x86_64/trap.c:630
#8 0xffffffff8072f9be in calltrap () at /usr/src/sys/platform/pc64/x86_64/exception.S:187
#9 0xffffffff8066868f in hammer_io_direct_write (hmp=0xffffffe0833ec000, bio=0xffffffe010b6b448, record=0xffffffe0a7b8b370)
at /usr/src/sys/vfs/hammer/hammer_io.c:1729
#10 0xffffffff80682455 in hammer_vop_strategy_write (ap=<optimized out>) at /usr/src/sys/vfs/hammer/hammer_vnops.c:3359
#11 hammer_vop_strategy (ap=0xffffffe0a647fa40) at /usr/src/sys/vfs/hammer/hammer_vnops.c:2662
#12 0xffffffff805037fc in vop_strategy (ops=0xffffffe00f10ecc0, vp=0x50000001, bio=0xd04) at /usr/src/sys/kern/vfs_vopops.c:834
#13 0xffffffff804dc28e in vn_strategy (vp=0xffffffe0ae56f400, bio=0xffffffe010b6b448) at /usr/src/sys/kern/vfs_bio.c:3697
#14 0xffffffff804dffd5 in bawrite (bp=0xffffffe010b6b3e8) at /usr/src/sys/kern/vfs_bio.c:1038
#15 0xffffffff804f5614 in vfsync_bp (bp=0xffffffe010b6b3e8, data=0xffffffe0a647fb90) at /usr/src/sys/kern/vfs_subr.c:865
#16 0xffffffff804f253c in buf_rb_tree_RB_SCAN (head=<optimized out>, scancmp=0xffffffff804f381a <vfsync_lazy_range_cmp>,
callback=<optimized out>, data=0xffffffe0a647fb90) at /usr/src/sys/kern/vfs_subr.c:133
#17 0xffffffff804f52d1 in vfsync (vp=0xffffffe0ae56f400, waitfor=<optimized out>, passes=<optimized out>, checkdef=0,
waitoutput=<optimized out>) at /usr/src/sys/kern/vfs_subr.c:682
#18 0xffffffff806832c7 in hammer_vop_fsync (ap=0xffffffe0a647fc40) at /usr/src/sys/vfs/hammer/hammer_vnops.c:293
#19 0xffffffff80503f60 in vop_fsync (ops=0xffffffe00f10ecc0, vp=0x50000001, waitfor=3332, flags=3333)
at /usr/src/sys/kern/vfs_vopops.c:540
#20 0xffffffff804fa3f2 in syncer_thread () at /usr/src/sys/kern/vfs_sync.c:234
#21 0xffffffff80478be0 in suspend_kproc (td=0xffffffe010b6b4d0, timo=1342177281) at /usr/src/sys/kern/kern_kthread.c:190
Backtrace stopped: previous frame inner to this frame (corrupt stack?)
(kgdb) quit
assertion: (*bmp x%x (1LU << bno)) == 0 in memfree
^C
Abort (core dumped)
sorrow#

kern.3 and vmcore.3 are uploaded to my crash folder on leaf.

I have not been able to reproduce the panic, but the networking issues persist. My ssh connections to the machine regularly end with a "Corrupted MAC on input" error, and FTP transfers consistently stall.