Bug #1190

HAMMER crash during unmount

Added by matthias over 5 years ago. Updated over 2 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

Hi,

during my recent work with the installer (add HAMMER support) my
DragonFly box paniced while unmounting a HAMMER volume:

panic: assertion: RB_EMPTY(&hmp->rb_inos_root) in hammer_free_hmp
Trace beginning at frame 0xda299bf0
panic(da299c14,1e,da287000,d270b958,da299c28) at panic+0x8c
panic(c0532657,c0559b86,c051a96d,0,d270b958) at panic+0x8c
hammer_free_hmp(d270bb60,d270b958,da299c6c,c032ae0d,d270b958) at
hammer_free_hmp+0xd8
hammer_vfs_unmount(d270b958,0,0,d270b978,1) at hammer_vfs_unmount+0x33
dounmount(d270b958,0,d270b958,0,0) at dounmount+0x1a8
sys_unmount(da299cf0,6,bd34,0,d270a518) at sys_unmount+0xdf
syscall2(da299d40) at syscall2+0x1ef
Xint0x80_syscall() at Xint0x80_syscall+0x36
Debugger("panic")
panic: from debugger
Uptime: 17h54m57s

(kgdb) bt
#0 dumpsys () at ./machine/thread.h:83
#1 0xc02dca22 in boot (howto=260) at
/usr/src/sys/kern/kern_shutdown.c:375
#2 0xc02dcb43 in panic (fmt=0xc0536036 "from debugger") at
/usr/src/sys/kern/kern_shutdown.c:800
#3 0xc01651e5 in db_panic (addr=-1068685624, have_addr=0, count=-1,
modif=0xda299aa4 "") at /usr/src/sys/ddb/db_command.c:447
#4 0xc0165850 in db_command_loop () at
/usr/src/sys/ddb/db_command.c:343
#5 0xc0167e04 in db_trap (type=3, code=0) at
/usr/src/sys/ddb/db_trap.c:71
#6 0xc04d281b in kdb_trap (type=3, code=0, regs=0xda299b9c) at
/usr/src/sys/platform/pc32/i386/db_interface.c:148
#7 0xc04e4075 in trap (frame=0xda299b9c) at
/usr/src/sys/platform/pc32/i386/trap.c:815
#8 0xc04d3527 in calltrap () at
/usr/src/sys/platform/pc32/i386/exception.s:785
#9 0xc04d26c8 in Debugger (msg=0xc054c032 "panic") at
./cpu/cpufunc.h:73
#10 0xc02dcb3a in panic (fmt=0xc0532657 "assertion: %s in %s") at
/usr/src/sys/kern/kern_shutdown.c:798
#11 0xc045cf76 in hammer_free_hmp (mp=0xd270b958) at
/usr/src/sys/vfs/hammer/hammer_vfsops.c:719
#12 0xc045d09f in hammer_vfs_unmount (mp=0xd270b958, mntflags=0) at
/usr/src/sys/vfs/hammer/hammer_vfsops.c:668
#13 0xc032ae0d in dounmount (mp=0xd270b958, flags=0) at
/usr/src/sys/kern/vfs_syscalls.c:699
#14 0xc032b05a in sys_unmount (uap=0xda299cf0) at
/usr/src/sys/kern/vfs_syscalls.c:584
#15 0xc04e3a15 in syscall2 (frame=0xda299d40) at
/usr/src/sys/platform/pc32/i386/trap.c:1357
#16 0xc04d35d6 in Xint0x80_syscall () at
/usr/src/sys/platform/pc32/i386/exception.s:876
#17 0x0804ace8 in ?? ()

The crash dump is already uploading to leaf in my crash/ dir
(vmcore/kernel.0).

The machine itself (VMWare VM) is running HAMMER as root file system:

Filesystem Size Used Avail Capacity Mounted
on
ROOT 5.6G 4.2G 1.4G 75% /
BUILD 20G 2.9G 17G 15% /build
/dev/ad0s1a 252M 107M 125M 46% /boot

ad0: 8192MB <VMware Virtual IDE Hard Drive 00000001> at ata0-master
UDMA33
ad1: 20480MB <VMware Virtual IDE Hard Drive 00000001> at ata0-slave
UDMA33
ad3: 1024MB <VMware Virtual IDE Hard Drive 00000001> at ata1-slave
UDMA33

The installer mounted a third HAMMER volume during the installation run.
I had to cancel the installer and thus had to unmount the volume by
hand. During that unmount the box crashed. Its running

DragonFly hammer01 2.1.1-DEVELOPMENT DragonFly 2.1.1-DEVELOPMENT #0: Sat
Dec 27 21:58:01 GMT 2008 root@hammer01:/usr/obj/usr/src/sys/GENERIC
i386

Regards

Matthias

History

#1 Updated by dillon over 5 years ago

:Hi,
:
:during my recent work with the installer (add HAMMER support) my
:DragonFly box paniced while unmounting a HAMMER volume:
:
:panic: assertion: RB_EMPTY(&hmp->rb_inos_root) in hammer_free_hmp
:Trace beginning at frame 0xda299bf0
:panic(da299c14,1e,da287000,d270b958,da299c28) at panic+0x8c
:panic(c0532657,c0559b86,c051a96d,0,d270b958) at panic+0x8c
:hammer_free_hmp(d270bb60,d270b958,da299c6c,c032ae0d,d270b958) at
:hammer_free_hmp+0xd8
:hammer_vfs_unmount(d270b958,0,0,d270b978,1) at hammer_vfs_unmount+0x33
:dounmount(d270b958,0,d270b958,0,0) at dounmount+0x1a8
:sys_unmount(da299cf0,6,bd34,0,d270a518) at sys_unmount+0xdf
:syscall2(da299d40) at syscall2+0x1ef
:Xint0x80_syscall() at Xint0x80_syscall+0x36
:Debugger("panic")
:panic: from debugger
:Uptime: 17h54m57s

Here's the dmesg from your crash dump on leaf:

pid 20683 (dfuibe_installer), uid 0: exited on signal 11 (core dumped)
pid 20966 (dfuibe_installer), uid 0: exited on signal 11 (core dumped)
unmount(/mnt): Cannot unmount: 0 namecache references still present
unmount(/mnt): Cannot unmount: 2 process references still present
HAMMER: umount flushing..........................giving up
panic: assertion: RB_EMPTY(&hmp->rb_inos_root) in hammer_free_hmp
Trace beginning at frame 0xda299bf0

For some reason it was having a hard time flushing dirty inodes.

Could you tell if the disk drive was solid on while it was trying
to umount? There should have been about a 25-second delay (each
'.' is one second, theoretically).

-Matt

#2 Updated by matthias over 5 years ago

Hope I got your question right. The disk is an additional VMWare disk
and was only used to test the installer. The procedure was to label the
disk, install DF on the HAMMER partition and let the installer unmount
it. Nothing special and no big load. The disk was idle most of the
time. Strange thing here was that this panic happened after about 10
slice/mount/install/unmount cycles. After a reboot of the VMWare VM it
worked again for about 10 test cycles.

Cheers

Matthias

#3 Updated by dillon over 5 years ago

:Hope I got your question right. The disk is an additional VMWare disk
:and was only used to test the installer. The procedure was to label the
:disk, install DF on the HAMMER partition and let the installer unmount
:it. Nothing special and no big load. The disk was idle most of the
:time. Strange thing here was that this panic happened after about 10
:slice/mount/install/unmount cycles. After a reboot of the VMWare VM it
:worked again for about 10 test cycles.
:
:Cheers
:
: Matthias

Very odd. Try w/ the lastest master. It has this patch:

http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/d40bfeca35caa9f1c470a05381618447b19746c9

Which I hope solves the umount issue. If not we will have to
investigate more.

-Matt
Matthew Dillon
<>

#4 Updated by matthias about 5 years ago

No luck. VM crashed after the first install cycle. Backtrace stays the
same:

#0 dumpsys () at ./machine/thread.h:83
#1 0xc02e21ee in boot (howto=260) at
/build/src/sys/kern/kern_shutdown.c:376
#2 0xc02e230f in panic (fmt=0xc053c516 "from debugger") at
/build/src/sys/kern/kern_shutdown.c:801
#3 0xc0165705 in db_panic (addr=-1068660116, have_addr=0, count=-1,
modif=0xd9965aa0 "") at /build/src/sys/ddb/db_command.c:447
#4 0xc0165d70 in db_command_loop () at
/build/src/sys/ddb/db_command.c:343
#5 0xc0168344 in db_trap (type=3, code=0) at
/build/src/sys/ddb/db_trap.c:71
#6 0xc04d8bbf in kdb_trap (type=3, code=0, regs=0xd9965b98) at
/build/src/sys/platform/pc32/i386/db_interface.c:148
#7 0xc04ea425 in trap (frame=0xd9965b98) at
/build/src/sys/platform/pc32/i386/trap.c:815
#8 0xc04d98d7 in calltrap () at
/build/src/sys/platform/pc32/i386/exception.s:785
#9 0xc04d8a6c in Debugger (msg=0xc0552b61 "panic") at
./cpu/cpufunc.h:73
#10 0xc02e2306 in panic (fmt=0xc0538b37 "assertion: %s in %s") at
/build/src/sys/kern/kern_shutdown.c:799
#11 0xc0463079 in hammer_free_hmp (mp=0xd270cfd8) at
/build/src/sys/vfs/hammer/hammer_vfsops.c:715
#12 0xc04631a2 in hammer_vfs_unmount (mp=0xd270cfd8, mntflags=0) at
/build/src/sys/vfs/hammer/hammer_vfsops.c:663
#13 0xc033087d in dounmount (mp=0xd270cfd8, flags=0) at
/build/src/sys/kern/vfs_syscalls.c:701
#14 0xc0330ace in sys_unmount (uap=0xd9965cf0) at
/build/src/sys/kern/vfs_syscalls.c:586
#15 0xc04e9dc5 in syscall2 (frame=0xd9965d40) at
/build/src/sys/platform/pc32/i386/trap.c:1357
#16 0xc04d9986 in Xint0x80_syscall () at
/build/src/sys/platform/pc32/i386/exception.s:876

The crash dump is available at:
http://www.mathematik.uni-marburg.de/~schmidtm/crash/ but I'm not sure if it
contains any new information.

Cheers

Matthias

#5 Updated by tuxillo over 4 years ago

Matthias,
I installed my VM under VMware HAMMER-only without noticing this problem.
How can I reproduce it?

#6 Updated by ftigeot over 2 years ago

  • Description updated (diff)
  • Status changed from New to Resolved
  • Assignee deleted (0)

Closing due to lack of recent feedback.

#7 Updated by ftigeot over 2 years ago

  • Status changed from Resolved to Closed

Also available in: Atom PDF