Bug #1126

panic: assertion: volume->io.lock.refs == 0 in hammer_unload_volume

Added by thomas.nikolajsen about 6 years ago. Updated over 4 years ago.

Status:ClosedStart date:
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

I've got this panic multiple times at shutdown;
this is on nfs server using hammer for exports.

Crash dump on leaf in crash/hammer/ask/*.4.

Panic only seen after running buildworld on client w/ usr/src & usr/obj nfs mounted;
no problem when client uses local file system (e.g. hammer) for /usr/obj.

-thomas
-
(kgdb) bt
#0 dumpsys () at ./machine/thread.h:83
#1 0xc01a2226 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:375
#2 0xc01a2347 in panic (fmt=0xc0322fcb "from debugger")
at /usr/src/sys/kern/kern_shutdown.c:800
#3 0xc014a059 in db_panic (addr=-1070669340, have_addr=0, count=-1,
modif=0xc4d6aa08 "") at /usr/src/sys/ddb/db_command.c:447
#4 0xc014a6c4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:343
#5 0xc014cc3c in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_trap.c:71
#6 0xc02ee337 in kdb_trap (type=3, code=0, regs=0xc4d6ab00)
at /usr/src/sys/platform/pc32/i386/db_interface.c:148
#7 0xc02fd586 in trap (frame=0xc4d6ab00)
at /usr/src/sys/platform/pc32/i386/trap.c:842
#8 0xc02ef047 in calltrap ()
at /usr/src/sys/platform/pc32/i386/exception.s:785
#9 0xc02ee1e4 in Debugger (msg=0xc0328dd7 "panic") at ./cpu/cpufunc.h:73
#10 0xc01a233e in panic (fmt=0xd208c47a "assertion: %s in %s")
at /usr/src/sys/kern/kern_shutdown.c:798
#11 0xd2079819 in ?? ()
#12 0xd2077daf in ?? ()
#13 0xd206e5a1 in ?? ()
#14 0xd206e60a in ?? ()
#15 0xc01ef04c in dounmount (mp=0xc0e1c984, flags=524288)
at /usr/src/sys/kern/vfs_syscalls.c:705
#16 0xc01e7f1a in vfs_umountall_callback (mp=0xc7a57c88, data=0x0)
at /usr/src/sys/kern/vfs_subr.c:1619
#17 0xc01eac41 in mountlist_scan (
callback=0xc01e7f07 <vfs_umountall_callback>, data=0x0,
how=<value optimized out>) at /usr/src/sys/kern/vfs_mount.c:816
#18 0xc01e7507 in vfs_unmountall () at /usr/src/sys/kern/vfs_subr.c:1608
#19 0xc01a2086 in boot (howto=0) at /usr/src/sys/kern/kern_shutdown.c:363
#20 0xc01a23e1 in sys_reboot (uap=0xc4d6acf0)
at /usr/src/sys/kern/kern_shutdown.c:192
#21 0xc02fcf0d in syscall2 (frame=0xc4d6ad40)
at /usr/src/sys/platform/pc32/i386/trap.c:1384
#22 0xc02ef0f6 in Xint0x80_syscall ()
at /usr/src/sys/platform/pc32/i386/exception.s:876
#23 0x0804fa10 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

History

#1 Updated by thomas.nikolajsen almost 6 years ago

Have also seen this panic when using local file system for /usr/obj

Crash dump on leaf in crash/hammer/ask/*.12.
(using GENERIC kernel)

#2 Updated by dillon almost 6 years ago

:Thomas Nikolajsen <> added the comment:
:
:Have also seen this panic when using local file system for /usr/obj
:
:Crash dump on leaf in crash/hammer/ask/*.12.
:(using GENERIC kernel)
:
:----------
:priority: -> bug
:status: unread -> chatting

Thomas, please try this patch and tell me if it works.

This code path is very rarely run. It will panic every
time if it gets in there due to the incorrect assertion.

-Matt
Matthew Dillon
<>

Index: hammer_io.c
===================================================================
RCS file: /cvs/src/sys/vfs/hammer/hammer_io.c,v
retrieving revision 1.54
diff -u -p -r1.54 hammer_io.c
--- hammer_io.c 29 Aug 2008 20:19:08 -0000 1.54
+++ hammer_io.c 8 Sep 2008 22:05:47 -0000
@@ -279,7 +279,7 @@ hammer_io_inval(hammer_volume_t volume,
bundirty(bp);
iou->io.reclaim = 1;
iou->io.waitdep = 1;
- KKASSERT(iou->io.lock.refs == 0);
+ KKASSERT(iou->io.lock.refs == 1);
hammer_rel_buffer(&iou->buffer, 0);
/*hammer_io_deallocate(bp);*/
} else {

#3 Updated by dillon almost 6 years ago

:Thomas Nikolajsen <> added the comment:
:
:Have also seen this panic when using local file system for /usr/obj
:
:Crash dump on leaf in crash/hammer/ask/*.12.
:(using GENERIC kernel)

Did you test that patch I emailed? I have included it again.

-Matt

Index: hammer_io.c
===================================================================
RCS file: /cvs/src/sys/vfs/hammer/hammer_io.c,v
retrieving revision 1.54
diff -u -p -r1.54 hammer_io.c
--- hammer_io.c 29 Aug 2008 20:19:08 -0000 1.54
+++ hammer_io.c 8 Sep 2008 22:05:47 -0000
@@ -279,7 +279,7 @@ hammer_io_inval(hammer_volume_t volume,
bundirty(bp);
iou->io.reclaim = 1;
iou->io.waitdep = 1;
- KKASSERT(iou->io.lock.refs == 0);
+ KKASSERT(iou->io.lock.refs == 1);
hammer_rel_buffer(&iou->buffer, 0);
/*hammer_io_deallocate(bp);*/
} else {

#4 Updated by thomas.nikolajsen almost 6 years ago

Works: haven't seen problem with patch.
(but extended patch in issue1139 works even better)

Btw, PFSs was used for nfs exports (as in issue1135 issue1138 issue1139),
but I donĀ“t know if this is relevant for this issue.

(also left another note a few days ago, but got mail that issue tracker barfed)

#5 Updated by dillon over 5 years ago

:I've got this panic multiple times at shutdown;
:this is on nfs server using hammer for exports.
:
:Crash dump on leaf in crash/hammer/ask/*.4.
:
:Panic only seen after running buildworld on client w/ usr/src & usr/obj nfs=
: mounted;
:no problem when client uses local file system (e.g. hammer) for /usr/obj.
:
: -thomas
:-

I made a commit which may or may not fix this. Please report if it
does not, there could also be an issue with shutting down NFS services
on the server side.

-Matt

#6 Updated by corecode over 5 years ago

commit d40bfeca35caa9f1c470a05381618447b19746c9

Also available in: Atom PDF