Bug #1125

panic: assertion: iou->io.lock.refs == 0 in hammer_io_inval

Added by thomas.nikolajsen about 6 years ago. Updated almost 6 years ago.

Status:ClosedStart date:
Priority:LowDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

I get this panic consistently after 5-20 minutes when doing buildworld,
using nfs mount for /usr/src & /usr/obj and using hammer on nfs server.

Crash dump on leaf in crash/hammer/ask/*.2.

No panic when using local file system (e.g. hammer) for /usr/obj.

-thomas
-
(kgdb) bt
#0 dumpsys () at ./machine/thread.h:83
#1 0xc01a2226 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:375
#2 0xc01a2347 in panic (fmt=0xc0322fcb "from debugger") at /usr/src/sys/kern/kern_shutdown.c:800
#3 0xc014a059 in db_panic (addr=-1070669340, have_addr=0, count=-1, modif=0xd23a8460 "") at /usr/src/sys/ddb/db_command.c:447
#4 0xc014a6c4 in db_command_loop () at /usr/src/sys/ddb/db_command.c:343
#5 0xc014cc3c in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_trap.c:71
#6 0xc02ee337 in kdb_trap (type=3, code=0, regs=0xd23a8558) at /usr/src/sys/platform/pc32/i386/db_interface.c:148
#7 0xc02fd586 in trap (frame=0xd23a8558) at /usr/src/sys/platform/pc32/i386/trap.c:842
#8 0xc02ef047 in calltrap () at /usr/src/sys/platform/pc32/i386/exception.s:785
#9 0xc02ee1e4 in Debugger (msg=0xc0328dd7 "panic") at ./cpu/cpufunc.h:73
#10 0xc01a233e in panic (fmt=0xd208c47a "assertion: %s in %s") at /usr/src/sys/kern/kern_shutdown.c:798
#11 0xd207c5dc in ?? ()
#12 0xd207adcb in ?? ()
#13 0xd20859ba in ?? ()
#14 0xd2081ac7 in ?? ()
#15 0xd20824f6 in ?? ()
#16 0xd2070cd5 in ?? ()
#17 0xc01f2e16 in vop_strategy (ops=0xc4d86a98, vp=0xd27a4590, bio=0xc0e3bf98) at /usr/src/sys/kern/vfs_vopops.c:659
#18 0xc01d7a72 in vn_strategy (vp=0xc03ffb6c, bio=0x7020) at /usr/src/sys/kern/vfs_bio.c:3089
#19 0xc01db0b2 in bwrite (bp=0xc0e3bf68) at /usr/src/sys/kern/vfs_bio.c:797
#20 0xd2072125 in ?? ()
#21 0xc01f2e72 in vop_write (ops=0xc4d86a98, vp=0xd27a4590, uio=0xd23a8954, ioflag=12, cred=0xc7a7a658)
at /usr/src/sys/kern/vfs_vopops.c:351
#22 0xc0272f00 in nfsrv_write (nfsd=0xc7a7a5d8, slp=0xc4d8b958, td=0xc7a77e78, mrq=0xd23a8bac)
at /usr/src/sys/vfs/nfs/nfs_serv.c:1105
#23 0xc027f46d in sys_nfssvc (uap=0xd23a8cf0) at /usr/src/sys/vfs/nfs/nfs_syscalls.c:590
#24 0xc02fcf0d in syscall2 (frame=0xd23a8d40) at /usr/src/sys/platform/pc32/i386/trap.c:1384
#25 0xc02ef0f6 in Xint0x80_syscall () at /usr/src/sys/platform/pc32/i386/exception.s:876
#26 0x080498c0 in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

History

#1 Updated by thomas.nikolajsen about 6 years ago

Panic is on nfs server; no crash seen on client.

-thomas
-
Thomas Nikolajsen wrote:
> I get this panic consistently after 5-20 minutes when doing buildworld,
> using nfs mount for /usr/src & /usr/obj and using hammer on nfs
> server.
>
> Crash dump on leaf in crash/hammer/ask/*.2.
>
> No panic when using local file system (e.g. hammer) for /usr/obj.
>
> -thomas
..

#2 Updated by thomas.nikolajsen almost 6 years ago

I have no problem reproducing this panic.

Generated another crash dump using GENERIC kernel
(might be easier to debug: HAMMER not loaded as module):
Crash dump on leaf in crash/hammer/ask/*.11.

Also tried another host as nfs server: minimal setup:
- fresh HAMMER file system, with just one PFS (#1)
- data for /usr/src cpdup'ed from other nfs server
- empty /usr/obj
- more RAM (512MB vs 128MB)
Got same panic, after a few minutes of buildworld;
crash dump avail. on request (200MB).

It seems like I get this panic when using write intensive operations
on nfs mounts. Read mostly operations doesn't trigger bug.

#3 Updated by thomas.nikolajsen almost 6 years ago

Was nfs exporting HAMMER PFSs which isn't supported yet.

See issue1135 and issue1138

Also available in: Atom PDF