DragonFlyBSD

Am 13/06/2010 01:09 μμ, schrieb Alex Hornung (via DragonFly issue tracker):

New submission from Alex Hornung<ahornung@gmail.com>:

Dump is in my ~/crash on leaf, it's kern.0/vmcore.0. It happend with the most
recent master as of now and doing an 'ls' after a branch switch in git.

Well, the last 3 tokrefs in td->td_toks_array are to the vm_token (this
is OK, recursive acquisition is normal in this path) while the rest 29
tokrefs are to proc_token (this is not OK, in-kernel recursion can't be
that deep). Seems like something is wrong on some release path :)

I'd add a check for remaining tokrefs in ->td_toks_array just before
returning to userland to narrow down the search. Matt and/or Venkatesh
who did the token changes might already have some suspect path in mind :)

Aggelos

9:59:41 dragon:/var/crash
kgdb kern.0 vmcore.0
GNU gdb (GDB) 7.0
Copyright (C) 2009 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later<http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "i386-dragonfly".
For bug reporting instructions, please see:
<http://bugs.dragonflybsd.org/&gt;...
Reading symbols from /var/crash/kern.0...done.

Unread portion of the kernel message buffer:
panic: assertion: ref< &td->td_toks_end in lwkt_gettoken
mp_lock = 00000002; cpuid = 2
Trace beginning at frame 0xd37b1b60
panic(ffffffff) at panic+0x14f
panic(c05cc483,c05e851f,c05ac87a,d38ee250,c1098650) at panic+0x14f
lwkt_gettoken(c0702698,d38ee250,c1098650,d37b1bc4,c04f6590) at lwkt_gettoken+0x36
vm_page_remove(c1098650,c1098650,c1098650,c1098650,d37b1bd4) at vm_page_remove+0x2a
vm_page_free_toq(c1098650,c1098650,d37b1bfc,c04f437e,c1098650) at
vm_page_free_toq+0xb7
vm_object_terminate_callback(c1098650,0,0,0,c0e0bea0) at
vm_object_terminate_callback+0x46
vm_page_rb_tree_RB_SCAN(d49ad6e0,0,c04f21d4,0,0) at vm_page_rb_tree_RB_SCAN+0xad
vm_object_terminate(d49ad6cc,d49ae618,d49ad6cc,284ce000,d37b1c78) at
vm_object_terminate+0x182
vm_object_deallocate(d49ad6cc) at vm_object_deallocate+0x2bb
vm_map_delete(d3a1ac70,284ca000,284ce000,d37b1c90,4) at vm_map_delete+0x2b2
vm_map_remove(d3a1ac70,284ca000,284ce000,d0bfd1d0,d47927e8) at vm_map_remove+0x52
sys_munmap(d37b1cf0,6,65e82,0,c0691aec) at sys_munmap+0x87
syscall2(d37b1d40) at syscall2+0x3ac
Xint0x80_syscall() at Xint0x80_syscall+0x36
Debugger("panic")

CPU2 stopping CPUs: 0x0000000b
stopped
Physical memory: 1015 MB
Dumping 234 MB: 219 203 187 171 155 139 123 107 91 75 59 43 27 11

Reading symbols from /boot/modules/dsched_fq.ko...done.
Loaded symbols for /boot/modules/dsched_fq.ko
Reading symbols from /boot/modules/acpi.ko...done.
Loaded symbols for /boot/modules/acpi.ko
Reading symbols from /boot/modules/linux.ko...done.
Loaded symbols for /boot/modules/linux.ko
get_mycpu (di=0xc06fce00) at ./machine/thread.h:83
83 __asm ("movl %%fs:globaldata,%0" : "=r" (gd) : "m"(_mycpu__dummy));
(kgdb) bt
#0 _get_mycpu (di=0xc06fce00) at ./machine/thread.h:83
#1 md_dumpsys (di=0xc06fce00) at /usr/src/sys/platform/pc32/i386/dump_machdep.c:263
#2 0xc03204e9 in dumpsys () at /usr/src/sys/kern/kern_shutdown.c:838
#3 0xc017a455 in db_fncall (dummy1=2, dummy2=0, dummy3=-1068058868,
dummy4=0xd37b1a08 "") at /usr/src/sys/ddb/db_command.c:542
#4 0xc017a946 in db_command () at /usr/src/sys/ddb/db_command.c:344
#5 db_command_loop () at /usr/src/sys/ddb/db_command.c:470
#6 0xc017cf84 in db_trap (type=3, code=0) at /usr/src/sys/ddb/db_trap.c:71
#7 0xc056b92f in kdb_trap (type=3, code=0, regs=0xd37b1b10)
at /usr/src/sys/platform/pc32/i386/db_interface.c:152
#8 0xc05849e3 in trap (frame=0xd37b1b10) at
/usr/src/sys/platform/pc32/i386/trap.c:837
#9 0xc056cce7 in calltrap () at /usr/src/sys/platform/pc32/i386/exception.s:785
#10 0xc056b70c in breakpoint (msg=0xc05e7d82 "panic") at ./cpu/cpufunc.h:73
#11 Debugger (msg=0xc05e7d82 "panic")
at /usr/src/sys/platform/pc32/i386/db_interface.c:334
#12 0xc0321015 in panic (fmt=0xc05cc483 "assertion: %s in %s")
at /usr/src/sys/kern/kern_shutdown.c:742
#13 0xc032c863 in lwkt_gettoken (tok=0xc077650c) at
/usr/src/sys/kern/lwkt_token.c:425
#14 0xc04f6113 in vm_page_remove (m=0xc1098650) at /usr/src/sys/vm/vm_page.c:439
#15 0xc04f6590 in vm_page_free_toq (m=0xc1098650) at /usr/src/sys/vm/vm_page.c:1030
#16 0xc04f221a in vm_page_free (p=0xc1098650, data=0x0) at
/usr/src/sys/vm/vm_page.h:613
#17 vm_object_terminate_callback (p=0xc1098650, data=0x0)
at /usr/src/sys/vm/vm_object.c:493
#18 0xc04f437e in vm_page_rb_tree_RB_SCAN (head=0xd49ad6e0,
scancmp=0xc04f4288<vm_page_rb_tree_SCANCMP_ALL>,
callback=0xc04f21d4<vm_object_terminate_callback>, data=0x0)
at /usr/src/sys/vm/vm_page.c:108
#19 0xc04f378c in vm_object_terminate (object=0xd49ad6cc)
at /usr/src/sys/vm/vm_object.c:456
#20 0xc04f4265 in vm_object_deallocate (object=0xd49ad6cc)
at /usr/src/sys/vm/vm_object.c:392
#21 0xc04ee19a in vm_map_delete (map=0xd3a1ac70, start=676110336, end=676126720,
countp=0xd37b1c90) at /usr/src/sys/vm/vm_map.c:2571
#22 0xc04ee21d in vm_map_remove (map=0xd3a1ac70, start=676110336, end=676126720)
at /usr/src/sys/vm/vm_map.c:2727
#23 0xc04f1b46 in sys_munmap (uap=0xd37b1cf0) at /usr/src/sys/vm/vm_mmap.c:566
#24 0xc0583d6a in syscall2 (frame=0xd37b1d40)
at /usr/src/sys/platform/pc32/i386/trap.c:1319
#25 0xc056cd96 in Xint0x80_syscall () at
/usr/src/sys/platform/pc32/i386/exception.s:876
#26 0x0000001f in ?? ()
Backtrace stopped: previous frame inner to this frame (corrupt stack?)

----------
messages: 8666
nosy: alexh
priority: urgent
status: unread
title: lwkt mpsafing related panic

_____________________________________________
DragonFly issue tracker<bugs@lists.dragonflybsd.org>
<http://bugs.dragonflybsd.org/issue1781>
_____________________________________________

On 6/13/2010 13:35, Aggelos Economopoulos wrote:

Am 13/06/2010 01:09 μμ, schrieb Alex Hornung (via DragonFly issue tracker):

New submission from Alex Hornung<ahornung@gmail.com>:

Dump is in my ~/crash on leaf, it's kern.0/vmcore.0. It happend with
the most
recent master as of now and doing an 'ls' after a branch switch in git.

I have similar issues here (assertion in lwkt_gettoken()) on x86_64 when
doing git related things. One panic was when doing a git status, another
when grepping for things in git log output. lwkt_gettoken() was called
from different places (pipeclose() or pmap_release_free_page()).

Sascha

Ok, should hopefully be fixed now. Two of the procedures in kern_proc.c
were leaking tokens. I need to put a test in the system call code so
the panic is caught earlier.

If not fixed there's another leaker somewhere.

-Matt

Fixed in 46270ec662f96487806e6b37f7403538568019f3 and
6f9db61576fd340f7ae1d1431fb6ad845bd73aa2