NNINPABE1NB02 dumped core - see /var/crash/vmcore.0 Thu Feb 20 05:12:54 EST 2014 Version String: DragonFly v3.6.0-RELEASE #8: Fri Nov 22 13:22:00 PST 2013 root@pkgbox64.dragonflybsd.org:/usr/obj/build/home/justin/src/sys/X86_64_GENERIC panic: from debugger GNU gdb (GDB) 7.6.1 Copyright (C) 2013 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-dragonfly". For bug reporting instructions, please see: ... Reading symbols from /boot/kernel/kernel...done. Unread portion of the kernel message buffer: panic: assertion "tdio->qlength == 0" failed in dsched_thread_io_destroy at /build/home/justin/src/sys/kern/kern_dsched.c:842 cpuid = 3 Trace beginning at frame 0xffffffe12e1771c0 panic() at panic+0x223 0xffffffff80561c0c panic() at panic+0x223 0xffffffff80561c0c dsched_thread_io_destroy() at dsched_thread_io_destroy+0x4a 0xffffffff805795f6 dsched_thread_io_unref_destroy() at dsched_thread_io_unref_destroy+0x63 0xffffffff8057994e dsched_thread_ctx_unref() at dsched_thread_ctx_unref+0x152 0xffffffff80579476 dsched_exit_buf() at dsched_exit_buf+0x28 0xffffffff8057aad8 bqrelse() at bqrelse+0x406 0xffffffff805c2ce3 bpdone() at bpdone+0x3f3 0xffffffff805c30e8 biodone() at biodone+0x1d7 0xffffffff805c3350 hammer_io_direct_write_complete() at hammer_io_direct_write_complete+0xd0 0xffffffff80764682 biodone() at biodone+0x1bb 0xffffffff805c3334 fq_completed() at fq_completed+0x1c1 0xffffffff8080808d biodone() at biodone+0x1bb 0xffffffff805c3334 dadone() at dadone+0x32b 0xffffffff8029dd47 camisr_runqueue() at camisr_runqueue+0x3ff 0xffffffff80292db8 swi_cambio() at swi_cambio+0x126 0xffffffff80292fab ithread_handler() at ithread_handler+0x1e5 0xffffffff80535709 Debugger("panic") CPU3 stopping CPUs: 0x000000f7 stopped panic: from debugger cpuid = 3 boot() called on cpu#3 Uptime: 15m35s Physical memory: 8154 MB Dumping 1868 MB: 1853 1837 1821 1805 1789 1773 1757 1741 1725 1709 1693 1677 1661 1645 1629 1613 1597 1581 1565 1549 1533 1517 1501 1485 1469 1453 1437 1421 1405 1389 1373 1357 1341 1325 1309 1293 1277 1261 1245 1229 1213 1197 1181 1165 1149 1133 1117 1101 1085 1069 1053 1037 1021 1005 989 973 957 941 925 909 893 877 861 845 829 813 797 781 765 749 733 717 701 685 669 653 637 621 605 589 573 557 541 525 509 493 477 461 445 429 413 397 381 365 349 333 317 301 285 269 253 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13 [New pid 17799/0, time] [New pid 17738/0, bmake] [New pid 863/0, bash] [New pid 775/0, dhclient] [New pid 767/0, getty] [New pid 623/0, usbd] [New kernel vnlru] [New kernel pagezero] [New kernel hammer-S2] [New kernel usb4] [New kernel ithread17 7] [New kernel ithread16 7] [New kernel ithreadE 7] [New kernel mpt_recovery0] [New kernel crypto 7] [New kernel taskq_cpu 7] [New kernel ifnet 7] [New kernel netisr_cpu 7] [New kernel usched 7] [New kernel usched 7] [New kernel dsched 7] [New kernel softclock 7] [New kernel idle_7] [New pid 17798/0, sh] [New kernel fq_balance_da1] [New pid 881/0, bmake] [New pid 874/0, su] [New pid 862/0, sshd] [New pid 766/0, getty] [New kernel bufdaemon_hw] [New kernel hammer-S1] [New kernel usb3] [New kernel ithread17 6] [New kernel ithread16 6] [New kernel ithreadE 6] [New kernel acpi_task] [New kernel crypto 6] [New kernel taskq_cpu 6] [New kernel ifnet 6] [New kernel netisr_cpu 6] [New kernel usched 6] [New kernel usched 6] [New kernel dsched 6] [New kernel softclock 6] [New kernel idle_6] [New pid 901/0, sh] [New kernel fq_dispatch_da1] [New pid 860/0, sshd] [New pid 789/0, sshd] [New pid 764/0, login] [New pid 765/0, getty] [New kernel bufdaemon] [New kernel hammer-S0] [New kernel usb2] [New kernel ithread17 5] [New kernel ithread16 5] [New kernel xpt_thrd] [New kernel ithread197 5] [New kernel ithreadE 5] [New kernel crypto 5] [New kernel taskq_cpu 5] [New kernel ifnet 5] [New kernel netisr_cpu 5] [New kernel usched 5] [New kernel usched 5] [New kernel dsched 5] [New kernel softclock 5] [New kernel idle_5] [New pid 17804/0, dd] [New kernel fq_balance_da0] [New pid 771/0, getty] [New pid 676/0, sendmail] [New kernel hammer-M] [New kernel usb1] [New kernel ithread17 4] [New kernel ithread16 4] [New kernel ithreadE 4] [New kernel crypto returns] [New kernel crypto 4] [New kernel taskq_cpu 4] [New kernel ifnet 4] [New kernel netisr_cpu 4] [New kernel usched 4] [New kernel usched 4] [New kernel dsched 4] [New kernel softclock 4] [New kernel idle_4] [New pid 17803/0, dd] [New kernel fq_dispatch_da0] [New pid 875/0, sh] [New pid 787/0, sshd] [New pid 770/0, getty] [New kernel vmdaemon] [New kernel syncer3] [New kernel usbtask-dr] [New kernel ithread17 3] [New kernel ithread16 3] [New kernel ithread195 3] [New kernel ithreadE 3] [New kernel crypto 3] [New kernel firmware taskq] [New kernel taskq_cpu 3] [New kernel ifnet 3] [New kernel netisr_cpu 3] [New kernel usched 3] [New kernel usched 3] [New kernel dsched 3] [New kernel softclock 3] [New kernel idle_3] [New pid 17802/0, sh] [New pid 890/0, bmake] [New pid 786/0, dhclient] [New pid 769/0, getty] [New pid 658/0, sshd] [New kernel syncer13] [New kernel syncer5] [New kernel swapcached] [New kernel syncer2] [New kernel usbtask-hc] [New kernel ithread17 2] [New kernel ithread16 2] [New kernel ithreadE 2] [New kernel crypto 2] [New kernel taskq_cpu 2] [New kernel ifnet 2] [New kernel disk_msg_core] [New kernel netisr_cpu 2] [New kernel usched 2] [New kernel usched 2] [New kernel dsched 2] [New kernel softclock 2] [New kernel idle_2] [New pid 17801/0, sh] [New pid 790/0, bash] [New pid 768/0, getty] [New pid 328/0, syslogd] [New kernel syncer4] [New kernel pagedaemon] [New pid 1/0, init] [New kernel syncer1] [New kernel usb0] [New kernel ithread17 1] [New kernel ithread16 1] [New kernel ithreadE 1] [New kernel crypto 1] [New kernel taskq_cpu 1] [New kernel ifnet 1] [New kernel netisr_cpu 1] [New kernel devfs_msg_core] [New kernel usched 1] [New kernel usched 1] [New kernel dsched 1] [New kernel softclock 1] [New kernel idle_1] [New pid 17800/0, time] [New pid 772/0, sh] [New pid 691/0, cron] [New pid 672/0, sendmail] [New kernel consttyd] [New kernel hammer-S3] [New kernel random] [New kernel ithread3 0] [New kernel ithread4 0] [New kernel ithread192 0] [New kernel usb5] [New kernel ithread19 0] [New kernel ithread18 0] [New kernel ithread17 0] [New kernel ithread16 0] [New kernel ithread22 0] [New kernel ithread60 0] [New pid 0/0, swapper] [New kernel ithread59 0] [New kernel ithread58 0] [New kernel ithread57 0] [New kernel mpt_raid0] [New kernel ithread56 0] [New kernel ithread9 0] [New kernel crypto 0] [New kernel taskq_cpu 0] [New kernel ifnet 0] [New kernel netisr_cpu 0] [New kernel usched 0] [New kernel usched 0] [New kernel dsched 0] [New kernel ithread196 0] [New kernel ithreadE 0] [New kernel softclock 0] [New kernel idle_0] Reading symbols from /boot/kernel/acpi.ko...(no debugging symbols found)...done. Loaded symbols for /boot/kernel/acpi.ko Reading symbols from /boot/kernel/ehci.ko...(no debugging symbols found)...done. Loaded symbols for /boot/kernel/ehci.ko _get_mycpu () at ./machine/thread.h:69 69 ./machine/thread.h: No such file or directory. (kgdb) #0 _get_mycpu () at ./machine/thread.h:69 #1 md_dumpsys (di=di@entry=0xffffffff80f2df60 ) at /build/home/justin/src/sys/platform/pc64/x86_64/dump_machdep.c:265 #2 0xffffffff80561472 in dumpsys () at /build/home/justin/src/sys/kern/kern_shutdown.c:912 #3 0xffffffff8056193f in boot (howto=howto@entry=260) at /build/home/justin/src/sys/kern/kern_shutdown.c:369 #4 0xffffffff80561c35 in panic ( fmt=fmt@entry=0xffffffff80984aa4 "from debugger") at /build/home/justin/src/sys/kern/kern_shutdown.c:818 #5 0xffffffff802b08c5 in db_panic (addr=, have_addr=, count=, modif=) at /build/home/justin/src/sys/ddb/db_command.c:445 #6 0xffffffff802b0f43 in db_command (aux_cmd_tablep_end=, aux_cmd_tablep=, cmd_table=, last_cmdp=0xffffffff80daa430 ) at /build/home/justin/src/sys/ddb/db_command.c:401 #7 db_command_loop () at /build/home/justin/src/sys/ddb/db_command.c:467 #8 0xffffffff802b3b19 in db_trap (type=type@entry=3, code=code@entry=0) at /build/home/justin/src/sys/ddb/db_trap.c:71 #9 0xffffffff8092a06f in kdb_trap (type=type@entry=3, code=code@entry=0, regs=regs@entry=0xffffffe12e1770f8) at /build/home/justin/src/sys/platform/pc64/x86_64/db_interface.c:174 #10 0xffffffff8092fb58 in trap (frame=0xffffffe12e1770f8) at /build/home/justin/src/sys/platform/pc64/x86_64/trap.c:726 #11 0xffffffff80919d2f in calltrap () at /build/home/justin/src/sys/platform/pc64/x86_64/exception.S:188 #12 0xffffffff8092a2f3 in breakpoint () at ./cpu/cpufunc.h:73 #13 Debugger (msg=msg@entry=0xffffffff80a06dae "panic") at /build/home/justin/src/sys/platform/pc64/x86_64/db_interface.c:357 #14 0xffffffff80561c1a in panic ( fmt=fmt@entry=0xffffffff809799e8 "assertion \"%s\" failed in %s at %s:%u") at /build/home/justin/src/sys/kern/kern_shutdown.c:813 #15 0xffffffff805795f6 in dsched_thread_io_destroy ( tdio=tdio@entry=0xffffffe134943520) at /build/home/justin/src/sys/kern/kern_dsched.c:842 #16 0xffffffff8057994e in dsched_thread_io_unref_destroy ( tdio=tdio@entry=0xffffffe134943520) at /build/home/justin/src/sys/kern/kern_dsched.c:824 #17 0xffffffff80579476 in dsched_thread_ctx_destroy (tdctx=0xffffffe1348a8cf0) at /build/home/justin/src/sys/kern/kern_dsched.c:953 #18 dsched_thread_ctx_unref (tdctx=0xffffffe1348a8cf0) at /build/home/justin/src/sys/kern/kern_dsched.c:928 #19 0xffffffff8057aad8 in dsched_exit_buf (bp=bp@entry=0xffffffe01b6120e0) at /build/home/justin/src/sys/kern/kern_dsched.c:1144 #20 0xffffffff805c2ce3 in bqrelse (bp=bp@entry=0xffffffe01b6120e0) at /build/home/justin/src/sys/kern/vfs_bio.c:1726 #21 0xffffffff805c30e8 in bpdone (bp=bp@entry=0xffffffe01b6120e0, elseit=elseit@entry=1) at /build/home/justin/src/sys/kern/vfs_bio.c:3919 #22 0xffffffff805c3350 in biodone (bio=0x0, bio@entry=0xffffffe01b612140) at /build/home/justin/src/sys/kern/vfs_bio.c:3966 #23 0xffffffff80764682 in hammer_io_direct_write_complete ( nbio=) at /build/home/justin/src/sys/vfs/hammer/hammer_io.c:1769 #24 0xffffffff805c3334 in biodone (bio=0xffffffe01b6121c8) at /build/home/justin/src/sys/kern/vfs_bio.c:3957 #25 0xffffffff8080808d in fq_completed (bp=0xffffffe01b612360) at /build/home/justin/src/sys/kern/dsched/fq/fq_diskops.c:328 #26 0xffffffff805c3334 in biodone (bio=bio@entry=0xffffffe01b612360) at /build/home/justin/src/sys/kern/vfs_bio.c:3957 #27 0xffffffff8029dd47 in dadone (periph=0xffffffe1398b4b10, done_ccb=0xffffffe15f27c580) at /build/home/justin/src/sys/bus/cam/scsi/scsi_da.c:1782 #28 0xffffffff80292db8 in camisr_runqueue (sim=sim@entry=0xffffffe01ab4e400) at /build/home/justin/src/sys/bus/cam/cam_xpt.c:7381 #29 0xffffffff80292fab in camisr (dummy=0x0) at /build/home/justin/src/sys/bus/cam/cam_xpt.c:7279 #30 swi_cambio (arg=, frame=) at /build/home/justin/src/sys/bus/cam/cam_xpt.c:7261 #31 0xffffffff80535709 in ithread_handler (arg=) at /build/home/justin/src/sys/kern/kern_intr.c:844 #32 0xffffffff80571897 in lwkt_deschedule_self (td=) at /build/home/justin/src/sys/kern/lwkt_thread.c:327 #33 0x0000000000000000 in ?? () (kgdb) (kgdb) Token collisions owner pmap_token 0 not held dev_token 0 not held vm_token 0 not held vmspace_token 0 not held kvm_token 0 not held No symbol "proc_token" in current context. (kgdb) Cputime limit exceeded ------------------------------------------------------------------------ ps -axl ps: _allproc: no such symbol ------------------------------------------------------------------------ vmstat -s 83691 cpu context switches 439525 device interrupts 1076 software interrupts 0 traps 207204 system calls 1 kernel threads created 155 fork() calls 223 vfork() calls 0 rfork() calls 259 exec() calls 0 swap pager pageins 0 swap pager pages paged in 0 swap pager pageouts 0 swap pager pages paged out 113 vnode pager pageins 113 vnode pager pages paged in 0 vnode pager pageouts 0 vnode pager pages paged out 0 page daemon wakeups 0 pages examined by the page daemon 0 pages reactivated 5607 copy-on-write faults 0 copy-on-write optimized faults 477083 zero fill pages zeroed 154898 zero fill pages prezeroed 23 intransit blocking page faults 607155 total VM faults taken 0 pages affected by kernel thread creation 9901 pages affected by fork() 14272 pages affected by vfork() 0 pages affected by rfork() 1666638 pages freed 0 pages freed by daemon 102309 pages freed by exiting processes 88450 pages active 714886 pages inactive 76 pages in VM cache 342075 pages wired down 876654 pages free 4096 bytes per page 319834 global smp invltlbs 2393 total path lookups 5510 total component lookups cache hits (85% pos + 4% neg) deletions 0%, falsehits 0% ------------------------------------------------------------------------ vmstat -m Memory statistics by type Type Kern Type InUse MemUse HighUse Limit Requests Limit Limit HAMMER-inodes 1860 1628K 0K 134203388K 1901 0 0 HAMMER-others 3014 560K 0K 808756K 219190 0 0 acpica 409 15K 0K 808756K 111193 0 0 acpidev 151 6K 0K 808756K 151 0 0 acpisem 20 1K 0K 808756K 20 0 0 pci_link 80 5K 0K 808756K 80 0 0 acpitask 0 0K 0K 808756K 2 0 0 lwkt message 21 13K 0K 808756K 2275 0 0 thread 131 181K 0K 808756K 131 0 0 atkbddev 2 1K 0K 808756K 2 0 0 memdesc 1 3K 0K 808756K 1 0 0 MPipe Array 2 3K 0K 808756K 2 0 0 cache 2809 260K 0K 808756K 2840 0 0 nexusdev 3 1K 0K 808756K 3 0 0 devbuf 820 994K 0K 808756K 892 0 0 temp 271 36K 0K 808756K 407796 0 0 syscons 41 167K 0K 808756K 41 0 0 ip6ndp 8 1K 0K 808756K 8 0 0 CAM dev queue 6 1K 0K 808756K 6 0 0 crypto 1 1K 0K 808756K 1 0 0 propstng 1387 55K 0K 808756K 1387 0 0 prop string 1381 11K 0K 808756K 1487 0 0 propnmbr 1625 89K 0K 808756K 1625 0 0 pdict16 20 2K 0K 808756K 20 0 0 propdict 679 128K 0K 808756K 679 0 0 prop dictionary 676 169K 0K 808756K 724 0 0 kbdmux 7 8K 0K 808756K 7 0 0 isadev 22 2K 0K 808756K 22 0 0 USB 112 12K 0K 808756K 130 0 0 USBdev 11 2K 0K 808756K 32 0 0 ZONE 1 1K 0K 808756K 1 0 0 uidinfo 5 9K 0K 808756K 16 0 0 cred 14 2K 0K 808756K 118 0 0 pgrp 26 4K 0K 808756K 64 0 0 session 20 2K 0K 808756K 25 0 0 vmspace 94 83K 0K 808756K 98 0 0 proc 37 42K 0K 808756K 17804 0 0 lwp 37 24K 0K 808756K 17804 0 0 subproc 53 124K 0K 808756K 17834 0 0 objcache 60 30K 0K 808756K 60 0 0 HAMMER-mount 1 68K 0K 808756K 1 0 0 objcache magazine 2852 2319K 0K 808756K 2852 0 0 devfs 3745 593K 0K 808756K 6639 0 0 lockf 3 1K 0K 808756K 108 0 0 UFS dirhash 9 7K 0K 808756K 9 0 0 UFS mount 3 5K 0K 808756K 3 0 0 UFS ihash 1 2048K 0K 808756K 1 0 0 FFS node 3 1K 0K 134203388K 3 0 0 pagedep 1 1024K 0K 808756K 1 0 0 inodedep 1 8192K 0K 808756K 1 0 0 newblk 1 1K 0K 808756K 1 0 0 p1003.1b 1 1K 0K 808756K 1 0 0 atexit 2 1K 0K 808756K 2 0 0 proc-args 36 2K 0K 808756K 19891 0 0 exec-args 11 2860K 0K 808756K 11 0 0 kqueue 18 3K 0K 808756K 2764 0 0 kenv 34 5K 0K 808756K 34 0 0 file desc 41 39K 0K 808756K 17845 0 0 file 80 9K 0K 808756K 192890 0 0 sigio 1 1K 0K 808756K 1 0 0 kld 99 6K 0K 808756K 105 0 0 NFS daemon 1 16K 0K 808756K 1 0 0 NFS hash 1 8192K 0K 808756K 1 0 0 syncache 8 96K 0K 808756K 10 0 0 tcptemp 25 2K 0K 808756K 25 0 0 ipq 250 12K 0K 808756K 250 0 0 in_multi 9 1K 0K 808756K 9 0 0 igmp 1 1K 0K 808756K 1 0 0 module 344 32K 0K 808756K 344 0 0 varsym 252 9K 0K 808756K 264 0 0 routetbl 322 52K 0K 808756K 690 0 0 CAM XPT 345 217K 0K 808756K 1086 0 0 faith 1 2K 0K 808756K 1 0 0 scsi_da 0 0K 0K 808756K 3 0 0 CAM queue 25 5K 0K 808756K 1075 0 0 ISOFS mount 1 8192K 0K 808756K 1 0 0 clone 4 16K 0K 808756K 4 0 0 ifaddr 81 52K 0K 808756K 81 0 0 ether_multi 30 2K 0K 808756K 30 0 0 BPF 11 9K 0K 808756K 11 0 0 MSDOSFS mount 1 8192K 0K 808756K 1 0 0 NULLFS mount 7 4K 0K 808756K 7 0 0 vnodes 2023 822K 0K 134203388K 2023 0 0 vnodeops 18 11K 0K 808756K 18 0 0 nameibufs 19 19K 0K 808756K 19 0 0 mount 12 11K 0K 808756K 15 0 0 cluster_save 0 0K 0K 808756K 1019 0 0 vfscache 10082 8963K 0K 808756K 10174 0 0 BIO buffer 2 3K 0K 808756K 33 0 0 unpcb 10 3K 0K 808756K 48 0 0 socket 17 12K 0K 808756K 159 0 0 soname 2 1K 0K 808756K 202 0 0 pcb 51 109K 0K 808756K 219 0 0 tag 0 0K 0K 808756K 60 0 0 mbuf 5454 2727K 0K 808756K 5454 0 0 mbufcl 4887 9774K 0K 808756K 4887 0 0 mclmeta 4887 77K 0K 808756K 4887 0 0 acd_driver 1 4K 0K 808756K 1 0 0 ptys 258 129K 0K 808756K 258 0 0 ata_cam 4 3K 0K 808756K 32 0 0 ttys 879 113K 0K 808756K 2437 0 0 shm 1 40K 0K 808756K 1 0 0 ata_dma 4 1K 0K 808756K 4 0 0 sem 1 144K 0K 808756K 1 0 0 ata_generic 23 7K 0K 808756K 23 0 0 msg 4 27K 0K 808756K 4 0 0 rman 124 13K 0K 808756K 483 0 0 pipe 158 37K 0K 808756K 11092 0 0 ioctlops 0 0K 0K 808756K 29 0 0 taskqueue 20 2K 0K 808756K 20 0 0 sbuf 0 0K 0K 808756K 20 0 0 SWAP 2 4373K 0K 808756K 2 0 0 MD disk 2 2K 0K 808756K 2 0 0 kobj 240 540K 0K 808756K 240 0 0 eventhandler 33 2K 0K 808756K 33 0 0 disk 6 1K 0K 808756K 6 0 0 CAM SIM 7 2K 0K 808756K 7 0 0 bus 1450 132K 0K 808756K 13828 0 0 CAM periph 11 2K 0K 808756K 109 0 0 scsi_cd 0 0K 0K 808756K 2 0 0 callout 8 8192K 0K 808756K 8 0 0 sysctl 0 0K 0K 808756K 349 0 0 sysctloid 2259 81K 0K 808756K 2259 0 0 tslpque 7 448K 0K 808756K 7 0 0 dsched 1457 452K 0K 808756K 1591 0 0 Memory Totals: In Use Free Requests 84193K 0K 1115595 ------------------------------------------------------------------------ vmstat -z ITEM SIZE LIMIT USED FREE REQUESTS SWAPMETA: 000192, 03050402, 000000, 000021, 00000001 PV ENTRY: 000080, 34876160, 150197, 2078014, 00779875 MAP ENTRY: 000104, 00000000, 000535, 001513, 00001368 MAP: 000264, 00000000, 000000, 000010, 00000000 VM OBJECT: 000280, 00000000, 000338, 000212, 00008113 ------------------------------------------------------------------------ vmstat -i interrupt total rate sio1 0 0 sio0 0 0 acpi0 0 0 ohci0/ohci1/atapci1 50 0 ehci0 9 0 ohci2/ohci3 0 0 ehci1 0 0 atapci0 0 0 mpt0 43974 33 bce0 combo 131 0 bce0 rx8 5 0 bce1 combo 0 0 bce1 rx8 0 0 swi_siopoll 0 0 swi_vm 0 0 bce0 rxtx1 1 0 bce1 rxtx1 0 0 bce0 rxtx2 0 0 bce1 rxtx2 0 0 bce0 rxtx3 0 0 bce1 rxtx3 0 0 swi_cambio 87992 66 bce0 rxtx4 10 0 bce1 rxtx4 0 0 bce0 rxtx5 0 0 bce1 rxtx5 0 0 swi_mp_taskq/swi_taskq 66 0 bce0 rxtx6 1 0 bce1 rxtx6 0 0 bce0 rxtx7 0 0 bce1 rxtx7 0 0 Total 132239 99 ------------------------------------------------------------------------ pstat -T 80/261184 files 0M/16383M swap space ------------------------------------------------------------------------ pstat -s Device 512-blocks Used Avail Capacity Type /dev/da1s1b 33554176 40 33554136 0% Interleaved ------------------------------------------------------------------------ ipcs -a Message Queues: T ID KEY MODE OWNER GROUP CREATOR CGROUP CBYTES QNUM QBYTES LSPID LRPID STIME RTIME CTIME Shared Memory: T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME Semaphores: T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME ------------------------------------------------------------------------ ipcs -T msginfo: msgmax: 16384 (max characters in a message) msgmni: 40 (# of message queues) msgmnb: 2048 (max characters in a message queue) msgtql: 40 (max # of messages in system) msgssz: 8 (size of a message segment) msgseg: 2048 (# of message segments in system) shminfo: shmmax: 5521793024 (max shared memory segment size) shmmin: 1 (min shared memory segment size) shmmni: 512 (max number of shared memory identifiers) shmseg: 1024 (max shared memory segments per process) shmall: 1348094 (max amount of shared memory in pages) seminfo: semmap: 128 (# of entries in semaphore map) semmni: 1024 (# of semaphore identifiers) semmns: 32767 (# of semaphores in system) semmnu: 1024 (# of undo structures in system) semmsl: 32767 (max # of semaphores per id) semopm: 100 (max # of operations per semop call) semume: 25 (max # of undo entries per process) semusz: 232 (size in bytes of undo structure) semvmx: 32767 (semaphore maximum value) semaem: 16384 (adjust on exit max value) ------------------------------------------------------------------------ nfsstat Client Info: Rpc Counts: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit GLease Vacate Evict 0 0 0 0 0 0 0 0 Rpc Info: TimedOut Invalid X Replies Retries Requests 0 0 0 0 0 Cache Info: Attr Hits Misses Lkup Hits Misses BioR Hits Misses BioW Hits Misses 0 0 0 0 0 0 0 0 BioRLHits Misses BioD Hits Misses DirE Hits Misses 0 0 0 0 0 0 Server Info: Getattr Setattr Lookup Readlink Read Write Create Remove 0 0 0 0 0 0 0 0 Rename Link Symlink Mkdir Rmdir Readdir RdirPlus Access 0 0 0 0 0 0 0 0 Mknod Fsstat Fsinfo PathConf Commit GLease Vacate Evict 0 0 0 0 0 0 0 0 Server Ret-Failed 0 Server Faults 0 Server Cache Stats: Inprog Idem Non-idem Misses 0 0 0 0 Server Lease Stats: Leases PeakL GLeases 0 0 0 Server Write Gathering: WriteOps WriteRPC Opsaved 0 0 0 ------------------------------------------------------------------------ netstat -s tcp: 0 packets sent 0 data packets (0 bytes) 0 data packets (0 bytes) retransmitted 0 data packets (0 bytes) retransmitted by SACK 0 data packets (0 bytes) sent by SACK recovery 0 SACK rescue packets sent (of 0 attempts) 0 Fast Retransmits (0 early) 0 packets sent by Limited Transmit 0 spurious RTO retransmits (0 Eifel-responses) 0 spurious Fast Retransmits (0 early) 0 Eifel-detected spurious retransmits (0 non-RTT) 0 RTT-detected spurious retransmits 0 resends initiated by MTU discovery 0 SACK options sent 0 D-SACK options sent 0 ack-only packets (0 delayed) 0 URG only packets 0 window probe packets 0 window update packets 0 control packets 0 packets received 0 acks (for 0 bytes) 0 duplicate acks 0 acks for unsent data 0 packets (0 bytes) received in-sequence 0 completely duplicate packets (0 bytes) 0 old duplicate packets (0 packets accepted) 0 packets with some dup. data (0 bytes duped) 0 out-of-order packets (0 bytes) 0 packets (0 bytes) of data after window 0 window probes 0 window update packets 0 packets received after close 0 discarded for bad checksums 0 discarded for bad header offset fields 0 discarded because packet too short 0 bad SACK options 0 other side reneged 0 connection requests 0 connection accepts 0 bad connection attempts 0 listen queue overflows 0 connections established (including accepts) 0 connections closed (including 0 drops) 0 connections updated cached RTT on close 0 connections updated cached RTT variance on close 0 connections updated cached ssthresh on close 0 embryonic connections dropped 0 segments updated rtt (of 0 attempts) 0 retransmit timeouts 0 connections dropped by rexmit timeout 0 persist timeouts 0 connections dropped by persist timeout 0 keepalive timeouts 0 keepalive probes sent 0 connections dropped by keepalive 0 correct ACK header predictions 0 correct data packet header predictions 0 send idles 0 syncache entries added 0 retransmitted 0 dupsyn 0 dropped 0 completed 0 bucket overflow 0 cache overflow 0 reset 0 stale 0 aborted 0 badack 0 unreach 0 zone failures 0 cookies sent 0 cookies received 0 SACK scoreboard updates 0 overflows 0 failures 0 records reused 0 records fast allocated udp: 23 datagrams received 0 with incomplete header 0 with bad data length field 0 with bad checksum 0 with no checksum 1 dropped due to no socket 22 broadcast/multicast datagrams dropped due to no socket 0 dropped due to full socket buffers 0 not for hashed pcb 0 delivered 0 datagrams output ip: 25 total packets received 0 bad header checksums 0 with size smaller than minimum 0 with data size < data length 0 with ip length > max ip packet size 0 with header length < data size 0 with data length < header length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 packets reassembled ok 23 packets for this host 1 packet for unknown/unsupported protocol 0 packets forwarded (0 packets fast forwarded) 1 packet not forwardable 0 packets received for unknown multicast group 0 redirects sent 1 packet sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 tunneling packets that can't find gif 0 datagrams with bad address in header icmp: 1 call to icmp_error 0 errors not generated 'cuz old message was icmp Output histogram: destination unreachable: 1 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length 0 multicast echo requests ignored 0 multicast timestamp requests ignored 0 message responses generated 0 invalid return addresses 0 no return routes ICMP address mask responses are disabled igmp: 1 message received 0 messages received with too few bytes 0 messages received with bad checksum 1 membership query received 0 membership queries received with invalid field(s) 0 membership reports received 0 membership reports received with invalid field(s) 0 membership reports received for groups to which we belong 0 membership reports sent carp: 0 packets received (IPv4) 0 packets received (IPv6) 0 packets discarded for wrong TTL 0 packets shorter than header 0 discarded for bad checksums 0 discarded packets with a bad version 0 discarded because packet too short 0 discarded for bad authentication 0 discarded for bad vhid 0 discarded because of a bad address list 0 packets sent (IPv4) 0 packets sent (IPv6) 0 send failed due to mbuf memory error ip6: 0 total packets received 0 with size smaller than minimum 0 with data size < data length 0 with bad options 0 with incorrect version number 0 fragments received 0 fragments dropped (dup or out of space) 0 fragments dropped after timeout 0 fragments that exceeded limit 0 packets reassembled ok 0 packets for this host 0 packets forwarded 0 packets not forwardable 0 redirects sent 6 packets sent from this host 0 packets sent with fabricated ip header 0 output packets dropped due to no bufs, etc. 0 output packets discarded due to no route 0 output datagrams fragmented 0 fragments created 0 datagrams that can't be fragmented 0 packets that violated scope rules 0 multicast packets which we don't join Mbuf statistics: 0 one mbuf 0 one ext mbuf 0 two or more ext mbuf 0 packets whose headers are not continuous 0 tunneling packets that can't find gif 0 packets discarded due to too may headers 0 failures of source address selection 0 forward cache hit 0 forward cache miss icmp6: 0 calls to icmp_error 0 errors not generated because old message was icmp error or so 0 errors not generated because rate limitation Output histogram: multicast listener report: 6 0 messages with bad code fields 0 messages < minimum length 0 bad checksums 0 messages with bad length Histogram of error messages to be generated: 0 no route 0 administratively prohibited 0 beyond scope 0 address unreachable 0 port unreachable 0 packet too big 0 time exceed transit 0 time exceed reassembly 0 erroneous header field 0 unrecognized next header 0 unrecognized option 0 redirect 0 unknown 0 message responses generated 0 messages with too many ND options 0 messages with bad ND options 0 bad neighbor solicitation messages 0 bad neighbor advertisement messages 0 bad router solicitation messages 0 bad router advertisement messages 0 bad redirect messages 0 path MTU changes rip6: 0 messages received 0 checksum calcurations on inbound 0 messages with bad checksum 0 messages dropped due to no socket 0 multicast messages dropped due to no socket 0 messages dropped due to full socket buffers 0 delivered 0 datagrams output ------------------------------------------------------------------------ netstat -m 9/294816 mbufs in use (current/max): 4593/16816 mbuf clusters in use (current/max) 0/8408 mbuf jumbo clusters in use (current/max) 9 mbufs and mbuf clusters allocated to data 633 mbufs and mbuf clusters allocated to oob data 9 mbufs and mbuf clusters allocated to ancillary data 18446744073709551584 mbufs and mbuf clusters allocated to packet headers 18446744073709551582 mbufs and mbuf clusters allocated to socket names and addresses 18446744073709551604 mbufs and mbuf clusters allocated to 9190 Kbytes allocated to network (4% of mb_map in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines ------------------------------------------------------------------------ netstat -id Name Mtu Network Address Ipkts Ierrs Opkts Oerrs Coll Drop bce0 1500 60:eb:69:ba:59:4f 3649 0 1865 0 0 0 bce0 1500 fe80:1::62e fe80:1::62eb:69ff 0 - 4 - - - bce0 1500 172.17.1/24 172.17.1.207 2823 - 1845 - - - bce1* 1500 60:eb:69:ba:59:50 0 0 0 0 0 0 faith0* 1500 0 0 0 0 0 0 lo0 16384 0 0 0 0 0 0 lo0 16384 your-net localhost 0 - 0 - - - lo0 16384 localhost ::1 0 - 0 - - - lo0 16384 fe80:4::1 fe80:4::1 0 - 0 - - - ppp0* 1500 0 0 0 0 0 0 sl0* 552 0 0 0 0 0 0 ------------------------------------------------------------------------ netstat -anr Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 172.17.1.1 UGSc 1 0 bce0 127.0.0.1 127.0.0.1 UH 0 0 lo0 172.17.1/24 link#1 UC 2 0 bce0 322 172.17.1.1 00:1e:4a:24:0e:17 UHLW 2 0 bce0 2012 172.17.1.5 00:12:3f:73:96:d3 UHLW 0 32 bce0 1522 172.31.255.11 172.17.1.1 UGHW3 0 0 bce0 4259 Internet6: Destination Gateway Flags Netif Expire ::1 ::1 UH lo0 fe80::%bce0/64 link#1 UC bce0 fe80::62eb:69ff:feba:594f%bce0 60:eb:69:ba:59:4f UHL lo0 fe80::%lo0/64 fe80::1%lo0 Uc lo0 fe80::1%lo0 link#4 UHL lo0 ff01::/32 ::1 U lo0 ff02::%bce0/32 link#1 UC bce0 ff02::%lo0/32 ::1 UC lo0 ------------------------------------------------------------------------ netstat -anA Active UNIX domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr ffffffe15e85fae0 dgram 0 0 ffffffe15d0d7320 0 0 0 /var/run/logpriv ffffffe15e85f940 dgram 0 0 ffffffe15d0d7180 0 0 0 /var/run/log ------------------------------------------------------------------------ netstat -aL ------------------------------------------------------------------------ fstat fstat: _allproc: no such symbol ------------------------------------------------------------------------ dmesg panic: assertion "tdio->qlength == 0" failed in dsched_thread_io_destroy at /build/home/justin/src/sys/kern/kern_dsched.c:842 cpuid = 3 Trace beginning at frame 0xffffffe12e1771c0 panic() at panic+0x223 0xffffffff80561c0c panic() at panic+0x223 0xffffffff80561c0c dsched_thread_io_destroy() at dsched_thread_io_destroy+0x4a 0xffffffff805795f6 dsched_thread_io_unref_destroy() at dsched_thread_io_unref_destroy+0x63 0xffffffff8057994e dsched_thread_ctx_unref() at dsched_thread_ctx_unref+0x152 0xffffffff80579476 dsched_exit_buf() at dsched_exit_buf+0x28 0xffffffff8057aad8 bqrelse() at bqrelse+0x406 0xffffffff805c2ce3 bpdone() at bpdone+0x3f3 0xffffffff805c30e8 biodone() at biodone+0x1d7 0xffffffff805c3350 hammer_io_direct_write_complete() at hammer_io_direct_write_complete+0xd0 0xffffffff80764682 biodone() at biodone+0x1bb 0xffffffff805c3334 fq_completed() at fq_completed+0x1c1 0xffffffff8080808d biodone() at biodone+0x1bb 0xffffffff805c3334 dadone() at dadone+0x32b 0xffffffff8029dd47 camisr_runqueue() at camisr_runqueue+0x3ff 0xffffffff80292db8 swi_cambio() at swi_cambio+0x126 0xffffffff80292fab ithread_handler() at ithread_handler+0x1e5 0xffffffff80535709 Debugger("panic") CPU3 stopping CPUs: 0x000000f7 stopped panic: from debugger cpuid = 3 boot() called on cpu#3 Uptime: 15m35s Physical memory: 8154 MB Dumping 1868 MB: 1853 1837 1821 1805 1789 1773 1757 1741 1725 1709 1693 1677 1661 1645 1629 1613 1597 1581 1565 1549 1533 1517 1501 1485 1469 1453 1437 1421 1405 1389 1373 1357 1341 1325 1309 1293 1277 1261 1245 1229 1213 1197 1181 1165 1149 1133 1117 1101 1085 1069 1053 1037 1021 1005 989 973 957 941 925 909 893 877 861 845 829 813 797 781 765 749 733 717 701 685 669 653 637 621 605 589 573 557 541 525 509 493 477 461 445 429 413 397 381 365 349 333 317 301 285 269 253 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13