Project

General

Profile

Bug #3143

assertion "0" failed in hammer2_inode_xop_chain_sync

Added by cbin 4 months ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
07/17/2018
Due date:
% Done:

0%

Estimated time:

Description

This happened on DragonFly 5.2.2

chain 00000000441da00a.01 key=0000000000008641 meth=30 CHECK FAIL (flags=00140002, bref/data 830e0b4c1fbfa512/9f5e0aac2b2788b2)
panic: assertion "0" failed in hammer2_inode_xop_chain_sync at /usr/src/sys/vfs/hammer2/hammer2_inode.c:1775
cpuid = 0
Trace beginning at frame 0xfffff9008a525920
panic() at panic+0x236 0xffffffff805f8666
panic() at panic+0x236 0xffffffff805f8666
hammer2_inode_xop_chain_sync() at hammer2_inode_xop_chain_sync+0x249 0xffffffff808bfde9
hammer2_primary_xops_thread() at hammer2_primary_xops_thread+0x26b 0xffffffff808be0fb
Debugger("panic")

This machine is mostly idle, except for once an hour (at 10 minutes to the hour) a dovecot backup is synced to it and on the hour it creates a snapshot of the pfs with the dovecot backup.

It came back up with no problems and it *seems* to work okay, but now there are hundreds errors like the following appearing:

Jul 17 15:43:14 archivist kernel: CHILD ERROR DURING FLUSH LOCK 0xfffff8009abb9420->0xfffff8009c118ca0
Jul 17 15:43:14 archivist kernel: chain 00000000441da40a.01 key=0000000000008639 meth=30 CHECK FAIL (flags=0114c002, bref/data 80bb95a42ef6c177/4762378c10af65e7)
Jul 17 15:43:14 archivist kernel: CHILD ERROR DURING FLUSH LOCK 0xfffff8009abb9420->0xfffff8009c11b0a0
Jul 17 15:43:14 archivist kernel: chain 00000000441da80a.01 key=000000000000863f meth=30 CHECK FAIL (flags=0114c002, bref/data 6c75050e70b2327a/1e4bf80cd16e453b)
Jul 17 15:43:14 archivist kernel: CHILD ERROR DURING FLUSH LOCK 0xfffff8009abb9420->0xfffff8009c118ca0

This on a vultr VM. The disk is small but there is plenty of space available:

Filesystem Size Used Avail Capacity Mounted on
/dev/mapper/root@DATA 21.6G 4469M 17.2G 20% /
/dev 1024B 1024B 0B 100% /dev
/dev/vbd0s1a 1022M 467M 473M 50% /boot
/build/usr.obj 21.6G 4469M 17.2G 20% /usr/obj
/build/var.crash 21.6G 4469M 17.2G 20% /var/crash
/build/var.cache 21.6G 4469M 17.2G 20% /var/cache
/build/var.spool 21.6G 4469M 17.2G 20% /var/spool
/build/var.log 21.6G 4469M 17.2G 20% /var/log
/build/var.tmp 21.6G 4469M 17.2G 20% /var/tmp
tmpfs 233M 12.0K 233M 0% /tmp
procfs 4096B 4096B 0B 100% /proc
@var.vmail 21.6G 4469M 17.2G 20% /var/vmail

History

#1 Updated by cbin 4 months ago

An automatic bulkfree was run during the daily cleanup, and it failed but after that the constant errors in the console stopped. Errors now only appear while running a bulkfree, which always fails.

This is all during a bulkfree:

Jul 18 19:34:24 archivist kernel: chains 50000 inodes 47897 dirents 70353 bytes 75MB
Jul 18 19:34:25 archivist kernel: chain 00000000441da40a.01 key=0000000000008639 meth=30 CHECK FAIL (flags=00144002, bref/data 80bb95a42ef6c177/4762378c10af65e7)
Jul 18 19:34:25 archivist kernel: chain 00000000441da40a.01 key=0000000000008639 meth=30 CHECK FAIL (flags=00144002, bref/data 80bb95a42ef6c177/4762378c10af65e7)
Jul 18 19:34:25 archivist kernel: chain 00000000441da80a.01 key=000000000000863f meth=30 CHECK FAIL (flags=00144002, bref/data 6c75050e70b2327a/1e4bf80cd16e453b)
Jul 18 19:34:25 archivist kernel: chain 00000000441da80a.01 key=000000000000863f meth=30 CHECK FAIL (flags=00144002, bref/data 6c75050e70b2327a/1e4bf80cd16e453b)
Jul 18 19:34:25 archivist kernel: chain 00000000441da00a.01 key=0000000000008641 meth=30 CHECK FAIL (flags=00144002, bref/data 830e0b4c1fbfa512/9f5e0aac2b2788b2)
Jul 18 19:34:25 archivist kernel: chain 00000000441da00a.01 key=0000000000008641 meth=30 CHECK FAIL (flags=00144002, bref/data 830e0b4c1fbfa512/9f5e0aac2b2788b2)
Jul 18 19:34:34 archivist kernel: chains 100000 inodes 92639 dirents 142454 bytes 160MB
Jul 18 19:34:44 archivist kernel: chains 150000 inodes 139282 dirents 189386 bytes 241MB
Jul 18 19:34:54 archivist kernel: chains 200000 inodes 186751 dirents 201388 bytes 322MB
Jul 18 19:34:57 archivist kernel: chain 00000000441da40a.01 key=0000000000008639 meth=30 CHECK FAIL (flags=00144002, bref/data 80bb95a42ef6c177/4762378c10af65e7)
Jul 18 19:34:57 archivist kernel: chain 00000000441da40a.01 key=0000000000008639 meth=30 CHECK FAIL (flags=00144002, bref/data 80bb95a42ef6c177/4762378c10af65e7)
Jul 18 19:34:57 archivist kernel: chain 00000000441da80a.01 key=000000000000863f meth=30 CHECK FAIL (flags=00144002, bref/data 6c75050e70b2327a/1e4bf80cd16e453b)
Jul 18 19:34:57 archivist kernel: chain 00000000441da80a.01 key=000000000000863f meth=30 CHECK FAIL (flags=00144002, bref/data 6c75050e70b2327a/1e4bf80cd16e453b)
Jul 18 19:34:57 archivist kernel: bulkfree lastdrop 1 0 error=0x0002
Jul 18 19:34:57 archivist kernel: bulkfree pass statistics (0.00% storage processed):
Jul 18 19:34:57 archivist kernel: bulkfree was aborted

And the terminal:

# hammer2 bulkfree /
ioctl: Numerical argument out of domain

Also available in: Atom PDF