Bug #2249: deadlock under high i/o load (e.g. hammer reblock) - DragonFlyBSD - DragonFlyBSD bugtracker

Actions

Copy link

Bug #2249

closed

deadlock under high i/o load (e.g. hammer reblock)

Added by rumcic about 13 years ago. Updated almost 13 years ago.

Status:

Closed

Priority:

Normal

Assignee:

Category:

Target version:

Start date:

Due date:

% Done:

Estimated time:

Description

Almost latest master (last commit from upstream should be
0e84c0db0d0bcd6d681bc30fd16e96c26bb92db2).

Whenever nightly cron runs and a few minutes after hammer cleanup starts doing
it's job, the machine becomes mostly unresponsive (most processes and most ssh
connections stop responding, but strangely pf still continues doing it's job).

Fortunately debugging through serial console still worked, so the dump is
available at leaf:~rumko/crash/deadlock/*.0 .
--
Please do not CC me, since I already receive everything from these MLs.

Regards,
Rumko

Actions

Copy link

Updated by tuxillo about 13 years ago

Hi Rumko,

I see your kern.0 is 7.5K, bad upload or bad dump?

~~rw-r--r-~~ 1 rumko wheel 231K Nov 28 13:51 core.txt.0
~~rw------~~ 1 rumko wheel 465B Nov 28 13:50 info.0
~~rw-r--r-~~ 1 rumko wheel 7.5K Nov 28 13:50 kern.0
~~rw------~~ 1 rumko wheel 407M Nov 28 13:51 vmcore.0

Thanks,
Antonio Huete

Actions

Copy link

Updated by rumcic about 13 years ago

Antonio M. Huete Jimenez via Redmine wrote:

Issue #2249 has been updated by Antonio M. Huete Jimenez.

Hi Rumko,

I see your kern.0 is 7.5K, bad upload or bad dump?

bad dump I'd guess, copied over the kernel from /boot and seems to be ok now

~~rw-r--r-~~ 1 rumko wheel 231K Nov 28 13:51 core.txt.0
~~rw------~~ 1 rumko wheel 465B Nov 28 13:50 info.0
~~rw-r--r-~~ 1 rumko wheel 7.5K Nov 28 13:50 kern.0
~~rw------~~ 1 rumko wheel 407M Nov 28 13:51 vmcore.0

Thanks,
Antonio Huete

<snip>
--
Please do not CC me, since I already receive everything from these MLs.

Regards,
Rumko

Actions

Copy link

Updated by tuxillo about 13 years ago

Rumko,

It's indeed a deadlock. I'm experiencing that during 'hammer cleanup' also but on x86_64. Explanation from Matt:

21:21 <@dillon> the pageout daemon deadlock is because the hammer backend locks hammer inodes and the pageout
daemon frontend can only detect locked vnodes
21:21 <@dillon> so the pageout daemon will happily lock a vnode and then issue the pageout request to the hammer
backend and cause the hammer backend to get stuck on the inode (waiting for new memory)

It doesn't seem to have an easy solution.

Cheers,
Antonio Huete

Actions

Copy link