https://bugs.dragonflybsd.org/https://bugs.dragonflybsd.org/favicon.ico?16293952082009-02-10T00:01:29ZDragonFlyBSD bugtrackerDragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61252009-02-10T00:01:29Zdillon
<ul></ul><p>:Hi,<br />:<br />:I got this error yesterday:<br />:<br />: hammer_ip_add_bulk reservation failed<br />:<br />:The message appeared every 1 second or so and pretty much slowed down<br />:the whole system (it became unusable). <br />:<br />:The system is 2.1.1-DEVELOPMENT as of 29th January from the snapshot<br />:build.<br />:<br />:Regards,<br />:<br />: Michael</p>
<pre><code>If it happens again do a 'sysctl vfs.hammer' and a 'df' and save the<br /> output, and get a kernel core.</code></pre>
<pre><code>How full was the HAMMER partition?</code></pre>
<pre><code>-Matt<br /> Matthew Dillon <br /> &lt;<a class="email" href="mailto:dillon@backplane.com">dillon@backplane.com</a>&gt;</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61262009-02-10T00:14:27Zdillon
<ul></ul><p>:91% full.<br />:<br />:Regards,<br />:<br />: Michael</p>
<pre><code>That's pretty full. How large a partition is it?</code></pre>
<pre><code>-Matt<br /> Matthew Dillon <br /> &lt;<a class="email" href="mailto:dillon@backplane.com">dillon@backplane.com</a>&gt;</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61272009-02-10T00:18:30Zmneumann
<ul></ul><p>Am Mon, 9 Feb 2009 08:00:45 -0800 (PST)<br />schrieb Matthew Dillon <<a class="email" href="mailto:dillon@apollo.backplane.com">dillon@apollo.backplane.com</a>>:</p>
<p>91% full.</p>
<p>Regards,</p>
<pre><code>Michael</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61282009-02-10T00:36:30Zdillon
<ul></ul><p>:Am Mon, 9 Feb 2009 08:08:18 -0800 (PST)<br />:schrieb Matthew Dillon <<a class="email" href="mailto:dillon@apollo.backplane.com">dillon@apollo.backplane.com</a>>:<br />:<br />:> :91% full.<br />:> :<br />:<br />:60 GB.</p>
<pre><code>Ok, it shouldn't have generated that message with so much free<br /> space. The only thing I can think of is that there is a leak<br /> in the hammer_reservation structure. sysctl vfs.hammer.count_reservations<br /> tells you how many structures are active.. it should usually be a low<br /> number. I look over the code.</code></pre>
<pre><code>-Matt<br /> Matthew Dillon <br /> &lt;<a class="email" href="mailto:dillon@backplane.com">dillon@backplane.com</a>&gt;</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61292009-02-10T01:05:58Zdillon
<ul></ul><p>Whoop, I think I found it. You should be seeing a build-up<br /> up reservations (vfs.hammer.count_reservations). There is a ref<br /> count leak in the hammer_buffer code in the buffer invalidation path.<br /> This is preventing reservations from being disposed of.</p>
<pre><code>I will commit a fix today.</code></pre>
<pre><code>-Matt</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61302009-02-10T02:21:33Zmneumann
<ul></ul><p>Am Mon, 9 Feb 2009 08:58:53 -0800 (PST)<br />schrieb Matthew Dillon <<a class="email" href="mailto:dillon@apollo.backplane.com">dillon@apollo.backplane.com</a>>:</p>
<p>Yep, a "hammer reblock" triggered the error again.</p>
<ol>
<li>sysctl vfs.hammer.count_reservations<br /> vfs.hammer.count_reservations: 2386</li>
<li>df<br /> "64 %"</li>
</ol>
<p>But as you said you already found the bug, I don't generate a core dump<br />right now.</p>
<p>Thanks!</p>
<p>Regards,</p>
<pre><code>Michael</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61322009-02-10T02:50:28Zdillon
<ul></ul><p>:Yep, a "hammer reblock" triggered the error again.<br />:<br />:# sysctl vfs.hammer.count_reservations<br />: vfs.hammer.count_reservations: 2386<br />:# df<br />: "64 %" <br />: <br />:But as you said you already found the bug, I don't generate a core dump<br />:right now.<br />:<br />:Thanks!<br />:<br />:Regards,<br />:<br />: Michael</p>
<pre><code>Ok, I committed a fix, please test.</code></pre>
<pre><code>-Matt<br /> Matthew Dillon <br /> &lt;<a class="email" href="mailto:dillon@backplane.com">dillon@backplane.com</a>&gt;</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61332009-02-10T02:58:29Zdillon
<ul></ul><p>::# sysctl vfs.hammer.count_reservations<br />:: vfs.hammer.count_reservations: 2386<br />::# df<br />:: "64 %" <br />:: Michael<br />:<br />: Ok, I committed a fix, please test.<br />:</p>
<pre><code>Oh, note that a large reservation count is only a bug if it nevers<br /> goes back down. e.g. idle the system, sync a few times... if the<br /> reservation count doesn't drop to near zero there is a problem.</code></pre>
<pre><code>-Matt<br /> Matthew Dillon <br /> &lt;<a class="email" href="mailto:dillon@backplane.com">dillon@backplane.com</a>&gt;</code></pre> DragonFlyBSD - Bug #1274: hammer_ip_add_bulk reservation failedhttps://bugs.dragonflybsd.org/issues/1274?journal_id=61472009-02-10T21:32:29Zcorecode
<ul></ul><p>committed fix in 4983f1f6fa80b8035dd5fac7d32c4e470fb5766a</p>