DragonFlyBSD

:hey,
:
:I just booted with hw.physmem=64m and got a panic when trying to load a module:
:
:panic: kld: malloc limit exceeded
:
:#3 0xc0182129 in kmalloc (size=78, type=0xc02f5600, flags=2)
: at /usr/build/src/sys/kern/kern_slaballoc.c:445
:#4 0xc01678b4 in linker_make_file (pathname=0xc641b000 "./nvidia.ko", priv=0xc63ea028,
: ops=0xc02f5bc8) at /usr/build/src/sys/kern/kern_linker.c:369
:...
:
:the problem seems to be that M_LINKER already used 10% of all memory (allegedly). In this case kmalloc() simply panics if passing M_WAITOK without M_NULLOK. This is quite unfortunate. Shouldn't we try to stay alive and print a warning and block, hoping that the problem will resolve itself?
:
:cheers
: simon

That's an administrative limit, it sounds like the solution is to simply
    assign a reasonable minimum to it.

Well, what's the malloc limit when you have 64MB of ram?  From gdb
    do 'print M_LINKER'.  And... is the amount of memory it thinks it is
    using roughly similar to the amount of memory the loaded modules ought
    to be taking up?

-Matt

The limit is ~6MB, the linker consumes 8MB. I just realized that we
have in struct malloc_type:

__uint16_t ks_limblocks; /* number of times blocked for hitting 
limit */

But we're never using this.

cheers
simon

:The limit is ~6MB, the linker consumes 8MB. I just realized that we
:have in struct malloc_type:
:
: __uint16_t ks_limblocks; /* number of times blocked for hitting
:limit */
:
:But we're never using this.
:
:cheers
: simon

It'l probably never be used.  There is no callback facility for 
    recovering kmalloc'd memory.  And, particularly for the linker,
    there wouldn't be anything to free anyway so it would just
    deadlock.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>

Well, yea, but I'd rather have a blocking kldload than a panic'ed
system. There are probably more such places. Actually I think this
limit business is stupid. Rather than panicing, we could print a
warning and go on. The worst thing could be that we'll panic lateron
because we don't have free memory anymore.

cheers
simon

:Well, yea, but I'd rather have a blocking kldload than a panic'ed
:system. There are probably more such places. Actually I think this
:limit business is stupid. Rather than panicing, we could print a
:warning and go on. The worst thing could be that we'll panic lateron
:because we don't have free memory anymore.
:
:cheers
: simon

Well, there aren't too many places where M_LINKER is used, you could
    adjust the code to pass M_NULLOK and deal with NULL returns.

We definitely do not want to block, since it will wind up blocking
    forever.

Matthew Dillon 
                    <dillon@backplane.com>

On Jan 11, 2008 9:12 AM, Simon 'corecode' Schubert
<corecode@fs.ei.tum.de> wrote:

Matthew Dillon wrote:

It'l probably never be used. There is no callback facility for
recovering kmalloc'd memory. And, particularly for the linker,
there wouldn't be anything to free anyway so it would just
deadlock.

Well, yea, but I'd rather have a blocking kldload than a panic'ed
system. There are probably more such places. Actually I think this
limit business is stupid. Rather than panicing, we could print a
warning and go on. The worst thing could be that we'll panic lateron
because we don't have free memory anymore.

I don't think it's the problem of our M_WAITOK implementation. You
could change the caller to use other kmalloc flag so that the system
will not panic; how to properly handle kmalloc failure in this case
may be difficult.

Best Regards,
sephe

I realize this. However, we should avoid panicing if possible. So why
not print a warning ("allocated more than the limit") and continue? If
nothing happens, great! If we run out of memory, we still can panic later.

cheers
simon

On Jan 11, 2008 9:34 AM, Simon 'corecode' Schubert
<corecode@fs.ei.tum.de> wrote:

Sepherosa Ziehau wrote:

I don't think it's the problem of our M_WAITOK implementation. You
could change the caller to use other kmalloc flag so that the system
will not panic; how to properly handle kmalloc failure in this case
may be difficult.

I realize this. However, we should avoid panicing if possible. So why
not print a warning ("allocated more than the limit") and continue? If

What do you mean by "continue" here?

Best Regards,
sephe

Just allocate some memory, even if you're over the limit. It's not that
we ran out of memory, it's just that we went over the (artificial) limit.

cheers
simon

:Just allocate some memory, even if you're over the limit. It's not that
:we ran out of memory, it's just that we went over the (artificial) limit.
:
:cheers
: simon

That seems reasonable insofar as M_LINKER goes.  How about this: Add a
    new malloc flag which tells it to ignore the administrative limit.  
    Instead of panicing malloc() would generate a rate-limited warning to
    the console but continue with the allocation.

In most cases hitting the administrative resource limit should be fatal,
    but clearly not in the M_LINKER case.  A flag would be appropriate.

-Matt
                    Matthew Dillon 
                    <dillon@backplane.com>