Bug #2566

No space available for static Thread Local Storage

Added by eric.j.christeson 11 months ago. Updated 10 months ago.

Status:ClosedStart date:06/06/2013
Priority:NormalDue date:
Assignee:-% Done:

0%

Category:-
Target version:-

Description

I noticed some of my cron jobs not running and upon investigation I found this in /var/log/messages:

cron: in try_dlopen(): /usr/lib/security/pam_opie.so.2: /usr/lib/libc.so.7: No space available for static Thread Local Storage

su seems to have a similar problem:

su: in try_dlopen(): /usr/lib/security/pam_opie.so.2: /usr/lib/libc.so.7: No space available for static Thread Local Storage

As a work-around I edited /etc/pam.d/system and commented out the first two auth lines that include pam_opie.so and pam_opieaccess.so
obviously if you're not using opie, this won't affect much.

I first noticed it after building a quickworld from master on Monday, June 3. The previous known good was Thursday, May 30.

readelf_a_bootstrap_bin_java.txt Magnifier (43.4 KB) ftigeot, 06/09/2013 03:05 AM

History

#1 Updated by swildner 11 months ago

On Thu, 06 Jun 2013 16:32:19 +0200, Eric Christeson via Redmine
<> wrote:

> I first noticed it after building a quickworld from master on Monday,
> June 3. The previous known good was Thursday, May 30.

My feeling (although I have no clear idea about what happened exactly) is
that it is a quickworld issue across the libc version bump. libc in master
is libc.so.8.

I didn't try a quickworld myself but all boxes I upgraded with a full
build do not have this issue.

Sascha

#2 Updated by ftigeot 11 months ago

This issue also occurs when trying to run the openjdk7 bootstrap binary on DragonFly 3.5

The bootstrap has been built on a previous DragonFly release and thus depends on libc.so.7

readelf -a bootstrap/bin/java reports:

Dynamic section at offset 0xc5c8 contains 22 entries:
[...]
0x0000000000000001 (NEEDED) Shared library: [libc.so.7]
[...]

libc.so.7 is the only libc name present in the ELF binary. Nothing unusual so far.

Yet, ldd reports bootstrap/bin/java is linked to both libc.so.7 and libc.so.8:

bootstrap/bin/java:
libz.so.3 => /usr/lib/libz.so.3 (0x800828000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x800a3c000)
libc.so.7 => /usr/lib/libc.so.7 (0x800c53000)
libc.so.8 => /usr/lib/libc.so.8 (0x800f8d000)

#3 Updated by ftigeot 11 months ago

Complete readelf -a output

#4 Updated by ftigeot 11 months ago

$ ldd -a bootstrap/bin/java

/red/@@-1:00007/dports/java/openjdk7/work/bootstrap/bin/java:
libz.so.3 => /usr/lib/libz.so.3 (0x800828000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x800a3c000)
libc.so.7 => /usr/lib/libc.so.7 (0x800c53000)
/usr/lib/libz.so.3:
libc.so.8 => /usr/lib/libc.so.8 (0x800f8d000)
/usr/lib/libpthread.so.0:
libc.so.8 => /usr/lib/libc.so.8 (0x800f8d000)

#5 Updated by ftigeot 10 months ago

This issue is also visible with some binaries only linked to libc.8.

The zsh 3.6:x86:64 package is one of these:

$ cd fo<TAB>
_tags:36: failed to load module `zsh/computil': /usr/lib/libc.so.8: No space available for static Thread Local Storage

zsh/computil.so is a shared library, not linked to multiple versions of shared libraries:

ldd ./computil.so
./computil.so:
libiconv.so.3 => /usr/local/lib/libiconv.so.3 (0x800d68000)
libdl.so.0 => /usr/lib/libdl.so.0 (0x801064000)
libncursesw.so.6 => /usr/lib/libncursesw.so.6 (0x801265000)
libm.so.4 => /usr/lib/libm.so.4 (0x8014db000)
libc.so.8 => /usr/lib/libc.so.8 (0x80081d000)

#6 Updated by marino 10 months ago

It appears to me that nmalloc is ultimately the culprit here.
It sets the thr_mags structure as TLS data. Right now NZONES equals 72 since ZALLOC_ZONE_LIMIT is 16K. That means on x86-64, the thr_mags structure size is 2*8*72 + 8 + 4 = 1164 bytes. Together will a couple of other TLS variables, the TLS section size of libc is 1172 bytes. (For i386 it's "only" 588 bytes)

In comparison, FreeBSD libc TLS section is 17 bytes.

The dmalloc implementation also sets a structure as TLS storage, but it seems to be more modest (depends on size of slab_zone + 42 bytes)

The immediate solution is to increase the value of RTLD_STATIC_TLS_EXTRA from 256 to likely 1280 on x86-64 and 640 on i386.

The mallocs should be reviewed to see if setting these structures to TLS was the right thing to do. In any case, I think we're stuck with increased *TLS_EXTRA space until we don't need to support libc's with such big TLS sections. (FreeBSD *TLS_EXTRA section is 128 bytes). This is per library I believe, so an executable with 20 linked libraries needs to allocate, for example, 1280x20 bytes = 25kb of TLS space during dynamic linking.

#7 Updated by marino 10 months ago

TLS space increased by this commit:
http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/4b2454d14575fe90aff242d1b282429ae99e2fea

Upgrading to latest world will probably fix these TLS storage errors of the 3.4 packages.

#8 Updated by dragonflybsd1 10 months ago

  • Status changed from New to Closed

no feedback, assumed fixed.

Also available in: Atom PDF