DragonFlyBSD

I'll summarize what I've learned so far.
Boehm GC is a mark-sweep garbage collector.
The crashes and the hangs happen because the GC garbage collects its own
internal data structures.
How could this happen?! Easy.
The mark procedure starts marking objects as reachable beginning from a
number of root regions, which include, but are not limited to, the
program's global data area and the global data areas of dynamically
loaded libraries. Unfortunately, on DragonFly, the latter part doesn't
work as intended. So, the garbage collector, which itself is a
dynamically loaded library, doesn't mark its own data structures, and
the sweeper frees the memory they occupy.
In the source code of the GC, functions of interest are
GC_add_roots_inner() in mark_rts.c and GC_FirstDLOpenedLinkMap(),
starting at line 493, in dyn_load.c.
GC_add_roots_inner() should get called once for the main executable and
for each dynamic library loaded.
In GC_FirstDLOpenedLinkMap() we start from _DYNAMIC and in a loop try to
find a Elf_Dyn entry with tag DT_DEBUG.
On DragonFly, the main executable has such an entry, however when
GC_FirstDLOpenedLinkMap() iterates over the list it seems to iterate
over the list of entries of libgc - a dynamic library which lacks
DT_DEBUG entries in its .dynamic section. The DT_DEBUG entry we are
looking for isn't found.
On FreeBSD, when we iterate over the list, we get a list of tags that is
a prefix of the tags of the main executable.
The tags on the main executable:
1 1 1 15 12 13 4 5 6 10 11 21 20 23 17 18 19 0x6ffffffe 0x6fffffff
0x6ffffff0
What we get:
1 1 1 15 12 13 4 5 6 10 11 21

I'm leaving this problem report for now, to let my head recover.
If you've got any ideas, please let me know.

Here's some less important stuff:
And now, to see how well FreeBSD fares. Building the Boehm GC on FreeBSD
7.1 and running 'gmake check' produces:
Boehm GC version 6.8 built from ports:
FAIL: gctest
PASS: test_cpp ==================================
1 of 2 tests failed
Please report to Hans.Boehm@hp.com ==================================

Boehm GC version 6.8 built manually:
PASS: gctest ==================
All 1 tests passed ==================

Boehm GC version 7.1 built manually:
PASS: gctest
PASS: leaktest
PASS: middletest
PASS: smashtest
PASS: hugetest
PASS: threadleaktest ==================
All 6 tests passed ==================

Oh, it's obvious why on FreeBSD we get only a prefix of the list. The
search simply stops when we find a DT_DEBUG entry (code 21).
So, all the focus should be on what _DYNAMIC should point to.

Jordan Gordeev wrote:

Jordan Gordeev wrote:

On FreeBSD, when we iterate over the list, we get a list of tags that
is a prefix of the tags of the main executable.
The tags on the main executable:
1 1 1 15 12 13 4 5 6 10 11 21 20 23 17 18 19 0x6ffffffe 0x6fffffff
0x6ffffff0
What we get:
1 1 1 15 12 13 4 5 6 10 11 21

Oh, it's obvious why on FreeBSD we get only a prefix of the list. The
search simply stops when we find a DT_DEBUG entry (code 21).
So, all the focus should be on what _DYNAMIC should point to.

kudos, good detective work!

There's a difference between DragonFly's ld and FreeBSD's ld.
Let's take the following program:
extern void *vodka;

void *
f(void) {
return vodka;
}

We compile it into a shared library:
%cc -c -o vodka.o vodka.c
%ld -Bshareable -o libvodka.so vodka.o

Then we do:
%objdump -t libvodka.so

Doing the above steps on DragonFly produces:

libvodka.so: file format elf32-i386

SYMBOL TABLE:
00000094 l d .hash 00000000 .hash
000000c4 l d .dynsym 00000000 .dynsym
00000134 l d .dynstr 00000000 .dynstr
00000158 l d .rel.dyn 00000000 .rel.dyn
00000160 l d .text 00000000 .text
0000116c l d .dynamic 00000000 .dynamic
000011e4 l d .got.plt 00000000 .got.plt
00000000 l d .comment 00000000 .comment
00000000 l d ABS 00000000 .shstrtab
00000000 l d ABS 00000000 .symtab
00000000 l d ABS 00000000 .strtab
00000000 l df ABS 00000000 vodka.c
0000116c l O ABS 00000000 .hidden DYNAMIC
000011e4 l O ABS 00000000 .hidden _GLOBAL_OFFSET_TABLE
00000160 g F .text 0000000a f
00000000 UND 00000000 vodka
000011f0 g ABS 00000000 __bss_start
000011f0 g ABS 00000000 _edata
000011f0 g ABS 00000000 _end

Doing the steps on FreeBSD produces:

libvodka.so: file format elf32-i386-freebsd

SYMBOL TABLE:
00000094 l d .hash 00000000
00000128 l d .dynsym 00000000
00000248 l d .dynstr 00000000
00000288 l d .rel.dyn 00000000
00000290 l d .text 00000000
0000129c l d .data 00000000
0000129c l d .dynamic 00000000
00001314 l d .got 00000000
00001320 l d .bss 00000000
00000000 l d .comment 00000000
00000000 l d ABS 00000000
00000000 l d ABS 00000000
00000000 l d ABS 00000000
00000000 l df ABS 00000000 vodka.c
0000129c g O ABS 00000000 DYNAMIC
00000290 g F .text 0000000a f
00000000 UND 00000000 vodka
00001320 g ABS 00000000 __bss_start
00001320 g ABS 00000000 _edata
00001314 g O ABS 00000000 _GLOBAL_OFFSET_TABLE
00001320 g ABS 00000000 _end

Please, note that '_DYNAMIC' is a local symbol on DragonFly and a global
one on FreeBSD.

Jordan Gordeev wrote:

Please, note that '_DYNAMIC' is a local symbol on DragonFly and a global
one on FreeBSD.

I believe that's because FreeBSD uses an older linker. With more recent
linkers, it is the same as in DragonFly (I checked Linux x86_64).

I also checked, and &_DYNAMIC is different in a shared lib and in the
main program, on both DragonFly and Linux, so that also shouldn't be the
underlying issue.

cheers
simo

Did anyone else ever look into this?

This hacky patch fixes the non-threaded test on i386 running recent current (still gcc44)
- basically, it looks like the #ifdef's haven't kept up with the dynamic linker / elf toolkit,
so alot of the byte / symbol mangling isn't needed and GC can use fancier builtin API's to
programatically access the stuff.

I'll clean this up later and take a crack at the threaded version.

Cheers,

- Chris

1. diff urw dyn_load.c.orig dyn_load.c
   -- dyn_load.c.orig 2012-08-09 20:25:13.000000000 0000
   ++ dyn_load.c 2012-10-11 17:53:50.000000000 +0000
   @ -83,6 +83,12 @ # define ELFSIZE ARCH_ELFSIZE
   #endif

#if defined(DragonFly)
# include <elf.h>
# include <dlfcn.h>
# include <link.h>
#endif

#if defined(SCO_ELF) || defined(DGUX) || defined(HURD) \ || (defined(ELF) && (defined(LINUX) || defined(FREEBSD) \ || defined(NETBSD) || defined(OPENBSD)))
@ -398,7 +404,7 @ # pragma weak dl_iterate_phdr
#endif

-#if (defined(FREEBSD) && FreeBSD >= 7)
+#if (defined(FREEBSD) && FreeBSD >= 7) && defined(DragonFly)
/* On the FreeBSD system, any target system at major version 7 shall /
/ have dl_iterate_phdr; therefore, we need not make it weak as above. */ # define HAVE_DL_ITERATE_PHDR
@ -646,7 +652,8 @
return(0);
}
if( cachedResult == 0 ) {
-# if defined(NETBSD) && defined(RTLD_DI_LINKMAP)
/* HACK */
# if defined(DragonFly) && 1
struct link_map *lm = NULL;
if (!dlinfo(RTLD_SELF, RTLD_DI_LINKMAP, &lm))
cachedResult = lm;

Also: this was on gc-7.2d.tar.gz fetched from upstream today rather than 7.1 as
outlined in original report

well.. unfortunately threading doesn't appear to be a 'hack it and it works' kind
of an issue (as mostly expected).

ecl builds 99% of the way through to the link stage, but based on a truss, issues a GC during its link command, and due to some ifdeffery, there is some kind of threaded 'gc stop/start world'
signal mismatching going on. will need to dig deeper on this .. but not today.

this might be a different case for non-threaded operation - but my 'hack the files' level of understanding on this one is too low to say.