DragonFlyBSD

Core file is small, so I leave it here as an attach

Please also attach your sftp binary.

Regards,
Alex Hornung

I've recompiled sftp with -g and without -O options and attached the file. (I
have x86_64 system with DragonFlyBSD 2.8)

Here is some output from gdb:

(gdb) run shamazmazum,tprpg@frs.sourceforge.net
Starting program: /home/vasily/debug/sftp shamazmazum,tprpg@frs.sourceforge.net
shamazmazum,tprpg@frs.sourceforge.net's password:
Connected to frs.sourceforge.net.
sftp> get htdocs

Program received signal SIGILL, Illegal instruction.
0x0000000800570800 in ?? ()
(gdb) bt
#0 0x0000000800570800 in ?? ()
#1 0x0000000800bbf2ce in ?? () from /usr/lib/libc.so.7
#2 0x0000000800bbfafb in ?? () from /usr/lib/libc.so.7
#3 0x0000000800bc0039 in glob () from /usr/lib/libc.so.7
#4 0x000000000040cd8d in remote_glob (conn=0x8005700c0,
pattern=0x800570800 "/home/groups/t/tp/tprpg/htdocs", flags=8, errfunc=0,
pglob=0x7fffffffe640) at sftp-glob.c:155
#5 0x0000000000403f2f in process_get (conn=0x8005700c0,
src=0x8005500c8 "htdocs", dst=0x0,
pwd=0x8005600b8 "/home/groups/t/tp/tprpg", pflag=0, rflag=0) at sftp.c:508
#6 0x000000000040622a in parse_dispatch_command (conn=0x8005700c0,
cmd=0x7fffffffec50 "get htdocs", pwd=0x7ffffffff458, err_abort=0)
at sftp.c:1307
#7 0x0000000000407fad in interactive_loop (conn=0x8005700c0, file1=0x0,
file2=0x0) at sftp.c:1975
#8 0x0000000000408983 in main (argc=2, argv=0x7ffffffff5b8) at sftp.c:2249
(gdb) frame 4
#4 0x000000000040cd8d in remote_glob (conn=0x8005700c0,
pattern=0x800570800 "/home/groups/t/tp/tprpg/htdocs", flags=8, errfunc=0,
pglob=0x7fffffffe640) at sftp-glob.c:155
155 return(glob(pattern, flags | GLOB_ALTDIRFUNC, errfunc, pglob));
(gdb) print pattern
$1 = 0x800570800 "/home/groups/t/tp/tprpg/htdocs"
(gdb) frame 5
#5 0x0000000000403f2f in process_get (conn=0x8005700c0,
src=0x8005500c8 "htdocs", dst=0x0,
pwd=0x8005600b8 "/home/groups/t/tp/tprpg", pflag=0, rflag=0) at sftp.c:508
508 if (remote_glob(conn, abs_src, GLOB_MARK, NULL, &g)) {
(gdb) cont
Continuing.

Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.
(gdb) q
> exit

Process terminal finished

Glob without GLOB_ALTDIRFUNC works fine. Maybe there is something in pglob->gl_*
functions.

vasily postnicov (via DragonFly issue tracker) wrote:

I have experienced similar issues with ruby for example, X86_64 build, this
does not appear to be isolated to sftp. I will attempt to get ruby built
with full debugging, and see what I can find out.

RG

What would be usefuly is compiling libc with debug. In any case it'll be much
easier to debug sftp than ruby. If possible, please compile libc with
DEBUG_FLAGS=-g (I think) and get a backtrace with that.

Regards,
Alex

alexh, I've compiled libc (via make buildworld && make installworld) with -g
option. unfortunately I can not "build world" with -O0 option (An error occurs).
Here is some output:

Starting program: /home/vasily/debug/sftp shamazmazum,tprpg@frs.sourceforge.net
shamazmazum,tprpg@frs.sourceforge.net's password:
Connected to frs.sourceforge.net.
sftp> get htdocs

Breakpoint 1, g_lstat (pathbuf=0x7fffffff8440, pathend=<value optimized out>,
pathend_last=0x7fffffffa438, pattern=0x7fffffffa530, pglob=0x7fffffffe5f0,
limit=0x7fffffffe538) at /usr/src/lib/libc/../libc/gen/glob.c:873
873 if (pglob->gl_flags & GLOB_ALTDIRFUNC)
(gdb) list
868
869 if (g_Ctoc(fn, buf, sizeof(buf))) {
870 errno = ENAMETOOLONG;
871 return (-1);
872 }
873 if (pglob->gl_flags & GLOB_ALTDIRFUNC)
874 return((*pglob->gl_lstat)(buf, sb));
875 return(lstat(buf, sb));
876 }
877
(gdb) print buf
$6 = "/home/groups/t/tp/tprpg/htdocs", '\000' <repeats 993 times>
(gdb) cont
Continuing.

Program received signal SIGILL, Illegal instruction.
0x0000000800570800 in ?? ()
(gdb) bt
#0 0x0000000800570800 in ?? ()
#1 0x0000000800bbf2ce in g_lstat (pathbuf=0x7fffffff8440,
pathend=<value optimized out>, pathend_last=0x7fffffffa438,
pattern=0x7fffffffa530, pglob=0x7fffffffe5f0, limit=0x7fffffffe538)
at /usr/src/lib/libc/../libc/gen/glob.c:874
#2 glob2 (pathbuf=0x7fffffff8440, pathend=<value optimized out>,
pathend_last=0x7fffffffa438, pattern=0x7fffffffa530, pglob=0x7fffffffe5f0,
limit=0x7fffffffe538) at /usr/src/lib/libc/../libc/gen/glob.c:571
#3 0x0000000800bbfafb in glob1 (pattern=0x7fffffffc530, pglob=0x7fffffffe5f0,
limit=0x7fffffffe538) at /usr/src/lib/libc/../libc/gen/glob.c:546
#4 glob0 (pattern=0x7fffffffc530, pglob=0x7fffffffe5f0, limit=0x7fffffffe538)
at /usr/src/lib/libc/../libc/gen/glob.c:509
#5 0x0000000800bc0039 in glob (pattern=0x80057081e "", flags=<value optimized
out>,
errfunc=<value optimized out>, pglob=0x7fffffffe5f0)
at /usr/src/lib/libc/../libc/gen/glob.c:243
#6 0x00000000004085ad in remote_glob (conn=0x8005700c0,
pattern=0x800570800 "/home/groups/t/tp/tprpg/htdocs", flags=8, errfunc=0,
pglob=0x7fffffffe5f0) at sftp-glob.c:155
#7 0x000000000040387b in process_get (conn=0x8005700c0, src=0x8005500c8 "htdocs",
dst=0x0, pwd=0x8005600b8 "/home/groups/t/tp/tprpg", pflag=0, rflag=0) at
sftp.c:508
#8 0x0000000000405b76 in parse_dispatch_command (conn=0x8005700c0,
cmd=0x7fffffffec00 "get htdocs", pwd=0x7ffffffff408, err_abort=0) at sftp.c:1307
#9 0x00000000004078f9 in interactive_loop (conn=0x8005700c0, file1=0x0, file2=0x0)
at sftp.c:1975
#10 0x00000000004082cf in main (argc=2, argv=0x7ffffffff560) at sftp.c:2249
(gdb) cont
Continuing.

Program terminated with signal SIGILL, Illegal instruction.
The program no longer exists.

874 return((*pglob->gl_lstat)(buf, sb));

"Step" onto this line in gdb makes an error:
Program received signal SIGILL, Illegal instruction.

Maybe, something it wrong with pglob? I'll try to find it out.

Can you give the following values?
p pglob
p *pglob
p pglob->gl_lstat
x/i *pglob->gl_lstat (or without *, not sure)

On 11/9/2010 19:24, vasily postnicov (via DragonFly issue tracker) wrote:
>
> vasily postnicov<shamaz.mazum@gmail.com> added the comment:
>
> alexh, I've compiled libc (via make buildworld&& make installworld) with -g
> option. unfortunately I can not "build world" with -O0 option (An error occurs).
> Here is some output:

How did you specify -O0 and what error are you getting? Because world
and kernel build just fine here with -O0.

Sascha

> How did you specify -O0 and what error are you getting? Because world
and kernel build just fine here with -O0.

Uh, I am sorry. This is my fault, actually.

> Can you give the following values?

I have something interesting for you. Size of glob_t (type for glob() ) is 88
(at least on my machine):

#include <sys/types.h>
#include <glob.h>
#include <stdio.h>

int main ()
{
printf ("%i\n", sizeof(glob_t));
return 0;
}

$ cc -o test test.c
$ ./test
88

In glob () function it is 88 too, but in process_get() and remote_glob() it is
72. How could it be?

Some output from gdb:

sftp> get htdocs

Breakpoint 1, remote_glob (conn=0x8005800c0,
pattern=0x800580820 "/home/groups/t/tp/tprpg/htdocs", flags=8, errfunc=0,
pglob=0x7fffffffe7a0)
at /usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp-glob.c:148
148 return(glob(pattern, flags | GLOB_ALTDIRFUNC, errfunc, pglob));
(gdb) list
143 pglob->gl_stat = fudge_stat;
144
145 memset(&cur, 0, sizeof(cur));
146 cur.conn = conn;
147
148 return(glob(pattern, flags | GLOB_ALTDIRFUNC, errfunc, pglob));
149 }
(gdb) bt
#0 remote_glob (conn=0x8005800c0,
pattern=0x800580820 "/home/groups/t/tp/tprpg/htdocs", flags=8, errfunc=0,
pglob=0x7fffffffe7a0)
at /usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp-glob.c:148
#1 0x00000000004038db in process_get (conn=0x8005800c0, src=0x8005600f0 "htdocs",
dst=0x0, pwd=0x8005700b8 "/home/groups/t/tp/tprpg", pflag=0, rflag=0)
at /usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp.c:508
#2 0x0000000000405bd6 in parse_dispatch_command (conn=0x8005800c0,
cmd=0x7fffffffedb0 "get htdocs", pwd=0x7ffffffff5b8, err_abort=0)
at /usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp.c:1307
#3 0x0000000000407959 in interactive_loop (conn=0x8005800c0, file1=0x0, file2=0x0)
at /usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp.c:1975
#4 0x000000000040832f in main (argc=2, argv=0x7ffffffff728)
at /usr/src/secure/usr.bin/sftp/../../../crypto/openssh/sftp.c:2249
(gdb) print *pglob
$1 = {gl_pathc = 0, gl_matchc = 0, gl_offs = 0, gl_flags = 0, gl_pathv = 0x0,
gl_errfunc = 0, gl_closedir = 0x40c5ec <fudge_closedir>,
gl_readdir = 0x40c552 <fudge_readdir>, gl_opendir = 0x40c4f4 <fudge_opendir>,
gl_lstat = 0x40c60f <fudge_lstat>, gl_stat = 0x40c661 <fudge_stat>}
(gdb) print sizeof(*pglob)

$3 = 72
(gdb) x/72xb pglob
0x7fffffffe7a0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7a8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7b0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7b8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7c0: 0xec 0xc5 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7c8: 0x52 0xc5 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7d0: 0xf4 0xc4 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7d8: 0x0f 0xc6 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7e0: 0x61 0xc6 0x40 0x00 0x00 0x00 0x00 0x00
(gdb) step

Breakpoint 2, glob (pattern=0x800580820 "/home/groups/t/tp/tprpg/htdocs", flags=72,
errfunc=0, pglob=0x7fffffffe7a0) at /usr/src/lib/libc/../libc/gen/glob.c:187
187 patnext = pattern;
(gdb) print sizeof(*pglob)
$4 = 88
(gdb) x/88xb pglob

0x7fffffffe7a0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7a8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7b0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7b8: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7c0: 0xec 0xc5 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7c8: 0x52 0xc5 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7d0: 0xf4 0xc4 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7d8: 0x0f 0xc6 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7e0: 0x61 0xc6 0x40 0x00 0x00 0x00 0x00 0x00
0x7fffffffe7e8: 0x20 0x08 0x58 0x00 0x08 0x00 0x00 0x00
0x7fffffffe7f0: 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00
(gdb) print *pglob
$5 = {gl_pathc = 0, gl_matchc = 0, gl_offs = 0, gl_flags = 0, gl_pathv = 0x40c5ec,
gl_errfunc = 0x40c552 <fudge_readdir>, gl_closedir = 0x40c4f4 <fudge_opendir>,
gl_readdir = 0x40c60f <fudge_lstat>, gl_opendir = 0x40c661 <fudge_stat>,
gl_lstat = 0x800580820, gl_stat = 0}

:
> In glob () function it is 88 too, but in process_get() and remote_glob() it is
> 72. How could it be?

Hum, apparently sftp uses the OpenBSD definition of glob_t from
/usr/src/crypto/openssh/openbsd-compat/glob.h, whereas the glob()
in libc uses the one from /usr/include/glob.h. Unlike OpenBSD,
FreeBSD and Dragonfly use size_t for the first three members.
Since sizeof(size_t) == sizeof(int) this is not a problem on 32-bit
version of i386 CPUs, but in x86_64 sizeof(size_t) > sizeof(int),
so glob() uses the memory location different from what the caller
intended for function pointers, which led to a crash. BTW at least
gl_pathc and gl_offs are mentioned to be size_t according to the
following URL (the site may be down at the moment):

http://www.opengroup.org/onlinepubs/009695399/basedefs/glob.h.html

Can you try replacing the first three members in glob_t from int
to size_t, that is,

/usr/src/crypto/openssh/openbsd-compat/glob.h:
typedef struct {
int gl_pathc; /* Count of total paths so far. */
int gl_matchc; /* Count of paths matching pattern. */
int gl_offs; /* Reserved at beginning of gl_pathv. */

to:

typedef struct {
size_t gl_pathc; /* Count of total paths so far. */
size_t gl_matchc; /* Count of paths matching pattern. */
size_t gl_offs; /* Reserved at beginning of gl_pathv. */

then rebuild sftp and see if that helps?

> _____________________________________________________
> DragonFly issue tracker <bugs@lists.dragonflybsd.org>
> <http://bugs.dragonflybsd.org/issue1891&gt;
> _____________________________________________________

> Can you try replacing the first three members in glob_t from int
to size_t then rebuild sftp and see if that helps?

Thank you, it works!

On Thu, Nov 11, 2010 at 02:28:40PM +0000, vasily postnicov (via DragonFly issue tracker) wrote:
>
> vasily postnicov <shamaz.mazum@gmail.com> added the comment:
>
> > Can you try replacing the first three members in glob_t from int
> to size_t then rebuild sftp and see if that helps?
>
> Thank you, it works!
>

Instead of messing around with that, can you please try the attached
patch? This should make it use the system's <glob.h>.

--Peter

> can you please try the attached
patch?

It works too. Thanks

Fixed in 24721e97347d08eb0b0ff663136f73b15b78b7b6.