sed(1) adds trailing newline
Our gzip(1) has a trouble unpacking lokigames/idsoftware etc archives (a
shell script and tar.gz in one file) making these packages fail in
pkgsrc. The main trouble is that it's not gzip(1) itself and probably not
libz either that causes the problem, but something else. I have exactly
the same problem with GNU gzip from pkgsrc in DragonFly, but GNU gzip
works just fine with these archives on every other platform I have access
to. Our tar doesn't have problem either with these files (not using gzip,
but libz directly).
Finding out what exactly causes it is beyond my skills at the moment ...
$ fetch \
$ sed '1,265d' linuxq3apoint-1.32b.x86.run | gzip -cd > /dev/null
gzip: input not gziped (MAGIC0)
$ sed '1,265d' linuxq3apoint-1.32b.x86.run | /usr/pkg/bin/gzip -cd \
gzip: stdin: unexpected end of file
$ sed '1,265d' linuxq3apoint-1.32b.x86.run | /usr/bin/bsdtar zxfO - \
#1 Updated by delphij over 7 years ago
-----BEGIN PGP SIGNED MESSAGE-----
NetBSD r1.87 of src/usr.bin/gzip/gzip.c would give a better chance (you
would probably also want 1.88) for gzip(1) to survive with such archive
(issue a warning, but not give a fail case). FWIW FreeBSD's gzip is
doing it this way as well.
Personally I would be inclined to issue such warning (thus the user
would know that there is something wrong with the archive, but still
allow obtaining data) rather than just to 100% match gzip behavior.
What do you think about this case?
-----END PGP SIGNATURE-----
#2 Updated by hasso over 7 years ago
Yes, I know. I updated gzip locally to the latest code from NetBSD. It
then gives warning, but unpacks. But it also returns nonzero value which
means that "gtar zxf" doesn't work - gtar errors out.
I wouldn't mind to change the code to whatever behavior, but ... Why the
very same gzip works _without_ warning/error with the same archive on
NetBSD? Why GNU gzip works _without_ warning/error with the same archive
on every platform but DragonFly?
#3 Updated by qhwt+dfly over 7 years ago
gzip is complaining about the new line that sed appended.
$ sed '1,265d' linuxq3apoint-1.32b.x86.run >a
$ perl -e 'open(A,"./linuxq3apoint-1.32b.x86.run");$_=join("",<A>);$_=~s/.*?\neval \$finish; exit \$res\n//s;print' >b
$ ls -l a b
-rw-r--r-- 1 qhwt wheel 31472010 Mar 4 09:19 a
-rw-r--r-- 1 qhwt wheel 31472009 Mar 4 09:17 b
$ hd -vs 31472000 a
01e03980 01 b1 e3 8c 0a 00 70 47 02 0a |......pG..|
$ hd -vs 31472000 b
01e03980 01 b1 e3 8c 0a 00 70 47 02 |......pG.|
$ gzip -vt a b
gzip: input not gziped (MAGIC0)
a: NOT OK
gzip: a: uncompress failed
#10 Updated by alexh about 7 years ago
Standard dictates: "Whenever the pattern space is written to standard
output or a named file, sed shall immediately follow it with a <newline>."
In my personal opinion, though, I'd prefer to see a gnu-compatible sed, even
if that means breaking standards compliance.
So what is the decision on this? Stick to the standard or stick to gnu?
#11 Updated by corecode about 7 years ago
Alex Hornung (via DragonFly issue tracker) wrote:
> Alex Hornung <email@example.com> added the comment:
> Standard dictates: "Whenever the pattern space is written to standard
> output or a named file, sed shall immediately follow it with a <newline>."
> In my personal opinion, though, I'd prefer to see a gnu-compatible sed, even
> if that means breaking standards compliance.
The standard is exceptionally clear on this, and I don't quite see why we should introduce a regression for an insignificant script which assumes sed is operating in a non-conforming way. Their shell script is wrong.