Re: "musl" gzip-compressed tarballs are decompressed if using newest wegt release

[Rich Felker <dalias@libc.org>]

On Wed, Nov 01, 2017 at 04:50:01PM +0100, Jens Schleusener wrote:
> On Wed, 1 Nov 2017, Rich Felker wrote:
> 
> >On Wed, Nov 01, 2017 at 11:19:30AM +0100, Jens Schleusener wrote:
> >>Hi,
> >>
> >>not a real "musl" software problem but just an observation of a
> >>changed behaviour of "musl" tarball downloads with the newest "wegt"
> >>release (1.19.2): The tarballs are saved now decompressed but
> >>contains the original extension (.tar.gz). Although if the extension
> >>would be .tar it seems at least to me an undesired behaviour.
> >>
> >>Reason seems a new "wget" feature, here some extracted lines taken
> >>from the ChangeLog
> >>
> >> 2017-08-04
> >> Add gzip Content-Encoding decompression
> >> (gethttp): Decompress files with gzip Content-Encoding
> >>
> >>in combination with the HTTP header the www.musl-libc.org server delivers:
> >>
> >> Content-Type: application/x-tar
> >> Content-Encoding: gzip
> >>
> >>Solutions/workarounds may be on the server side delivering of a HTTP
> >>header like
> >>
> >> Content-Type: application/x-gzip
> >> (or Content-Type: application/octet-stream)
> >>
> >>or on the client side the use of the new "wget" option
> >>
> >> --compression=none
> >
> >IMO this is both a bug in the server and a major regression in wget,
> >since lots of servers have this bug. I'll look at fixing it on my side
> >but I think it should be reported to wget and reverted since this is
> >going to break a *huge* number of build scripts that download tarballs
> >from affected servers.
> 
> Thanks for the more clear statement (I agree). If not already done
> by another person I will send an accoding mail to the "bug-wget"
> list.

It should be fixed now. I've written a patch for thttpd that removes
the content-transfer-encoding nonsense and submitted it for inclusion
in Alpine Linux (used on musl's web host) and already installed the
patched package myself. Hopefully this will eventually make it
upstream to thttpd, but it seems mostly unmaintained.

FWIW I'd love to switch to a better httpd that's actually maintained,
but I need something with simple migration path and without loads of
unwanted bloat. Needs to be able to:

- support host header based vhosts, translated to dir names under web
  root.

- execute cgi (only!) from a configured whitelist of pathnames

- support url paths that go past the cgi program, i.e. /cgit/musl/...
  where /cgit is the cgi executable

- honor symlinks (without any attempt to [re]interpret them)

Solution which involve a process per static-content request are not
really desirable. Recommendations would be welcome.

Rich