From: Assaf Gordon <assafgordon@gmail.com>
To: musl@lists.openwall.com, vlse <vlse@veera.biz>
Cc: dalias@libc.org
Subject: Re: Supporting git access via smart HTTPS protocol for musl-libc
Date: Wed, 27 Mar 2019 11:26:22 -0600 [thread overview]
Message-ID: <b7652faa-0d88-c042-16bf-d038cc68bc48@gmail.com> (raw)
In-Reply-To: <20190327053933.GA2518@localhost>
Hello,
On 2019-03-26 11:39 p.m., vlse wrote:
> On Tue, Mar 26, 2019 at 08:15:42PM -0400, Rich Felker wrote:
>> On Tue, Mar 26, 2019 at 07:58:35PM -0400, Rich Felker wrote:
>>> On Tue, Mar 26, 2019 at 04:32:32PM -0600, Assaf Gordon wrote:
>>>>>
>>>>> [...] I suspect thttpd is doing something broken with
>>>>> the POST request since the git clone breaks during that.
>>>>
>>>> The same happened to me with busybox, and was solved by forcing:
>>>>
>>>> export HTTP_CONTENT_ENCODING=gzip
>>>>
>>> Amazingly, this works, but only if I do it only for
>>> REQUEST_METHOD=POST. Otherwise it breaks the GET request and it never
>>> makes it to the POST.
> This does not works:
> mkdir musl.git
> cd musl.git
> git init
> git remote add -t master -m master origin https://git.musl-libc.org/git/musl
>
> git-fetch gives errors:
> fatal: protocol error: bad line length character: erro
Thanks for checking and for the detailed reproducer.
This error tells me there is (again) a binary/text conflict,
The (text) git protocol always starts with 4 hex digits
indicating length. If the input is gzip'd and wasn't
decompressed, the length indicator will be invalid.
Digging further, it seems that in this case the "git fetch"
command sent POST data which is *not* compressed.
Forcing "CONTENT_ENCODING=gzip" always or for all POST request
is not sufficient.
I now use the following shell wrapper, which works for
both "git clone" and your "git fetch" case.
I won't claim it's pretty, but it works (with busybox httpd).
The "echo>&2" will show up on webserver's error log.
----
#!/bin/sh
export GIT_HTTP_EXPORT_ALL=true
export GIT_PROJECT_ROOT=/home/gordon/projects/
echo START - $REQUEST_METHOD $PATH_INFO >&2
# Check if POST data is text or binary, add HTTP header if needed.
if test "$REQUEST_METHOD" = POST ; then
t=$(mktemp -t git-http-backend-XXXXXX)
# Store STDIN and examine it
cat - > $t
# Git (text) protocol starts with 4 hex digits indicating length.
# If the first 4 bytes aren't hexdigits, assume STDIN is compressed.
if head -c4 $t | grep -q '^[0-9a-f][0-9a-f][0-9a-f][0-9a-f]$' ; then
echo "POST data is not gzipped" >&2
else
echo "POST data is gzipped" >&2
export HTTP_CONTENT_ENCODING=gzip
fi
# restore STDIN
exec < $t
rm $t
fi
/usr/lib/git-core/git-http-backend
echo END - $REQUEST_METHOD $PATH_INFO >&2
echo >&2
echo >&2
----
As Rich said, there's got to be a better way...
Regards,
-assaf
next prev parent reply other threads:[~2019-03-27 17:26 UTC|newest]
Thread overview: 29+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <20190324103306.GB1830@localhost>
[not found] ` <20190326003411.GC1872@localhost>
2019-03-26 1:09 ` vlse
2019-03-26 1:17 ` A. Wilcox
2019-03-26 1:37 ` Rich Felker
2019-03-26 1:54 ` vlse
2019-03-26 2:59 ` Rich Felker
2019-03-26 10:02 ` vlse
2019-03-26 10:36 ` Laurent Bercot
2019-03-26 15:04 ` Rich Felker
2019-03-26 15:09 ` Drew DeVault
2019-03-26 15:13 ` Rich Felker
2019-03-26 15:43 ` Drew DeVault
2019-03-26 15:47 ` Rich Felker
2019-03-26 15:57 ` Drew DeVault
2019-03-26 17:57 ` Rich Felker
2019-03-26 20:32 ` A. Wilcox
2019-03-26 20:39 ` Assaf Gordon
2019-03-26 22:02 ` Rich Felker
2019-03-26 22:32 ` Assaf Gordon
2019-03-26 23:58 ` Rich Felker
2019-03-27 0:15 ` Rich Felker
2019-03-27 5:39 ` vlse
2019-03-27 17:26 ` Assaf Gordon [this message]
2019-03-27 17:41 ` Assaf Gordon
2019-04-03 6:42 ` vlse
2019-03-26 10:19 ` Jens Gustedt
2019-03-26 10:30 ` vlse
2019-03-26 14:59 ` Rich Felker
2019-03-26 1:43 ` vlse
2019-03-26 2:29 ` A. Wilcox
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=b7652faa-0d88-c042-16bf-d038cc68bc48@gmail.com \
--to=assafgordon@gmail.com \
--cc=dalias@libc.org \
--cc=musl@lists.openwall.com \
--cc=vlse@veera.biz \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/musl/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).