Re: Supporting git access via smart HTTPS protocol for musl-libc

[vlse <vlse@veera.biz>]

On Wed, Mar 27, 2019 at 11:26:22AM -0600, Assaf Gordon wrote:
> Hello,
> 
> On 2019-03-26 11:39 p.m., vlse wrote:
> >On Tue, Mar 26, 2019 at 08:15:42PM -0400, Rich Felker wrote:
> >>On Tue, Mar 26, 2019 at 07:58:35PM -0400, Rich Felker wrote:
> >>>On Tue, Mar 26, 2019 at 04:32:32PM -0600, Assaf Gordon wrote:
> >>>>>
> >>>>>[...] I suspect thttpd is doing something broken with
> >>>>>the POST request since the git clone breaks during that. 
> >>>>
> >>>>The same happened to me with busybox, and was solved by forcing:
> >>>>
> >>>>     export HTTP_CONTENT_ENCODING=gzip
> >>>>
> >>>Amazingly, this works, but only if I do it only for
> >>>REQUEST_METHOD=POST. Otherwise it breaks the GET request and it never
> >>>makes it to the POST.
> 
> >This does not works:
> >mkdir musl.git
> >cd musl.git
> >git init
> >git remote add -t master -m master origin 
> >https://git.musl-libc.org/git/musl
> >
> >git-fetch gives errors:
> >   fatal: protocol error: bad line length character: erro
> 
> Thanks for checking and for the detailed reproducer.
> 
> This error tells me there is (again) a binary/text conflict,
> 
> The (text) git protocol always starts with 4 hex digits
> indicating length. If the input is gzip'd and wasn't
> decompressed, the length indicator will be invalid.
> 
> Digging further, it seems that in this case the "git fetch"
> command sent POST data which is *not* compressed.
> Forcing "CONTENT_ENCODING=gzip" always or for all POST request
> is not sufficient.
> 
> I now use the following shell wrapper, which works for
> both "git clone" and your "git fetch" case.
> I won't claim it's pretty, but it works (with busybox httpd).
> The "echo>&2" will show up on webserver's error log.
> 
> ----
> #!/bin/sh
> 
> export GIT_HTTP_EXPORT_ALL=true
> export GIT_PROJECT_ROOT=/home/gordon/projects/
> 
> echo START - $REQUEST_METHOD $PATH_INFO >&2
> 
> # Check if POST data is text or binary, add HTTP header if needed.
> if test "$REQUEST_METHOD" = POST ; then
>     t=$(mktemp -t git-http-backend-XXXXXX)
>     # Store STDIN and examine it
>     cat - > $t
> 
>     # Git (text) protocol starts with 4 hex digits indicating length.
>     # If the first 4 bytes aren't hexdigits, assume STDIN is compressed.
>     if head -c4 $t | grep -q '^[0-9a-f][0-9a-f][0-9a-f][0-9a-f]$' ; then
>         echo "POST data is not gzipped" >&2
>     else
>         echo "POST data is gzipped" >&2
>         export HTTP_CONTENT_ENCODING=gzip
>     fi
> 
>     # restore STDIN
>     exec < $t
>     rm $t
> fi
> /usr/lib/git-core/git-http-backend
> echo END - $REQUEST_METHOD $PATH_INFO >&2
> echo >&2
> echo >&2
> 
> ----
> 
> As Rich said, there's got to be a better way...
> 
> 
> Regards,
>   -assaf

Hi,

I previously reported that doing git clone and then git pull was showing no errors.
git pull said "Already upto Date"

But when I did git pull again, I got the following errors:

 fatal: protocol error: bad line length character: erro

This was same error which I told when a fresh git fetch was producing.

The git pull actually invokes git fetch followed by git merge.
After a fresh clone and then doing git pull immediately does not invokes git fetch at all.

Regards,
Veera