[musl] git cloning over https?

[musl] git cloning over https?

[Florian Fainelli]

Hi Rich,

Would you consider allowing git.musl-libc.org to serve the repositories 
over https rather than git? For some obscure reasons some corporate 
environments do block the git protocol whereas https goes through.

Thanks!
-- 
Florian

Re: [musl] git cloning over https?

[Leah Neukirchen]

Florian Fainelli <f.fainelli@gmail.com> writes:

> Hi Rich,
>
> Would you consider allowing git.musl-libc.org to serve the
> repositories over https rather than git? For some obscure reasons some
> corporate environments do block the git protocol whereas https goes
> through.

In the meantime, I run a hourly-updated mirror at:
https://git.vuxu.org/mirror/musl

-- 
Leah Neukirchen  <leah@vuxu.org>  https://leahneukirchen.org/

Re: [musl] git cloning over https?

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

Hi,

On Thu, Jan 13, 2022 at 01:44:23PM -0800, Florian Fainelli wrote:
> Would you consider allowing git.musl-libc.org to serve the repositories over
> https rather than git? For some obscure reasons some corporate environments
> do block the git protocol whereas https goes through.

You should be able to git clone https://git.musl-libc.org/cgit/musl.
It is quite slow though -- since it looks like cgit is serving, this
is presumably because cgit's built-in git serving feature only supports
the legacy ("dumb") HTTP program, not the fast ("smart") one used by
git http-backend, which would be a nice infrastructure improvement.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [musl] git cloning over https?

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 821 bytes --]

Sorry Florian, looks like I mistakenly dropped you somehow from the
recipient list of the below reply:

On Thu, Jan 13, 2022 at 09:53:43PM +0000, Alyssa Ross wrote:
> Hi,
>
> On Thu, Jan 13, 2022 at 01:44:23PM -0800, Florian Fainelli wrote:
> > Would you consider allowing git.musl-libc.org to serve the repositories over
> > https rather than git? For some obscure reasons some corporate environments
> > do block the git protocol whereas https goes through.
>
> You should be able to git clone https://git.musl-libc.org/cgit/musl.
> It is quite slow though -- since it looks like cgit is serving, this
> is presumably because cgit's built-in git serving feature only supports
> the legacy ("dumb") HTTP program, not the fast ("smart") one used by
> git http-backend, which would be a nice infrastructure improvement.



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [musl] git cloning over https?

[Florian Fainelli]

On 1/13/2022 1:53 PM, Alyssa Ross wrote:
> Hi,
> 
> On Thu, Jan 13, 2022 at 01:44:23PM -0800, Florian Fainelli wrote:
>> Would you consider allowing git.musl-libc.org to serve the repositories over
>> https rather than git? For some obscure reasons some corporate environments
>> do block the git protocol whereas https goes through.
> 
> You should be able to git clone https://git.musl-libc.org/cgit/musl.
> It is quite slow though -- since it looks like cgit is serving, this
> is presumably because cgit's built-in git serving feature only supports
> the legacy ("dumb") HTTP program, not the fast ("smart") one used by
> git http-backend, which would be a nice infrastructure improvement.

I don't seem to be able to get cloning from cgit to work either and not 
doing this in the corporate environment I vaguely mentioned but over my 
home residential provider (Cox). git:// works.
-- 
Florian

Re: [musl] git cloning over https?

[Markus Mayer]

On Thu, 13 Jan 2022 at 14:12, Florian Fainelli <f.fainelli@gmail.com> wrote:
>
>
>
> On 1/13/2022 1:53 PM, Alyssa Ross wrote:
> > Hi,
> >
> > On Thu, Jan 13, 2022 at 01:44:23PM -0800, Florian Fainelli wrote:
> >> Would you consider allowing git.musl-libc.org to serve the repositories over
> >> https rather than git? For some obscure reasons some corporate environments
> >> do block the git protocol whereas https goes through.
> >
> > You should be able to git clone https://git.musl-libc.org/cgit/musl.
> > It is quite slow though -- since it looks like cgit is serving, this
> > is presumably because cgit's built-in git serving feature only supports
> > the legacy ("dumb") HTTP program, not the fast ("smart") one used by
> > git http-backend, which would be a nice infrastructure improvement.
>
> I don't seem to be able to get cloning from cgit to work either and not
> doing this in the corporate environment I vaguely mentioned but over my
> home residential provider (Cox). git:// works.

Interestingly, it worked for me. Kind of. It is *extremely* slow. It
took about 15 minutes to clone the repo, which isn't big at all and
should take no more than a minute to transfer (likely a lot less).

Interestingly, the local clone created via cgit and https is 52 MB in
size while a clone via the git:// protocol is only 20 MB. Something
"interesting" seems to be happening when cloning via cgit. The repo
does seem to be fully functional, though.

Regards,
-Markus

Re: [musl] git cloning over https?

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 2029 bytes --]

On Thu, Jan 13, 2022 at 02:23:54PM -0800, Markus Mayer wrote:
> On Thu, 13 Jan 2022 at 14:12, Florian Fainelli <f.fainelli@gmail.com> wrote:
> >
> >
> >
> > On 1/13/2022 1:53 PM, Alyssa Ross wrote:
> > > Hi,
> > >
> > > On Thu, Jan 13, 2022 at 01:44:23PM -0800, Florian Fainelli wrote:
> > >> Would you consider allowing git.musl-libc.org to serve the repositories over
> > >> https rather than git? For some obscure reasons some corporate environments
> > >> do block the git protocol whereas https goes through.
> > >
> > > You should be able to git clone https://git.musl-libc.org/cgit/musl.
> > > It is quite slow though -- since it looks like cgit is serving, this
> > > is presumably because cgit's built-in git serving feature only supports
> > > the legacy ("dumb") HTTP program, not the fast ("smart") one used by
> > > git http-backend, which would be a nice infrastructure improvement.
> >
> > I don't seem to be able to get cloning from cgit to work either and not
> > doing this in the corporate environment I vaguely mentioned but over my
> > home residential provider (Cox). git:// works.
>
> Interestingly, it worked for me. Kind of. It is *extremely* slow. It
> took about 15 minutes to clone the repo, which isn't big at all and
> should take no more than a minute to transfer (likely a lot less).
>
> Interestingly, the local clone created via cgit and https is 52 MB in
> size while a clone via the git:// protocol is only 20 MB. Something
> "interesting" seems to be happening when cloning via cgit. The repo
> does seem to be fully functional, though.

From what I remember (and take this with a grain of salt, because it's
been a long time since I looked into it), the "dumb" HTTP protocol works
by downloading pre-built packfiles from a static web server, whereas the
"smart" protocol can negotiate exactly the objects it needs with the
server.  This might explain the size difference.

I imagine if you git gc you'll end up with the same size as the git://
clone.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [musl] git cloning over https?

[Markus Mayer]

On Thu, 13 Jan 2022 at 15:25, Alyssa Ross <hi@alyssa.is> wrote:
>
> From what I remember (and take this with a grain of salt, because it's
> been a long time since I looked into it), the "dumb" HTTP protocol works
> by downloading pre-built packfiles from a static web server, whereas the
> "smart" protocol can negotiate exactly the objects it needs with the
> server.  This might explain the size difference.
>
> I imagine if you git gc you'll end up with the same size as the git://
> clone.

You are absolutely correct.

$ du -sh musl-*
52M musl-http
20M musl-git
$ cd musl-http; git gc; cd ..
Counting objects: 100% (36120/36120), done.
Delta compression using up to 48 threads
Compressing objects: 100% (15465/15465), done.
Writing objects: 100% (36120/36120), done.
Total 36120 (delta 26317), reused 27955 (delta 20259)
$ du -sh musl-*
20M musl-http
20M musl-git

Regards,
-Markus

Re: [musl] git cloning over https?

[Rich Felker]

On Thu, Jan 13, 2022 at 01:44:23PM -0800, Florian Fainelli wrote:
> Hi Rich,
> 
> Would you consider allowing git.musl-libc.org to serve the
> repositories over https rather than git? For some obscure reasons
> some corporate environments do block the git protocol whereas https
> goes through.
> 
> Thanks!

I've been trying to, but there's a content transfer encoding issue
getting the requests through thttpd's cgi right (something about gzip)
that I haven't been able to resolve, that's blocking it from actually
working. The right solution is either migrating httpd (but there are
no good ones :/) or patching thttpd to convey this part of the
request to cgi correctly. If there are any volunteers to help with the
latter I'd be happy to test.

Rich