author/committer/tagger links -- kernel.org?

author/committer/tagger links -- kernel.org?

[Jason]

Hey Konstantin,

On Thu, Jan 9, 2014 at 4:21 PM, Konstantin Ryabitsev <mricon at kernel.org> wrote:
> That's pretty nifty. That reminds me -- I'm working on a web-of-trust
> site for kernel.org and something I wouldn't mind having is a way to
> link from cgit to the web of trust for that person. E.g. an email
> address for "torvalds at linux-foundation.org" on this page
> (http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d6e0a2dd12f4067a5bcefb8bbd8ddbeff800afbc)
> would be wrapped in a link such as:
>
> <a href="https://blah.kernel.org/?user=torvalds%40linux-foundation.org">
> torvalds at linux-foundation.org</a>
>
> which will bring up a page similar to:
> https://www.kernel.org/doc/wot/torvalds.html

Not sure if you've been following along, but we've got the
email-filter patches merged along with the lua support, so this should
allow for some pretty neat scripts to do exactly what you want --
<http://git.zx2c4.com/cgit/tree/filters/email-gravatar.lua> as a
sample. Is this API sufficient for you? If you don't want to have the
web of trust links everywhere, you can use the "page" argument to
filter_open as a distinguishing factor. Please let me know if you need
anything else!

Jason

author/committer/tagger links -- kernel.org?

[Jason]

On Thu, Jan 16, 2014 at 2:22 AM, Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>> <a href="https://blah.kernel.org/?user=torvalds%40linux-foundation.org">
>> torvalds at linux-foundation.org</a>

Oh, and documentation:
<http://git.zx2c4.com/cgit/tree/cgitrc.5.txt#n578>. Of note is html()
and html_txt() and html_attr() and ...

Jason

owner links? (was: author/committer/tagger links -- kernel.org?)

[mackyle]

On Jan 15, 2014, at 17:22, Jason A. Donenfeld wrote:
> On Thu, Jan 9, 2014 at 4:21 PM, Konstantin Ryabitsev <mricon at kernel.org 
> > wrote:
>> That's pretty nifty. That reminds me -- I'm working on a web-of-trust
>> site for kernel.org and something I wouldn't mind having is a way to
>> link from cgit to the web of trust for that person. E.g. an email
>> address for "torvalds at linux-foundation.org" on this page
>> (http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=d6e0a2dd12f4067a5bcefb8bbd8ddbeff800afbc 
>> )
>> would be wrapped in a link such as:
>>
>> <a href="https://blah.kernel.org/?user=torvalds%40linux-foundation.org 
>> ">
>> torvalds at linux-foundation.org</a>
>>
>> which will bring up a page similar to:
>> https://www.kernel.org/doc/wot/torvalds.html
>
> Not sure if you've been following along, but we've got the
> email-filter patches merged along with the lua support, so this should
> allow for some pretty neat scripts to do exactly what you want --
> <http://git.zx2c4.com/cgit/tree/filters/email-gravatar.lua> as a
> sample. Is this API sufficient for you? If you don't want to have the
> web of trust links everywhere, you can use the "page" argument to
> filter_open as a distinguishing factor. Please let me know if you need
> anything else!

On repo.or.cz I've hooked up project owner to a link that shows a list  
of all projects owned by the same person.

So, for example, the owner name is passed in and you end up getting  
back links like [1] and [2].  (You can see where this shows up by  
pulling up one of the project summary pages on repo.or.cz such as the  
cgit mirror [3].)

Looking at the new docs for this [4] I don't see an option to filter  
owners.  I see that the owner is passed in as "CGIT_REPO_OWNER", but I  
don't see an opportunity to produce the owner markup.  Is it there and  
I'm just not seeing it?

The repo.or.cz pages also try to make the email addresses more  
difficult to scrape by generating peculiar markup (the  
HTML::Email::Obfuscate package) so conceivably this could be handled  
in the owner filter as well.

Kyle

[1] http://repo.or.cz/projlist.cgi?name=18813f71506ebad74179bf8c5a136696
[2] http://repo.or.cz/projlist.cgi?name=86f09ae391e201f5d0c4b89906b2010b
[3] http://repo.or.cz/w/cgit.git
[4] http://git.zx2c4.com/cgit/tree/cgitrc.5.txt#n578

owner links? (was: author/committer/tagger links -- kernel.org?)

[Jason]

On Thu, Jan 16, 2014 at 4:28 AM, Kyle J. McKay <mackyle at gmail.com> wrote:
> On repo.or.cz I've hooked up project owner to a link that shows a list of
> all projects owned by the same person.
>
> So, for example, the owner name is passed in and you end up getting back
> links like [1] and [2].  (You can see where this shows up by pulling up one
> of the project summary pages on repo.or.cz such as the cgit mirror [3].)
>
> Looking at the new docs for this [4] I don't see an option to filter owners.
> I see that the owner is passed in as "CGIT_REPO_OWNER", but I don't see an
> opportunity to produce the owner markup.  Is it there and I'm just not
> seeing it?

This is certainly interesting, though slightly different, as the
owners aren't attached to email authors, like in the commits. Still
though, something worth exploring. There are a number of options for
this. I could add a filter wrapper around the owner column values. Or,
I could implement the search capability directly in cgit. Or other
things.

First, though -- is repo.or.cz considering moving to cgit from its
modified gitweb? Is this a serious investigation into cgit as a
replacement?

>
> The repo.or.cz pages also try to make the email addresses more difficult to
> scrape by generating peculiar markup (the HTML::Email::Obfuscate package) so
> conceivably this could be handled in the owner filter as well.

We have the noplainemail config flag already. Perhaps this could be
phased out, preferring the email-filter and commit-filter instead.

owner links? (was: author/committer/tagger links -- kernel.org?)

[mackyle]

On Jan 16, 2014, at 03:18, Jason A. Donenfeld wrote:
> On Thu, Jan 16, 2014 at 4:28 AM, Kyle J. McKay <mackyle at gmail.com>  
> wrote:
>> Looking at the new docs for this [4] I don't see an option to  
>> filter owners.
>> I see that the owner is passed in as "CGIT_REPO_OWNER", but I don't  
>> see an
>> opportunity to produce the owner markup.  Is it there and I'm just  
>> not
>> seeing it?
>
> This is certainly interesting, though slightly different, as the
> owners aren't attached to email authors, like in the commits. Still
> though, something worth exploring. There are a number of options for
> this. I could add a filter wrapper around the owner column values. Or,
> I could implement the search capability directly in cgit. Or other
> things.

Here's the relevant change to gitweb on repo that supports this:

   # owner link hook given owner name (full and NOT obfuscated)
   # should return full URL-escaped link to attach to owner, for  
example:
   #    sub { return "/showowner.cgi?owner=".CGI::Util::escape($_[0]); }
   our $owner_link_hook = undef;

And we use this hook:

   $owner_link_hook =
     sub { url_path($Girocco::Config::webadmurl).
           "/projlist.cgi?name=".md5_hex($_[0]); };

> First, though -- is repo.or.cz considering moving to cgit from its
> modified gitweb? Is this a serious investigation into cgit as a
> replacement?

Replacement's a strong word at this point.  I've been looking into it  
ever since I found this in the Apache config:

   RewriteRule ^/c/(.*)$ /cgit.cgi?url=$1 [L,QSA]

So a cgit version of the cgit mirror would be http://repo.or.cz/c/cgit.git 
  (don't bother, 404 at this point).

I do slip in the occasional preparatory work [1] to make cgit support  
easier when I become aware of such things. :)

Looking at top, gitweb is clearly a CPU pig on repo.or.cz (git clones  
can also be piggy but I expect that to drop significantly when the new  
pack bitmap support is deployed).  And besides, the cgit display just  
looks good.  I'd like to add support for cgit to girocco [2] as an  
alternative to gitweb.  When that's sufficiently mature I'd like to  
deploy it alongside gitweb on repo for a time and then we can talk  
about switching.  :)

>> The repo.or.cz pages also try to make the email addresses more  
>> difficult to
>> scrape by generating peculiar markup (the HTML::Email::Obfuscate  
>> package) so
>> conceivably this could be handled in the owner filter as well.
>
> We have the noplainemail config flag already. Perhaps this could be
> phased out, preferring the email-filter and commit-filter instead.

As you can see from the hook text above the repo gitweb hook only  
offers the ability to attach a link, it doesn't actually let one  
generate the markup or change the displayed owner name.  That's all I  
really need.  On the project list page [3] there are thousands of  
these, so running a shell script for each one is a non-starter.  A  
simple regex is not good enough either because it needs to generate  
the md5 hash (probably ought to be md5_hex(lc($_[0])) too -- another  
TODO item).

Kyle

[1] http://repo.or.cz/w/girocco.git/commit/abe9554c1bf1699ac0aaea08639310f404458d14
[2] http://repo.or.cz/w/girocco.git/blob/HEAD:/README
[3] http://repo.or.cz/w?a=project_list

owner links? (was: author/committer/tagger links -- kernel.org?)

[Jason]

On Thu, Jan 16, 2014 at 2:46 PM, Kyle J. McKay <mackyle at gmail.com> wrote:
> And we use this hook:
>
>   $owner_link_hook =
>     sub { url_path($Girocco::Config::webadmurl).
>           "/projlist.cgi?name=".md5_hex($_[0]); };
>

Well, that wound up being totally trivial, and a logical thing to have
by default anyway:

http://git.zx2c4.com/cgit/commit/?id=a58e6863cfe4f1411fedd556a96d8f8b9bf75761
You can try it out on: http://git.zx2c4.com/



> Looking at top, gitweb is clearly a CPU pig on repo.or.cz (git clones can
> also be piggy but I expect that to drop significantly when the new pack
> bitmap support is deployed).  And besides, the cgit display just looks good.
> I'd like to add support for cgit to girocco [2] as an alternative to gitweb.
> When that's sufficiently mature I'd like to deploy it alongside gitweb on
> repo for a time and then we can talk about switching.  :)

That'd be very cool to see. What types of things do you see cgit
needing to facilitate this?

> As you can see from the hook text above the repo gitweb hook only offers the
> ability to attach a link, it doesn't actually let one generate the markup or
> change the displayed owner name.  That's all I really need.  On the project
> list page [3] there are thousands of these, so running a shell script for
> each one is a non-starter.  A simple regex is not good enough either because
> it needs to generate the md5 hash (probably ought to be md5_hex(lc($_[0]))
> too -- another TODO item).

I don't know how closely you read the list, but some exciting things
have been happening the last two weeks. I too didn't like the idea of
having to run a shell script everytime I wanted to add a gravatar to
an email address, but I didn't l like the idea of hard coding the
logic to do so into cgit itself. So I integrated Lua/LuaJIT. It's
super super fast, only needs to evaluate the script one time (and then
each email it filters reuses the same context), and most importantly,
doesn't need to fork ever. We're using this for the new authentication
framework too:

http://git.zx2c4.com/cgit/tree/cgitrc.5.txt#n578
http://git.zx2c4.com/cgit/tree/filters/email-gravatar.lua
http://git.zx2c4.com/cgit/tree/filters/simple-authentication.lua

So it seems most feasible to use this for email obfuscation. (As I
mentioned before, we already have the "noplainemail" config option,
but we should probably remove this in favor of using this new filter
API.)

owner links? (was: author/committer/tagger links -- kernel.org?)

[mackyle]

On Jan 16, 2014, at 14:02, Jason A. Donenfeld wrote:
> On Thu, Jan 16, 2014 at 2:46 PM, Kyle J. McKay <mackyle at gmail.com>  
> wrote:
>> And we use this hook:
>>
>>  $owner_link_hook =
>>    sub { url_path($Girocco::Config::webadmurl).
>>          "/projlist.cgi?name=".md5_hex($_[0]); };
>>
>
> Well, that wound up being totally trivial, and a logical thing to have
> by default anyway:
>
> http://git.zx2c4.com/cgit/commit/?id=a58e6863cfe4f1411fedd556a96d8f8b9bf75761
> You can try it out on: http://git.zx2c4.com/

Nice.  :)

Of course Girocco requires owner names to be email addresses (and if  
you ever want to update your email address and/or list of public ssh  
keys it needs to be an email address that works) so that would expose  
those to scraping by spam bots...

But we can cross that bridge when we come to it.

>> And besides, the cgit display just looks good.
>> I'd like to add support for cgit to girocco [2] as an alternative  
>> to gitweb.
>> When that's sufficiently mature I'd like to deploy it alongside  
>> gitweb on
>> repo for a time and then we can talk about switching.  :)
>
> That'd be very cool to see. What types of things do you see cgit
> needing to facilitate this?

I wouldn't even propose switching until there would be no lost  
functionality, but it would be okay to run side-by-side with gitweb  
still as the default without feature parity.

For example, this link [1] shows ALL refs in the repository, not just  
those under refs/heads and refs/tags.  That's not just important for  
mirrored repositories, the "Personal Mob Branches" feature [2] also  
needs it to be able to easily browse those.

GitWeb lets you insert custom links into the top of the page, so you  
can see every Girocco project page has a "graphiclog", "edit" and  
"fork" link (I presume that will be easy and possibly already  
supported by cgit).  Handling the fork display may be a bit tricker in  
cgit (especially perhaps on the projects list page).  And while the  
graphiclog browser is not the prettiest thing to use, it does draw  
graphs that can be much easier to follow than the text-based graphs.   
Compare [3] to [4].  That's not to say that graphiclog doesn't have  
its own issues (it is pretty old and not really maintained).  It would  
be nice to update cgit to draw graphs more like googlecode does (see  
[10] for an example).

I think cgit is also missing blame (see [5] for an example).  The  
project tags would also have to be supported, but will probably be  
easy in comparison to blame.

> I don't know how closely you read the list, but some exciting things
> have been happening the last two weeks.

I have been lurking for a while (since before the list moved to  
zx2c4).  I just haven't got to cgit support yet (send me a round toit  
please [11]).  Now that repo.or.cz has https push support there's just  
a few more things I need to tidy up in Girocco before I can really  
focus on cgit support.  Which reminds me that cgit will need non- 
scheme-escaping links.  What I mean is that if I browse to say "http://git.zx2c4.com/cgit 
" all the links on the returned page are "http:" where as if I browse  
to "https://git.zx2c4.com/cgit" all the links on the page come back as  
"https:" (possibly excluding, of course, non-cgit links such as  
gravitars and so on).  As part of adding https support to Girocco, I  
corrected this so that the returned pages do not contain any links  
that escape the scheme that was used to get there.  It may be that  
cgit already has this, but since "https://git.zx2c4.com/cgit" doesn't  
load for me I can't be sure. ;)

> I too didn't like the idea of
> having to run a shell script everytime I wanted to add a gravatar to
> an email address, but I didn't l like the idea of hard coding the
> logic to do so into cgit itself.

Yes.  Gitweb's ability to include perl snippets as part of the gitweb  
config file is very powerful and Lua brings that to cgit.  I haven't  
looked at the cgit auth stuff (yet) because I don't expect it will be  
applicable to Girocco.

> So I integrated Lua/LuaJIT.

May I suggest adding a LuaJIT mirror (see [6] and [7]) to zx2c4 and  
adding that to the .gitmodules file and making it the default LuaJIT  
implementation built as part of the normal cgit make process?  The  
LuaJIT repository is so small (3.5M) compared to the git repository  
already in .gitmodules (205M) that it does not seem like an issue to  
add it.

And while you're in there, why not make the the URLs in .gitmodules  
relative since zx2c4 is already mirroring git?  That would mean that  
anyone else that mirrors cgit, git and luajit would have their mirrors  
used automatically.  (Blender's main git repo does this in  
their .gitmodules file.)  Not really a big deal, just a suggestion.

The Lua project does not have a public source repository (see [8] and  
[9]) and that reason right there is more than enough to convince me to  
always build cgit with LuaJIT instead of Lua.

--Kyle

P.S. Now stepping down off the soap box that I seemed to find myself  
standing on there at the end of this email. ;)

[1] http://repo.or.cz/w/class-dump.git/refs
[2] http://repo.or.cz/h/mob.html
[3] http://git.zx2c4.com/git/log/
[4] http://repo.or.cz/git-browser/by-commit.html?r=git.git
[5] <http://repo.or.cz/w/cgit.git/blame/a58e6863cfe4f1411fedd556a96d8f8b9bf75761:/ui-repolist.c 
 >
[6] http://luajit.org/download.html
[7] http://luajit.org/git/luajit-2.0.git
[8] http://www.lua.org/faq.html#1.8
[9] http://lua-users.org/lists/lua-l/2008-06/msg00407.html
[10] http://code.google.com/p/git-mirror/source/list
[11] http://bytesdaily.blogspot.com/2011/11/round-tuit.html

owner links? (was: author/committer/tagger links -- kernel.org?)

[Jason]

On Fri, Jan 17, 2014 at 7:58 AM, Kyle J. McKay <mackyle at gmail.com> wrote:
> For example, this link [1] shows ALL refs in the repository, not just those
> under refs/heads and refs/tags.  That's not just important for mirrored
> repositories, the "Personal Mob Branches" feature [2] also needs it to be
> able to easily browse those.

enable-remote-branches=1

For investigation into "what about the other refs? how do these work
with cgit?", if you send a well researched separate email for
something like this to the list, I or someone else will have a look at
it in do time.

>
> GitWeb lets you insert custom links into the top of the page, so you can see
> every Girocco project page has a "graphiclog", "edit" and "fork" link (I
> presume that will be easy and possibly already supported by cgit).  Handling
> the fork display may be a bit tricker in cgit (especially perhaps on the
> projects list page).  And while the graphiclog browser is not the prettiest
> thing to use, it does draw graphs that can be much easier to follow than the
> text-based graphs.  Compare [3] to [4].  That's not to say that graphiclog
> doesn't have its own issues (it is pretty old and not really maintained).
> It would be nice to update cgit to draw graphs more like googlecode does
> (see [10] for an example).

That's very pretty indeed. If you'd like to revamp our graph display,
a patch would be most welcome.

>
> I think cgit is also missing blame (see [5] for an example).  The project
> tags would also have to be supported, but will probably be easy in
> comparison to blame.

Blame support would be very nice; I personally would enjoy this
feature. Could you send a [well researched and thought-out] separate
thread about this? (A patch would be nicest, but in lieu of that,
maybe someone else will write it for us...)


> Which
> reminds me that cgit will need non-scheme-escaping links.  What I mean is
> that if I browse to say "http://git.zx2c4.com/cgit" all the links on the
> returned page are "http:" where as if I browse to
> "https://git.zx2c4.com/cgit" all the links on the page come back as "https:"
> (possibly excluding, of course, non-cgit links such as gravitars and so on).
> As part of adding https support to Girocco, I corrected this so that the
> returned pages do not contain any links that escape the scheme that was used
> to get there.  It may be that cgit already has this, but since
> "https://git.zx2c4.com/cgit" doesn't load for me I can't be sure. ;)

Whenever we reference a resource by a full host name (which is rare),
we use "//", including for Gravatars, so that they're loaded from the
same scheme as the page. Do you have any indication that we do
otherwise?

>
> May I suggest adding a LuaJIT mirror (see [6] and [7]) to zx2c4 and adding
> that to the .gitmodules file and making it the default LuaJIT implementation
> built as part of the normal cgit make process?  The LuaJIT repository is so
> small (3.5M) compared to the git repository already in .gitmodules (205M)
> that it does not seem like an issue to add it.

That's not a bad idea. OTOH, LuaJIT and Lua are best installed as
system wide shared libraries, not a static library we link to, as in
the case of libgit.a. Further, LuaJIT doesn't run on all archs. In any
case, starting a new thread for this would make most sense.

> And while you're in there, why not make the the URLs in .gitmodules relative
> since zx2c4 is already mirroring git?  That would mean that anyone else that
> mirrors cgit, git and luajit would have their mirrors used automatically.
> (Blender's main git repo does this in their .gitmodules file.)  Not really a
> big deal, just a suggestion.

That's a nice idea. I'll consider updating the git repo in modules to
point to my mirror. I don't see much technical benefit of this, other
than not relying on anyone else's infra to build cgit, which is a
plus.