List for cgit developers and users
 help / color / mirror / Atom feed
* [PATCH] native inline gravatar
@ 2018-07-01  3:20 andy
  2018-07-03 23:49 ` Jason
  0 siblings, 1 reply; 8+ messages in thread
From: andy @ 2018-07-01  3:20 UTC (permalink / raw)


This adds native inline gravatar images where cgit
already produces names and emails.  Its server
load is minimal compared to using an external filter.
By using a simple caching scheme, it reduces the
number of md5 digest computations needed by reusing
the last result if the email is the same as is
often the case with patch series.

Set

gravatar=1

in config to enable it.

At the time of writing, libravatar is shutting down in
2 months, so it's not targeted.

https://blog.libravatar.org/posts/Libravatar.org_is_shutting_down_on_2018-09-01/

It duplicates the css and style used on the cgit site.

Generation of the log view normally involves 50 x invocations
of the email-filter, even for lua that is expensive if your site
is busy.

This implementation stashes the last email + md5 result it
processed, on the basis patches often come in a series from the
same author.  If so, it can skip the md5 compuatation.  On
the first page of local cgit tree this reduces the number of md5s
in the 50-entry log list to 12, without needing a more complex cache.

Signed-off-by: Andy Green <andy at warmcat.com>
---
 cgit.c       |    2 
 cgit.css     |   30 ++++++
 cgit.h       |    4 +
 cgit.mk      |    1 
 cgitrc.5.txt |    6 +
 md5.c        |  291 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 shared.c     |   54 +++++++++++
 ui-commit.c  |    4 -
 ui-log.c     |    2 
 ui-refs.c    |    6 +
 ui-tag.c     |    2 
 11 files changed, 395 insertions(+), 7 deletions(-)
 create mode 100644 md5.c

diff --git a/cgit.c b/cgit.c
index 2ae8895..daf0652 100644
--- a/cgit.c
+++ b/cgit.c
@@ -172,6 +172,8 @@ static void config_cb(const char *name, const char *value)
 		ctx.cfg.virtual_root = ensure_end(value, '/');
 	else if (!strcmp(name, "noplainemail"))
 		ctx.cfg.noplainemail = atoi(value);
+	else if (!strcmp(name, "gravatar"))
+		ctx.cfg.gravatar = atoi(value);
 	else if (!strcmp(name, "noheader"))
 		ctx.cfg.noheader = atoi(value);
 	else if (!strcmp(name, "snapshots"))
diff --git a/cgit.css b/cgit.css
index 2035a64..cc9020d 100644
--- a/cgit.css
+++ b/cgit.css
@@ -983,3 +983,33 @@ div#cgit table.ssdiff td.space {
 div#cgit table.ssdiff td.space div {
 	min-height: 3em;
 }
+
+div#cgit span.libravatar img.onhover {
+        display: none;
+        border: 1px solid gray;
+        padding: 0px;
+        -webkit-border-radius: 4px;
+        -moz-border-radius: 4px;
+        border-radius: 4px;
+        width: 128px;
+        height: 128px;
+}
+
+div#cgit span.libravatar img.inline {
+        -webkit-border-radius: 3px;
+        -moz-border-radius: 3px;
+        border-radius: 3px;
+        width: 13px;
+        height: 13px;
+        margin-right: 0.2em;
+        opacity: 0.6;
+}
+
+div#cgit span.libravatar:hover > img.onhover {
+        display: block;
+        position: absolute;
+        margin-left: 1.5em;
+        background-color: #eeeeee;
+        box-shadow: 2px 2px 7px rgba(100,100,100,0.75);
+}
+
diff --git a/cgit.h b/cgit.h
index 093abb3..1d5b7d6 100644
--- a/cgit.h
+++ b/cgit.h
@@ -242,6 +242,7 @@ struct cgit_config {
 	int enable_html_serving;
 	int enable_tree_linenumbers;
 	int enable_git_config;
+	int gravatar;
 	int local_time;
 	int max_atom_items;
 	int max_repo_count;
@@ -398,4 +399,7 @@ extern char *expand_macros(const char *txt);
 extern char *get_mimetype_for_filename(const char *filename);
 extern struct cgit_filter *get_render_for_filename(const char *filename);
 
+extern int cgit_md5( const unsigned char *input, size_t ilen,
+              unsigned char output[16] );
+
 #endif /* CGIT_H */
diff --git a/cgit.mk b/cgit.mk
index 3fcc1ca..b3195f4 100644
--- a/cgit.mk
+++ b/cgit.mk
@@ -73,6 +73,7 @@ CGIT_OBJ_NAMES += cmd.o
 CGIT_OBJ_NAMES += configfile.o
 CGIT_OBJ_NAMES += filter.o
 CGIT_OBJ_NAMES += html.o
+CGIT_OBJ_NAMES += md5.o
 CGIT_OBJ_NAMES += parsing.o
 CGIT_OBJ_NAMES += scan-tree.o
 CGIT_OBJ_NAMES += shared.o
diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 39f50c7..555559e 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -226,6 +226,12 @@ footer::
 	verbatim at the bottom of all pages (i.e. it replaces the standard
 	"generated by..." message. Default value: none.
 
+gravatar::
+	If set to nonzero, enables the native generation of gravatar images
+	inline with names and emails.  This should not be set if you are
+	using "email-filter" to similarly produce the images.  However this
+	option is cheaper serverside than using an external script.
+
 head-include::
 	The content of the file specified with this option will be included
 	verbatim in the html HEAD section on all pages. Default value: none.
diff --git a/md5.c b/md5.c
new file mode 100644
index 0000000..aa57660
--- /dev/null
+++ b/md5.c
@@ -0,0 +1,291 @@
+/*
+ *  Copyright (C) 2006-2015, ARM Limited, All Rights Reserved
+ *  SPDX-License-Identifier: Apache-2.0
+ *
+ *  Licensed under the Apache License, Version 2.0 (the "License"); you may
+ *  not use this file except in compliance with the License.
+ *  You may obtain a copy of the License at
+ *
+ *  http://www.apache.org/licenses/LICENSE-2.0
+ *
+ *  Unless required by applicable law or agreed to in writing, software
+ *  distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+ *  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ *  See the License for the specific language governing permissions and
+ *  limitations under the License.
+ */
+
+#include <stdint.h>
+#include <stddef.h>
+#include <stdint.h>
+
+typedef struct {
+    uint32_t total[2];          /*!< number of bytes processed  */
+    uint32_t state[4];          /*!< intermediate digest state  */
+    unsigned char buffer[64];   /*!< data block being processed */
+} mbedtls_md5_context;
+
+/*
+ *  The MD5 algorithm was designed by Ron Rivest in 1991.
+ *
+ *  http://www.ietf.org/rfc/rfc1321.txt
+ */
+
+#include <string.h>
+#include <stdio.h>
+
+#define GET_UINT32_LE(n,b,i)                            \
+{                                                       \
+    (n) = ( (uint32_t) (b)[(i)    ]       )             \
+        | ( (uint32_t) (b)[(i) + 1] <<  8 )             \
+        | ( (uint32_t) (b)[(i) + 2] << 16 )             \
+        | ( (uint32_t) (b)[(i) + 3] << 24 );            \
+}
+
+#define PUT_UINT32_LE(n,b,i)                                    \
+{                                                               \
+    (b)[(i)    ] = (unsigned char) ( ( (n)       ) & 0xFF );    \
+    (b)[(i) + 1] = (unsigned char) ( ( (n) >>  8 ) & 0xFF );    \
+    (b)[(i) + 2] = (unsigned char) ( ( (n) >> 16 ) & 0xFF );    \
+    (b)[(i) + 3] = (unsigned char) ( ( (n) >> 24 ) & 0xFF );    \
+}
+
+static int
+mbedtls_internal_md5_process( mbedtls_md5_context *ctx,
+                              const unsigned char data[64] )
+{
+    uint32_t X[16], A, B, C, D;
+
+    GET_UINT32_LE( X[ 0], data,  0 );
+    GET_UINT32_LE( X[ 1], data,  4 );
+    GET_UINT32_LE( X[ 2], data,  8 );
+    GET_UINT32_LE( X[ 3], data, 12 );
+    GET_UINT32_LE( X[ 4], data, 16 );
+    GET_UINT32_LE( X[ 5], data, 20 );
+    GET_UINT32_LE( X[ 6], data, 24 );
+    GET_UINT32_LE( X[ 7], data, 28 );
+    GET_UINT32_LE( X[ 8], data, 32 );
+    GET_UINT32_LE( X[ 9], data, 36 );
+    GET_UINT32_LE( X[10], data, 40 );
+    GET_UINT32_LE( X[11], data, 44 );
+    GET_UINT32_LE( X[12], data, 48 );
+    GET_UINT32_LE( X[13], data, 52 );
+    GET_UINT32_LE( X[14], data, 56 );
+    GET_UINT32_LE( X[15], data, 60 );
+
+#define S(x,n) ((x << n) | ((x & 0xFFFFFFFF) >> (32 - n)))
+
+#define P(a,b,c,d,k,s,t)                                \
+{                                                       \
+    a += F(b,c,d) + X[k] + t; a = S(a,s) + b;           \
+}
+
+    A = ctx->state[0];
+    B = ctx->state[1];
+    C = ctx->state[2];
+    D = ctx->state[3];
+
+#define F(x,y,z) (z ^ (x & (y ^ z)))
+
+    P( A, B, C, D,  0,  7, 0xD76AA478 );
+    P( D, A, B, C,  1, 12, 0xE8C7B756 );
+    P( C, D, A, B,  2, 17, 0x242070DB );
+    P( B, C, D, A,  3, 22, 0xC1BDCEEE );
+    P( A, B, C, D,  4,  7, 0xF57C0FAF );
+    P( D, A, B, C,  5, 12, 0x4787C62A );
+    P( C, D, A, B,  6, 17, 0xA8304613 );
+    P( B, C, D, A,  7, 22, 0xFD469501 );
+    P( A, B, C, D,  8,  7, 0x698098D8 );
+    P( D, A, B, C,  9, 12, 0x8B44F7AF );
+    P( C, D, A, B, 10, 17, 0xFFFF5BB1 );
+    P( B, C, D, A, 11, 22, 0x895CD7BE );
+    P( A, B, C, D, 12,  7, 0x6B901122 );
+    P( D, A, B, C, 13, 12, 0xFD987193 );
+    P( C, D, A, B, 14, 17, 0xA679438E );
+    P( B, C, D, A, 15, 22, 0x49B40821 );
+
+#undef F
+
+#define F(x,y,z) (y ^ (z & (x ^ y)))
+
+    P( A, B, C, D,  1,  5, 0xF61E2562 );
+    P( D, A, B, C,  6,  9, 0xC040B340 );
+    P( C, D, A, B, 11, 14, 0x265E5A51 );
+    P( B, C, D, A,  0, 20, 0xE9B6C7AA );
+    P( A, B, C, D,  5,  5, 0xD62F105D );
+    P( D, A, B, C, 10,  9, 0x02441453 );
+    P( C, D, A, B, 15, 14, 0xD8A1E681 );
+    P( B, C, D, A,  4, 20, 0xE7D3FBC8 );
+    P( A, B, C, D,  9,  5, 0x21E1CDE6 );
+    P( D, A, B, C, 14,  9, 0xC33707D6 );
+    P( C, D, A, B,  3, 14, 0xF4D50D87 );
+    P( B, C, D, A,  8, 20, 0x455A14ED );
+    P( A, B, C, D, 13,  5, 0xA9E3E905 );
+    P( D, A, B, C,  2,  9, 0xFCEFA3F8 );
+    P( C, D, A, B,  7, 14, 0x676F02D9 );
+    P( B, C, D, A, 12, 20, 0x8D2A4C8A );
+
+#undef F
+
+#define F(x,y,z) (x ^ y ^ z)
+
+    P( A, B, C, D,  5,  4, 0xFFFA3942 );
+    P( D, A, B, C,  8, 11, 0x8771F681 );
+    P( C, D, A, B, 11, 16, 0x6D9D6122 );
+    P( B, C, D, A, 14, 23, 0xFDE5380C );
+    P( A, B, C, D,  1,  4, 0xA4BEEA44 );
+    P( D, A, B, C,  4, 11, 0x4BDECFA9 );
+    P( C, D, A, B,  7, 16, 0xF6BB4B60 );
+    P( B, C, D, A, 10, 23, 0xBEBFBC70 );
+    P( A, B, C, D, 13,  4, 0x289B7EC6 );
+    P( D, A, B, C,  0, 11, 0xEAA127FA );
+    P( C, D, A, B,  3, 16, 0xD4EF3085 );
+    P( B, C, D, A,  6, 23, 0x04881D05 );
+    P( A, B, C, D,  9,  4, 0xD9D4D039 );
+    P( D, A, B, C, 12, 11, 0xE6DB99E5 );
+    P( C, D, A, B, 15, 16, 0x1FA27CF8 );
+    P( B, C, D, A,  2, 23, 0xC4AC5665 );
+
+#undef F
+
+#define F(x,y,z) (y ^ (x | ~z))
+
+    P( A, B, C, D,  0,  6, 0xF4292244 );
+    P( D, A, B, C,  7, 10, 0x432AFF97 );
+    P( C, D, A, B, 14, 15, 0xAB9423A7 );
+    P( B, C, D, A,  5, 21, 0xFC93A039 );
+    P( A, B, C, D, 12,  6, 0x655B59C3 );
+    P( D, A, B, C,  3, 10, 0x8F0CCC92 );
+    P( C, D, A, B, 10, 15, 0xFFEFF47D );
+    P( B, C, D, A,  1, 21, 0x85845DD1 );
+    P( A, B, C, D,  8,  6, 0x6FA87E4F );
+    P( D, A, B, C, 15, 10, 0xFE2CE6E0 );
+    P( C, D, A, B,  6, 15, 0xA3014314 );
+    P( B, C, D, A, 13, 21, 0x4E0811A1 );
+    P( A, B, C, D,  4,  6, 0xF7537E82 );
+    P( D, A, B, C, 11, 10, 0xBD3AF235 );
+    P( C, D, A, B,  2, 15, 0x2AD7D2BB );
+    P( B, C, D, A,  9, 21, 0xEB86D391 );
+
+#undef F
+
+    ctx->state[0] += A;
+    ctx->state[1] += B;
+    ctx->state[2] += C;
+    ctx->state[3] += D;
+
+    return( 0 );
+}
+
+static int
+mbedtls_md5_update_ret( mbedtls_md5_context *ctx,
+                            const unsigned char *input,
+                            size_t ilen )
+{
+    int ret;
+    size_t fill;
+    uint32_t left;
+
+    if( ilen == 0 )
+        return( 0 );
+
+    left = ctx->total[0] & 0x3F;
+    fill = 64 - left;
+
+    ctx->total[0] += (uint32_t) ilen;
+    ctx->total[0] &= 0xFFFFFFFF;
+
+    if( ctx->total[0] < (uint32_t) ilen )
+        ctx->total[1]++;
+
+    if( left && ilen >= fill )
+    {
+        memcpy( (void *) (ctx->buffer + left), input, fill );
+        if( ( ret = mbedtls_internal_md5_process( ctx, ctx->buffer ) ) != 0 )
+            return( ret );
+
+        input += fill;
+        ilen  -= fill;
+        left = 0;
+    }
+
+    while( ilen >= 64 )
+    {
+        if( ( ret = mbedtls_internal_md5_process( ctx, input ) ) != 0 )
+            return( ret );
+
+        input += 64;
+        ilen  -= 64;
+    }
+
+    if( ilen > 0 )
+    {
+        memcpy( (void *) (ctx->buffer + left), input, ilen );
+    }
+
+    return( 0 );
+}
+
+static const unsigned char md5_padding[64] =
+{
+ 0x80, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
+    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
+};
+
+static int
+mbedtls_md5_finish_ret( mbedtls_md5_context *ctx,
+                        unsigned char output[16] )
+{
+    int ret;
+    uint32_t last, padn;
+    uint32_t high, low;
+    unsigned char msglen[8];
+
+    high = ( ctx->total[0] >> 29 )
+         | ( ctx->total[1] <<  3 );
+    low  = ( ctx->total[0] <<  3 );
+
+    PUT_UINT32_LE( low,  msglen, 0 );
+    PUT_UINT32_LE( high, msglen, 4 );
+
+    last = ctx->total[0] & 0x3F;
+    padn = ( last < 56 ) ? ( 56 - last ) : ( 120 - last );
+
+    if( ( ret = mbedtls_md5_update_ret( ctx, md5_padding, padn ) ) != 0 )
+            return( ret );
+
+    if( ( ret = mbedtls_md5_update_ret( ctx, msglen, 8 ) ) != 0 )
+            return( ret );
+
+    PUT_UINT32_LE( ctx->state[0], output,  0 );
+    PUT_UINT32_LE( ctx->state[1], output,  4 );
+    PUT_UINT32_LE( ctx->state[2], output,  8 );
+    PUT_UINT32_LE( ctx->state[3], output, 12 );
+
+    return( 0 );
+}
+
+int cgit_md5( const unsigned char *input, size_t ilen,
+              unsigned char output[16] )
+{
+    int ret;
+    mbedtls_md5_context ctx;
+    memset( &ctx, 0, sizeof(ctx) );
+
+    ctx.state[0] = 0x67452301;
+    ctx.state[1] = 0xEFCDAB89;
+    ctx.state[2] = 0x98BADCFE;
+    ctx.state[3] = 0x10325476;
+
+    if( ( ret = mbedtls_md5_update_ret( &ctx, input, ilen ) ) != 0 )
+        goto exit;
+
+    if( ( ret = mbedtls_md5_finish_ret( &ctx, output ) ) != 0 )
+        goto exit;
+
+exit:
+
+    return( ret );
+}
diff --git a/shared.c b/shared.c
index 3d7296d..68c00d3 100644
--- a/shared.c
+++ b/shared.c
@@ -7,9 +7,12 @@
  */
 
 #include "cgit.h"
+#include "html.h"
 
 struct cgit_repolist cgit_repolist;
 struct cgit_context ctx;
+char last_email[128];
+char last_md5_hex[(2 * 16) + 1];
 
 int chk_zero(int result, char *msg)
 {
@@ -595,3 +598,54 @@ struct cgit_filter *get_render_for_filename(const char *filename)
 
 	return NULL;
 }
+
+static const char *hex = "0123456789abcdef";
+
+void html_email(const char *name, const char *email)
+{
+	unsigned char *md5_in = (unsigned char *)email,
+		      md5_out[16];
+	char *p;
+	int len, n;
+
+	if (!ctx.cfg.gravatar) {
+		html_txt(email);
+
+		return;
+	}
+
+	/* patches often come in series... */
+	if (!strcmp(email, last_email))
+		goto emit;
+
+	if (*email == '<')
+		md5_in++;
+
+	len = strlen((char *)md5_in);
+
+	if (md5_in[len - 1] == '>')
+		len --;
+
+	cgit_md5(md5_in, len, md5_out);
+	p = last_md5_hex;
+	for (n = 0; n < sizeof(md5_out); n++) {
+		*p++ = hex[md5_out[n] >> 4];
+		*p++ = hex[md5_out[n] & 0xf];
+	}
+	*p = '\0';
+
+	strncpy(last_email, email, sizeof(last_email) - 1);
+	last_email[sizeof(last_email) - 1] = '\0';
+
+emit:
+	htmlf("<span class='libravatar'>"
+	      "<img class='inline' src='//www.gravatar.com/avatar/%s"
+	      "?s=13&amp;d=retro' width='13' height='13' alt='Gravatar' />"
+	      "<img class='onhover' src='//www.gravatar.com/avatar/%s"
+	      "?s=128&amp;d=retro'/>", last_md5_hex, last_md5_hex);
+	if (name)
+		html_txt(name);
+	else
+		html_txt(email);
+}
+
diff --git a/ui-commit.c b/ui-commit.c
index 995cb93..30655b9 100644
--- a/ui-commit.c
+++ b/ui-commit.c
@@ -51,7 +51,7 @@ void cgit_print_commit(char *hex, const char *prefix)
 	html_txt(info->author);
 	if (!ctx.cfg.noplainemail) {
 		html(" ");
-		html_txt(info->author_email);
+		html_email(NULL, info->author_email);
 	}
 	cgit_close_filter(ctx.repo->email_filter);
 	html("</td><td class='right'>");
@@ -63,7 +63,7 @@ void cgit_print_commit(char *hex, const char *prefix)
 	html_txt(info->committer);
 	if (!ctx.cfg.noplainemail) {
 		html(" ");
-		html_txt(info->committer_email);
+		html_email(NULL, info->committer_email);
 	}
 	cgit_close_filter(ctx.repo->email_filter);
 	html("</td><td class='right'>");
diff --git a/ui-log.c b/ui-log.c
index d696e20..37f603d 100644
--- a/ui-log.c
+++ b/ui-log.c
@@ -242,7 +242,7 @@ static void print_commit(struct commit *commit, struct rev_info *revs)
 	show_commit_decorations(commit);
 	html("</td><td>");
 	cgit_open_filter(ctx.repo->email_filter, info->author_email, "log");
-	html_txt(info->author);
+	html_email(info->author, info->author_email);
 	cgit_close_filter(ctx.repo->email_filter);
 
 	if (revs->graph) {
diff --git a/ui-refs.c b/ui-refs.c
index 2ec3858..2d7b625 100644
--- a/ui-refs.c
+++ b/ui-refs.c
@@ -70,7 +70,7 @@ static int print_branch(struct refinfo *ref)
 		cgit_commit_link(info->subject, NULL, NULL, name, NULL, NULL);
 		html("</td><td>");
 		cgit_open_filter(ctx.repo->email_filter, info->author_email, "refs");
-		html_txt(info->author);
+		html_email(info->author, info->author_email);
 		cgit_close_filter(ctx.repo->email_filter);
 		html("</td><td colspan='2'>");
 		cgit_print_age(info->committer_date, info->committer_tz, -1);
@@ -116,12 +116,12 @@ static int print_tag(struct refinfo *ref)
 	if (info) {
 		if (info->tagger) {
 			cgit_open_filter(ctx.repo->email_filter, info->tagger_email, "refs");
-			html_txt(info->tagger);
+			html_email(info->tagger, info->tagger_email);
 			cgit_close_filter(ctx.repo->email_filter);
 		}
 	} else if (ref->object->type == OBJ_COMMIT) {
 		cgit_open_filter(ctx.repo->email_filter, ref->commit->author_email, "refs");
-		html_txt(ref->commit->author);
+		html_email(ref->commit->author, ref->commit->author_email);
 		cgit_close_filter(ctx.repo->email_filter);
 	}
 	html("</td><td colspan='2'>");
diff --git a/ui-tag.c b/ui-tag.c
index 2c96c37..387fadf 100644
--- a/ui-tag.c
+++ b/ui-tag.c
@@ -86,7 +86,7 @@ void cgit_print_tag(char *revname)
 			html_txt(info->tagger);
 			if (info->tagger_email && !ctx.cfg.noplainemail) {
 				html(" ");
-				html_txt(info->tagger_email);
+				html_email(NULL, info->tagger_email);
 			}
 			cgit_close_filter(ctx.repo->email_filter);
 			html("</td></tr>\n");



^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-01  3:20 [PATCH] native inline gravatar andy
@ 2018-07-03 23:49 ` Jason
  2018-07-04  0:00   ` andy
  0 siblings, 1 reply; 8+ messages in thread
From: Jason @ 2018-07-03 23:49 UTC (permalink / raw)


Sorry, but not a chance something like this can be accepted. This is
exactly the reason we put the time into making the Lua scripting
support. Is your reason for implementing this C performance? In that
case, could you send some performance metrics and some details about
your performance requirements on a given system? Considering the Lua
crypto module links against OpenSSL, I'm pretty sure that its hashing
performance will be considerably better the slow reference
implementation in this patch [1].

[1] https://github.com/openssl/openssl/blob/master/crypto/md5/asm/md5-x86_64.pl


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-03 23:49 ` Jason
@ 2018-07-04  0:00   ` andy
  2018-07-04  0:14     ` Jason
  0 siblings, 1 reply; 8+ messages in thread
From: andy @ 2018-07-04  0:00 UTC (permalink / raw)




On 07/04/2018 07:49 AM, Jason A. Donenfeld wrote:
> Sorry, but not a chance something like this can be accepted. This is
> exactly the reason we put the time into making the Lua scripting

No worries.

> support. Is your reason for implementing this C performance? In that
> case, could you send some performance metrics and some details about

The main idea this is part of is it should be possible to get rid of all 
the filters and still get the common functionality.

Currently this

https://warmcat.com/git/cgit/log

https://warmcat.com/git/cgit/tree/
https://warmcat.com/git/cgit/tree/shared.c


doesn't use any filters for syntax highlight and markdown render, it's 
all done in clientside JS.  The gravatar is done using this patch.

This of course is a big performance win on the server since no 
subprocesses spawned at all.  Even on large files on an Android tablet, 
it's quick.

Even line numbers for /tree/ are done clientside, not generated and sent 
from the server any more.  URLs are corrected in the markdown as in the 
previous series, but done clientside.

> your performance requirements on a given system? Considering the Lua
> crypto module links against OpenSSL, I'm pretty sure that its hashing
> performance will be considerably better the slow reference
> implementation in this patch [1].
> 
> [1] https://github.com/openssl/openssl/blob/master/crypto/md5/asm/md5-x86_64.pl

OpenSSL is accelerated for some things on some architectures, it's true. 
  If that was the actual problem that can be ported as a special case of 
x86_64.

-Andy


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-04  0:00   ` andy
@ 2018-07-04  0:14     ` Jason
  2018-07-04  0:28       ` andy
  0 siblings, 1 reply; 8+ messages in thread
From: Jason @ 2018-07-04  0:14 UTC (permalink / raw)


On Wed, Jul 4, 2018 at 2:01 AM Andy Green <andy at warmcat.com> wrote:
> doesn't use any filters for syntax highlight and markdown render, it's
> all done in clientside JS.  The gravatar is done using this patch.

If you're into doing things clientside, you could do gravatar
clientside too of course...

> This of course is a big performance win on the server since no
> subprocesses spawned at all.  Even on large files on an Android tablet,
> it's quick.

It's worth noting that Lua does not spawn sub processes, so if you do
rendering from Lua, you're in good shape.

However, there are indeed advantages -- such as bandwidth -- to doing
this all clientside.

> OpenSSL is accelerated for some things on some architectures, it's true.
>   If that was the actual problem that can be ported as a special case of
> x86_64.

We're not going to start copy and pasting OpenSSL into cgit to
superoptimize our own MD5 implementation...


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-04  0:14     ` Jason
@ 2018-07-04  0:28       ` andy
  2018-07-04  0:34         ` Jason
  0 siblings, 1 reply; 8+ messages in thread
From: andy @ 2018-07-04  0:28 UTC (permalink / raw)




On 07/04/2018 08:14 AM, Jason A. Donenfeld wrote:
> On Wed, Jul 4, 2018 at 2:01 AM Andy Green <andy at warmcat.com> wrote:
>> doesn't use any filters for syntax highlight and markdown render, it's
>> all done in clientside JS.  The gravatar is done using this patch.
> 
> If you're into doing things clientside, you could do gravatar
> clientside too of course...

I looked at it, but there's no md5 api in JS... you have to do it by 
hand in JS.  It's possible but I think it might be slow if it hits a 
page of 50 different email addresses.

>> This of course is a big performance win on the server since no
>> subprocesses spawned at all.  Even on large files on an Android tablet,
>> it's quick.
> 
> It's worth noting that Lua does not spawn sub processes, so if you do
> rendering from Lua, you're in good shape.

Is it supported to do the job of pygments in Lua?

What it does today is spawn the python runtime, start that up etc.

> However, there are indeed advantages -- such as bandwidth -- to doing
> this all clientside.

I know... bandwidth and server compute time.

Also there's no reason (other than not enough care taken with frees atm) 
cgit should only be buildable as a cgi starting up its own process each 
time.

>> OpenSSL is accelerated for some things on some architectures, it's true.
>>    If that was the actual problem that can be ported as a special case of
>> x86_64.
> 
> We're not going to start copy and pasting OpenSSL into cgit to
> superoptimize our own MD5 implementation...

It could optionally link against OpenSSL then :-)  It can optionally 
link against lua.

If you're not interested in going where this stuff is going, you can 
save us both a lot of time by just saying it now, and I'll stop trying 
to sell it here.

-Andy


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-04  0:28       ` andy
@ 2018-07-04  0:34         ` Jason
  2018-07-04  0:44           ` andy
  0 siblings, 1 reply; 8+ messages in thread
From: Jason @ 2018-07-04  0:34 UTC (permalink / raw)


On Wed, Jul 4, 2018 at 2:28 AM Andy Green <andy at warmcat.com> wrote:
> I looked at it, but there's no md5 api in JS... you have to do it by
> hand in JS.  It's possible but I think it might be slow if it hits a
> page of 50 different email addresses.

MD5 is really fast and fits easily into javascript's integer types.
There should be reasonably fast implementations available.
Alternatively, you could generate asm.js and web assembly code using
something like emscripten.

> Is it supported to do the job of pygments in Lua?

You can do any type of text processing you want from Lua. If you'd
like to contribute a Lua highlighter, that'd be quite nice.

> What it does today is spawn the python runtime, start that up etc.

Have you measured latency issues with this?

> I know... bandwidth and server compute time.
>
> Also there's no reason (other than not enough care taken with frees atm)
> cgit should only be buildable as a cgi starting up its own process each
> time.

You're talking about FastCGI support? I have no idea how this is
related to the current line of discussion -- perhaps just by being
another gripe or something -- but this is actually something we've
looked at in depth. (See the list archives.) Our last conclusion from
examining it was that so much of libgit is not re-entrant, and so we'd
need to move to something like libgit2 for this to be feasible. Too
many globals, etc.

> It could optionally link against OpenSSL then :-)  It can optionally
> link against lua.
>
> If you're not interested in going where this stuff is going, you can
> save us both a lot of time by just saying it now, and I'll stop trying
> to sell it here.

I made that clear in my very first message to you:

"Sorry, but not a chance something like this can be accepted."


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-04  0:34         ` Jason
@ 2018-07-04  0:44           ` andy
  2018-07-04  1:05             ` Jason
  0 siblings, 1 reply; 8+ messages in thread
From: andy @ 2018-07-04  0:44 UTC (permalink / raw)




On 07/04/2018 08:34 AM, Jason A. Donenfeld wrote:
> On Wed, Jul 4, 2018 at 2:28 AM Andy Green <andy at warmcat.com> wrote:
>> I looked at it, but there's no md5 api in JS... you have to do it by
>> hand in JS.  It's possible but I think it might be slow if it hits a
>> page of 50 different email addresses.
> 
> MD5 is really fast and fits easily into javascript's integer types.
> There should be reasonably fast implementations available.
> Alternatively, you could generate asm.js and web assembly code using
> something like emscripten.
> 
>> Is it supported to do the job of pygments in Lua?
> 
> You can do any type of text processing you want from Lua. If you'd
> like to contribute a Lua highlighter, that'd be quite nice.
> 
>> What it does today is spawn the python runtime, start that up etc.
> 
> Have you measured latency issues with this?

Nope, but it's obviously more efficient to not be spawning python apps.

Don't you think?

>> I know... bandwidth and server compute time.
>>
>> Also there's no reason (other than not enough care taken with frees atm)
>> cgit should only be buildable as a cgi starting up its own process each
>> time.
> 
> You're talking about FastCGI support? I have no idea how this is
> related to the current line of discussion -- perhaps just by being
> another gripe or something -- but this is actually something we've

What was the first gripe?  I am providing patches for all these things 
not moaning about them.

I mention it because starting up cgit as a process each time is also 
overhead, I think it would be interesting also to look at eliminating that.

> looked at in depth. (See the list archives.) Our last conclusion from
> examining it was that so much of libgit is not re-entrant, and so we'd
> need to move to something like libgit2 for this to be feasible. Too
> many globals, etc.

Let me guess, not enough people around working on this project, for some 
reason, to move to libgit2.

>> It could optionally link against OpenSSL then :-)  It can optionally
>> link against lua.
>>
>> If you're not interested in going where this stuff is going, you can
>> save us both a lot of time by just saying it now, and I'll stop trying
>> to sell it here.
> 
> I made that clear in my very first message to you:
> 
> "Sorry, but not a chance something like this can be accepted."

If that's your feeling about the whole stack of clientside stuff, I 
understand you don't want to cooperate on it.

No worries I'll leave you to it then.

-Andy


^ permalink raw reply	[flat|nested] 8+ messages in thread

* [PATCH] native inline gravatar
  2018-07-04  0:44           ` andy
@ 2018-07-04  1:05             ` Jason
  0 siblings, 0 replies; 8+ messages in thread
From: Jason @ 2018-07-04  1:05 UTC (permalink / raw)


On Wed, Jul 4, 2018 at 2:44 AM Andy Green <andy at warmcat.com> wrote:
> > looked at in depth. (See the list archives.) Our last conclusion from
> > examining it was that so much of libgit is not re-entrant, and so we'd
> > need to move to something like libgit2 for this to be feasible. Too
> > many globals, etc.
>
> Let me guess, not enough people around working on this project, for some
> reason, to move to libgit2.

No, actually. The reason is that we'd like to stay close to what
git.git is doing.

> If that's your feeling about the whole stack of clientside stuff, I
> understand you don't want to cooperate on it.

I think you misinterpreted me. I think the clientside work you're
doing is interesting and worth considering. I was speaking to somebody
earlier today who is actually working on a JSON API for cgit, so that
_all_ rendering can be done clientside.

What I'm specifically rejecting in my comment here is adding gravatar
support, some web service, in C directly to the core of cgit, when the
lua support has no demonstrable problems.


^ permalink raw reply	[flat|nested] 8+ messages in thread

end of thread, other threads:[~2018-07-04  1:05 UTC | newest]

Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-01  3:20 [PATCH] native inline gravatar andy
2018-07-03 23:49 ` Jason
2018-07-04  0:00   ` andy
2018-07-04  0:14     ` Jason
2018-07-04  0:28       ` andy
2018-07-04  0:34         ` Jason
2018-07-04  0:44           ` andy
2018-07-04  1:05             ` Jason

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).