[PATCH 00/17] Add support for namespaced repositories

[PATCH 00/17] Add support for namespaced repositories

[richard.maw]

These patches are to support viewing the namespaces of a git repository
as independent git repositories.

You might want to do this for projects that share a lot of history
(e.g. systemd, udev and eudev), or to permit a user to "fork" a repository
so users on the same server can cheaply copy a large repository
to follow a GitHub-like pull-request process.

Namespace support in git isn't perfect,
only supporting enough operations for the recieve-pack and upload-pack commands.

This is enough to push to and pull from,
but not for viewing a namespaced repository's contents,
so this requires more work than just setting GIT_NAMESPACE.

This has been tested with browsing repositores,
cloning them over the dumb-http protocol
and downloading a snapshot of a namespaced tag.

As well as adding "repo.namespace",
this patch series adds "repo.agefile"
so that namespaced repositories can have their own age files,
and modifications to the contributed agefile post-receive hook.

Some ideas of other potentially useful changes related to namespaces include:

1.  Being able to set a default namespace globally.

    You might decide to put eveything in a namespace by default
    so that you can have additional namespaces alongside
    rather than inside the default namespace.

2.  Displaying git notes for all subnamespaces

    I had to make some changes to avoid displaying notes of the root namespace
    in other namespaces.

    I made it only display the commit notes for the current namespace,
    but since you can see all commits in subnamespaces,
    it might make sense to include those notes as well.

3.  Automatic discovery of namespaces with scan-path

    For a namespace to be useful it needs a HEAD symbolic ref.

    From what I can tell this is always unpacked on-disk,
    so it would be possible to extend the scan-path logic to look for HEADs.

4.  Smarter handling of README files.

    The repo.readme option lets you specify a readme as ref:file/path,
    so you could show the readme by prepending the namespace's base ref,
    but it might be better to have CGit do the prepending.

Richard Maw (17):
  Fix qry.head leak on error
  Fix archive generation
  Add a wrapper for get_sha1 called cgit_get_sha1
  Parse repo.namespace
  Print out parsed namespace on request
  Set GIT_NAMESPACE when repo.namespace is provided
  Look up refs in namespace with cgit_get_sha1
  Guess the default branch based on current namespace
  Add cgit_for_each_namespaced_ref_in helper
  Find the default branch based on the contents of the namespace
  Only display refs in current namespace
  Add namespace support to dumb-clone
  Display notes from namespace
  Add documentation for repo.namespace
  Allow agefile to be set per-repository
  Update contrib script to update agefiles per namespace
  Add documentation for repo.agefile

 cgit.c                             | 71 ++++++++++++++++++++++++++++++++------
 cgit.h                             |  6 ++++
 cgitrc.5.txt                       | 46 ++++++++++++++++++++++++
 contrib/hooks/post-receive.agefile | 15 +++++++-
 shared.c                           | 60 ++++++++++++++++++++++++++++++++
 ui-blob.c                          |  6 ++--
 ui-clone.c                         | 23 +++++++++++-
 ui-commit.c                        |  2 +-
 ui-diff.c                          |  4 +--
 ui-log.c                           |  2 +-
 ui-patch.c                         |  4 +--
 ui-plain.c                         |  2 +-
 ui-refs.c                          |  6 ++--
 ui-repolist.c                      |  3 +-
 ui-shared.c                        |  2 +-
 ui-snapshot.c                      | 14 ++++----
 ui-tag.c                           |  2 +-
 ui-tree.c                          |  2 +-
 18 files changed, 233 insertions(+), 37 deletions(-)

-- 
2.9.0

[PATCH 01/17] Fix qry.head leak on error

[richard.maw]

This is run soon before exiting so it wasn't leaked for long.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/cgit.c b/cgit.c
index ab3fadb..9427c4a 100644
--- a/cgit.c
+++ b/cgit.c
@@ -616,11 +616,11 @@ static int prepare_repo_cmd(void)
 	}
 
 	if (get_sha1(ctx.qry.head, sha1)) {
-		char *tmp = xstrdup(ctx.qry.head);
-		ctx.qry.head = ctx.repo->defbranch;
+		char *old_head = ctx.qry.head;
+		ctx.qry.head = xstrdup(ctx.repo->defbranch);
 		cgit_print_error_page(404, "Not found",
-				"Invalid branch: %s", tmp);
-		free(tmp);
+				"Invalid branch: %s", old_head);
+		free(old_head);
 		return 1;
 	}
 	string_list_sort(&ctx.repo->submodules);
-- 
2.9.0

[PATCH 02/17] Fix archive generation

[richard.maw]

The get_ref_from_filename function is expected to return a sha1.
It didn't actually do this,
instead returning the ref that would under normal circumstances resolve to that.

Since we're going to resolve refs in a way that is namespace aware
we need to return the sha1 rather than the ref,
since the archive is created by libgit code that is not namespace aware,
and it would try to resolve the ref again.

This previously worked fine
because it would resolve the ref the same way both times.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 ui-snapshot.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/ui-snapshot.c b/ui-snapshot.c
index f68e877..c6c3656 100644
--- a/ui-snapshot.c
+++ b/ui-snapshot.c
@@ -174,10 +174,10 @@ static const char *get_ref_from_filename(const char *url, const char *filename,
 		goto out;
 
 	result = 0;
-	strbuf_release(&snapshot);
 
 out:
-	return result ? strbuf_detach(&snapshot, NULL) : NULL;
+	strbuf_release(&snapshot);
+	return result ? xstrdup(sha1_to_hex(sha1)) : NULL;
 }
 
 void cgit_print_snapshot(const char *head, const char *hex,
-- 
2.9.0

[PATCH 03/17] Add a wrapper for get_sha1 called cgit_get_sha1

[richard.maw]

This will later be changed to include namespace resolution,
but the call sites are changed now to keep the changes small.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c        |  2 +-
 cgit.h        |  2 ++
 shared.c      |  5 +++++
 ui-blob.c     |  6 +++---
 ui-commit.c   |  2 +-
 ui-diff.c     |  4 ++--
 ui-log.c      |  2 +-
 ui-patch.c    |  4 ++--
 ui-plain.c    |  2 +-
 ui-shared.c   |  2 +-
 ui-snapshot.c | 10 +++++-----
 ui-tag.c      |  2 +-
 ui-tree.c     |  2 +-
 13 files changed, 26 insertions(+), 19 deletions(-)

diff --git a/cgit.c b/cgit.c
index 9427c4a..94d452c 100644
--- a/cgit.c
+++ b/cgit.c
@@ -615,7 +615,7 @@ static int prepare_repo_cmd(void)
 		return 1;
 	}
 
-	if (get_sha1(ctx.qry.head, sha1)) {
+	if (cgit_get_sha1(ctx.qry.head, sha1)) {
 		char *old_head = ctx.qry.head;
 		ctx.qry.head = xstrdup(ctx.repo->defbranch);
 		cgit_print_error_page(404, "Not found",
diff --git a/cgit.h b/cgit.h
index 325432b..49f0557 100644
--- a/cgit.h
+++ b/cgit.h
@@ -391,4 +391,6 @@ extern char *expand_macros(const char *txt);
 
 extern char *get_mimetype_for_filename(const char *filename);
 
+extern int cgit_get_sha1(const char *name, unsigned char *sha1);
+
 #endif /* CGIT_H */
diff --git a/shared.c b/shared.c
index a63633b..d82c07b 100644
--- a/shared.c
+++ b/shared.c
@@ -601,3 +601,8 @@ char *get_mimetype_for_filename(const char *filename)
 	fclose(file);
 	return NULL;
 }
+
+int cgit_get_sha1(const char *name, unsigned char *sha1)
+{
+	return get_sha1(name, sha1);
+}
diff --git a/ui-blob.c b/ui-blob.c
index d388489..21d21ed 100644
--- a/ui-blob.c
+++ b/ui-blob.c
@@ -52,7 +52,7 @@ int cgit_ref_path_exists(const char *path, const char *ref, int file_only)
 		.file_only = file_only
 	};
 
-	if (get_sha1(ref, sha1))
+	if (cgit_get_sha1(ref, sha1))
 		return 0;
 	if (sha1_object_info(sha1, &size) != OBJ_COMMIT)
 		return 0;
@@ -82,7 +82,7 @@ int cgit_print_file(char *path, const char *head, int file_only)
 		.file_only = file_only
 	};
 
-	if (get_sha1(head, sha1))
+	if (cgit_get_sha1(head, sha1))
 		return -1;
 	type = sha1_object_info(sha1, &size);
 	if (type == OBJ_COMMIT) {
@@ -132,7 +132,7 @@ void cgit_print_blob(const char *hex, char *path, const char *head, int file_onl
 			return;
 		}
 	} else {
-		if (get_sha1(head, sha1)) {
+		if (cgit_get_sha1(head, sha1)) {
 			cgit_print_error_page(404, "Not found",
 					"Bad ref: %s", head);
 			return;
diff --git a/ui-commit.c b/ui-commit.c
index 099d294..5ba0791 100644
--- a/ui-commit.c
+++ b/ui-commit.c
@@ -26,7 +26,7 @@ void cgit_print_commit(char *hex, const char *prefix)
 	if (!hex)
 		hex = ctx.qry.head;
 
-	if (get_sha1(hex, sha1)) {
+	if (cgit_get_sha1(hex, sha1)) {
 		cgit_print_error_page(400, "Bad request",
 				"Bad object id: %s", hex);
 		return;
diff --git a/ui-diff.c b/ui-diff.c
index edee793..9256d12 100644
--- a/ui-diff.c
+++ b/ui-diff.c
@@ -402,7 +402,7 @@ void cgit_print_diff(const char *new_rev, const char *old_rev,
 
 	if (!new_rev)
 		new_rev = ctx.qry.head;
-	if (get_sha1(new_rev, new_rev_sha1)) {
+	if (cgit_get_sha1(new_rev, new_rev_sha1)) {
 		cgit_print_error_page(404, "Not found",
 			"Bad object name: %s", new_rev);
 		return;
@@ -416,7 +416,7 @@ void cgit_print_diff(const char *new_rev, const char *old_rev,
 	new_tree_sha1 = commit->tree->object.oid.hash;
 
 	if (old_rev) {
-		if (get_sha1(old_rev, old_rev_sha1)) {
+		if (cgit_get_sha1(old_rev, old_rev_sha1)) {
 			cgit_print_error_page(404, "Not found",
 				"Bad object name: %s", old_rev);
 			return;
diff --git a/ui-log.c b/ui-log.c
index d6d94f6..d110988 100644
--- a/ui-log.c
+++ b/ui-log.c
@@ -324,7 +324,7 @@ static const char *disambiguate_ref(const char *ref, int *must_free_result)
 	struct strbuf longref = STRBUF_INIT;
 
 	strbuf_addf(&longref, "refs/heads/%s", ref);
-	if (get_sha1(longref.buf, oid.hash) == 0) {
+	if (cgit_get_sha1(longref.buf, oid.hash) == 0) {
 		*must_free_result = 1;
 		return strbuf_detach(&longref, NULL);
 	}
diff --git a/ui-patch.c b/ui-patch.c
index 4c051e8..ff09a8e 100644
--- a/ui-patch.c
+++ b/ui-patch.c
@@ -24,7 +24,7 @@ void cgit_print_patch(const char *new_rev, const char *old_rev,
 	if (!new_rev)
 		new_rev = ctx.qry.head;
 
-	if (get_sha1(new_rev, new_rev_sha1)) {
+	if (cgit_get_sha1(new_rev, new_rev_sha1)) {
 		cgit_print_error_page(404, "Not found",
 				"Bad object id: %s", new_rev);
 		return;
@@ -37,7 +37,7 @@ void cgit_print_patch(const char *new_rev, const char *old_rev,
 	}
 
 	if (old_rev) {
-		if (get_sha1(old_rev, old_rev_sha1)) {
+		if (cgit_get_sha1(old_rev, old_rev_sha1)) {
 			cgit_print_error_page(404, "Not found",
 					"Bad object id: %s", old_rev);
 			return;
diff --git a/ui-plain.c b/ui-plain.c
index 97cf639..88dd1a8 100644
--- a/ui-plain.c
+++ b/ui-plain.c
@@ -181,7 +181,7 @@ void cgit_print_plain(void)
 	if (!rev)
 		rev = ctx.qry.head;
 
-	if (get_sha1(rev, sha1)) {
+	if (cgit_get_sha1(rev, sha1)) {
 		cgit_print_error_page(404, "Not found", "Not found");
 		return;
 	}
diff --git a/ui-shared.c b/ui-shared.c
index 562fa0e..4e1c2ba 100644
--- a/ui-shared.c
+++ b/ui-shared.c
@@ -1077,7 +1077,7 @@ void cgit_print_snapshot_links(const char *repo, const char *head,
 	size_t prefixlen;
 	unsigned char sha1[20];
 
-	if (get_sha1(fmt("refs/tags/%s", hex), sha1) == 0 &&
+	if (cgit_get_sha1(fmt("refs/tags/%s", hex), sha1) == 0 &&
 	    (hex[0] == 'v' || hex[0] == 'V') && isdigit(hex[1]))
 		hex++;
 	strbuf_addf(&filename, "%s-%s", cgit_repobasename(repo), hex);
diff --git a/ui-snapshot.c b/ui-snapshot.c
index c6c3656..93af28e 100644
--- a/ui-snapshot.c
+++ b/ui-snapshot.c
@@ -111,7 +111,7 @@ static int make_snapshot(const struct cgit_snapshot_format *format,
 {
 	unsigned char sha1[20];
 
-	if (get_sha1(hex, sha1)) {
+	if (cgit_get_sha1(hex, sha1)) {
 		cgit_print_error_page(404, "Not found",
 				"Bad object id: %s", hex);
 		return 1;
@@ -150,7 +150,7 @@ static const char *get_ref_from_filename(const char *url, const char *filename,
 	strbuf_addstr(&snapshot, filename);
 	strbuf_setlen(&snapshot, snapshot.len - strlen(format->suffix));
 
-	if (get_sha1(snapshot.buf, sha1) == 0)
+	if (cgit_get_sha1(snapshot.buf, sha1) == 0)
 		goto out;
 
 	reponame = cgit_repobasename(url);
@@ -162,15 +162,15 @@ static const char *get_ref_from_filename(const char *url, const char *filename,
 		strbuf_splice(&snapshot, 0, new_start - snapshot.buf, "", 0);
 	}
 
-	if (get_sha1(snapshot.buf, sha1) == 0)
+	if (cgit_get_sha1(snapshot.buf, sha1) == 0)
 		goto out;
 
 	strbuf_insert(&snapshot, 0, "v", 1);
-	if (get_sha1(snapshot.buf, sha1) == 0)
+	if (cgit_get_sha1(snapshot.buf, sha1) == 0)
 		goto out;
 
 	strbuf_splice(&snapshot, 0, 1, "V", 1);
-	if (get_sha1(snapshot.buf, sha1) == 0)
+	if (cgit_get_sha1(snapshot.buf, sha1) == 0)
 		goto out;
 
 	result = 0;
diff --git a/ui-tag.c b/ui-tag.c
index 6b838cb..7372770 100644
--- a/ui-tag.c
+++ b/ui-tag.c
@@ -51,7 +51,7 @@ void cgit_print_tag(char *revname)
 		revname = ctx.qry.head;
 
 	strbuf_addf(&fullref, "refs/tags/%s", revname);
-	if (get_sha1(fullref.buf, sha1)) {
+	if (cgit_get_sha1(fullref.buf, sha1)) {
 		cgit_print_error_page(404, "Not found",
 			"Bad tag reference: %s", revname);
 		goto cleanup;
diff --git a/ui-tree.c b/ui-tree.c
index 120066c..ae2acce 100644
--- a/ui-tree.c
+++ b/ui-tree.c
@@ -306,7 +306,7 @@ void cgit_print_tree(const char *rev, char *path)
 	if (!rev)
 		rev = ctx.qry.head;
 
-	if (get_sha1(rev, sha1)) {
+	if (cgit_get_sha1(rev, sha1)) {
 		cgit_print_error_page(404, "Not found",
 			"Invalid revision name: %s", rev);
 		return;
-- 
2.9.0

[PATCH 04/17] Parse repo.namespace

[richard.maw]

This contains the unexpanded name of the namespace
rather than the base ref of the namespace,
since the git namespace mechanism works by setting GIT_NAMESPACE
and on the first call to get_git_namespace() it gets expanded.

We need to save this for a later call to prepare_repo_cmd,
rather than trying to process it here,
since we can only do it once,
and we have other uses for the unexpanded name.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 2 ++
 cgit.h | 1 +
 2 files changed, 3 insertions(+)

diff --git a/cgit.c b/cgit.c
index 94d452c..e525264 100644
--- a/cgit.c
+++ b/cgit.c
@@ -45,6 +45,8 @@ static void repo_config(struct cgit_repo *repo, const char *name, const char *va
 		repo->homepage = xstrdup(value);
 	else if (!strcmp(name, "defbranch"))
 		repo->defbranch = xstrdup(value);
+	else if (!strcmp(name, "namespace"))
+		repo->namespace = xstrdup(value);
 	else if (!strcmp(name, "snapshots"))
 		repo->snapshots = ctx.cfg.snapshots & cgit_parse_snapshots_mask(value);
 	else if (!strcmp(name, "enable-commit-graph"))
diff --git a/cgit.h b/cgit.h
index 49f0557..93de0ea 100644
--- a/cgit.h
+++ b/cgit.h
@@ -87,6 +87,7 @@ struct cgit_repo {
 	struct string_list readme;
 	char *section;
 	char *clone_url;
+	char *namespace;
 	char *logo;
 	char *logo_link;
 	int snapshots;
-- 
2.9.0

[PATCH 05/17] Print out parsed namespace on request

[richard.maw]

This is not strictly necessary,
as we do not have any way to generate namespace entries from a scan-path,
but I'd rather not leave this as a surprise
to someone who comes up with a good namespace discovery mechanism.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/cgit.c b/cgit.c
index e525264..9b2733b 100644
--- a/cgit.c
+++ b/cgit.c
@@ -778,6 +778,8 @@ static void print_repo(FILE *f, struct cgit_repo *repo)
 	fprintf(f, "repo.url=%s\n", repo->url);
 	fprintf(f, "repo.name=%s\n", repo->name);
 	fprintf(f, "repo.path=%s\n", repo->path);
+	if (repo->namespace)
+		fprintf(f, "repo.namespace=%s\n", repo->namespace);
 	if (repo->owner)
 		fprintf(f, "repo.owner=%s\n", repo->owner);
 	if (repo->desc) {
-- 
2.9.0

[PATCH 06/17] Set GIT_NAMESPACE when repo.namespace is provided

[richard.maw]

This causes any namespace-aware code
to only handle refs under that namespace.

Currently this doesn't do much
as the only namespace aware code is in recieve-pack and upload-pack,
which are not handled by CGit.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/cgit.c b/cgit.c
index 9b2733b..eae2f34 100644
--- a/cgit.c
+++ b/cgit.c
@@ -566,6 +566,11 @@ static int prepare_repo_cmd(void)
 	/* The path to the git repository. */
 	setenv("GIT_DIR", ctx.repo->path, 1);
 
+	/*  Set the namespace in the environment,
+	 *  so it gets loaded on the first get_git_namespace() */
+	if (ctx.repo->namespace)
+		setenv("GIT_NAMESPACE", ctx.repo->namespace, 1);
+
 	/* Do not look in /etc/ for gitconfig and gitattributes. */
 	setenv("GIT_CONFIG_NOSYSTEM", "1", 1);
 	setenv("GIT_ATTR_NOSYSTEM", "1", 1);
-- 
2.9.0

[PATCH 07/17] Look up refs in namespace with cgit_get_sha1

[richard.maw]

This causes all ref resolving to look for the requested branch
inside the current namespace.

Previously any form of git revision would be accepted,
but ref resolving isn't namespace aware
and it would be infeasible to replicate all its behaviour,
so we stick to providing the most common cases
of a sha1, an absolute ref, or a partial ref.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 shared.c | 47 ++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 46 insertions(+), 1 deletion(-)

diff --git a/shared.c b/shared.c
index d82c07b..81a5cd8 100644
--- a/shared.c
+++ b/shared.c
@@ -602,7 +602,52 @@ char *get_mimetype_for_filename(const char *filename)
 	return NULL;
 }
 
+static int namespaced_dwim_ref_get_sha1(const char *name, unsigned char *sha1)
+{
+	/* The standard git name disambiguation order is:
+	     $name
+	     refs/$name
+	     refs/tags/$name
+	     refs/heads/$name
+	     refs/remotes/$name (not sure why)
+	     refs/remotes/$name/HEAD
+	   we don't care about remotes, so we can skip those,
+	   and we can't specify a prefix for dwm_ref,
+	   so we have to do this ourselves */
+	static const char *namespaced_ref_patterns[] = {
+		"%s%s",
+		"%srefs/%s",
+		"%srefs/tags/%s",
+		"%srefs/heads/%s",
+		NULL,
+	};
+	const char **p;
+
+	for (p = namespaced_ref_patterns; *p; p++) {
+		char *fullref = NULL;
+		const char *r;
+		fullref = mkpathdup(*p, get_git_namespace(), name);
+		r = resolve_ref_unsafe(fullref, RESOLVE_REF_READING, sha1, NULL);
+		free(fullref);
+		if (r)
+			return 0;
+	}
+	return 1;
+}
+
 int cgit_get_sha1(const char *name, unsigned char *sha1)
 {
-	return get_sha1(name, sha1);
+	if (ctx.repo->namespace) {
+		/* If we have a namespace, we can get either a sha1,
+		   or a possibly abbreviated ref.
+		   Advanced ref forms are not supported at this time
+		   as this would require reimplementing all of ref parsing.
+		   If get_sha1_with_context grows support for a namespaced flag
+		   then this code may go away. */
+		if (get_sha1_hex(name, sha1) == 0)
+			return 0;
+		return namespaced_dwim_ref_get_sha1(name, sha1);
+	} else {
+		return get_sha1(name, sha1);
+	}
 }
-- 
2.9.0

[PATCH 08/17] Guess the default branch based on current namespace

[richard.maw]

resolve_ref_unsafe() can't be told to be namespace aware,
so we need to prepend the namespace beforehand.

Additionally, we need to add the RESOLVE_REF_NO_RECURSE flag,
since otherwise if the commit that is pointed to exists in the root namespace,
it will opt to return that rather than the value in the namespace,
presumably preferring shorter ref names to longer ones.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 14 ++++++++++----
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/cgit.c b/cgit.c
index eae2f34..d9fe2b9 100644
--- a/cgit.c
+++ b/cgit.c
@@ -474,11 +474,17 @@ static char *guess_defbranch(void)
 {
 	const char *ref;
 	unsigned char sha1[20];
-
-	ref = resolve_ref_unsafe("HEAD", 0, sha1, NULL);
-	if (!ref || !starts_with(ref, "refs/heads/"))
+	char *namespaced_head = NULL;
+
+	if (get_git_namespace())
+		namespaced_head = mkpathdup("%sHEAD", get_git_namespace());
+	/* NOTE: RESOLVE_REF_NO_RECURSE is required to prevent it resolving HEAD
+	   into a ref outside of the namespace. */
+	ref = resolve_ref_unsafe(namespaced_head ?: "HEAD", RESOLVE_REF_NO_RECURSE, sha1, NULL);
+	free(namespaced_head);
+	if (!ref || !starts_with(strip_namespace(ref), "refs/heads/"))
 		return "master";
-	return xstrdup(ref + 11);
+	return xstrdup(strip_namespace(ref) + 11);
 }
 /* The caller must free filename and ref after calling this. */
 static inline void parse_readme(const char *readme, char **filename, char **ref, struct cgit_repo *repo)
-- 
2.9.0

[PATCH 09/17] Add cgit_for_each_namespaced_ref_in helper

[richard.maw]

libgit has a for_each_namespaced_ref,
but lacks equivalents for just branches, tags or remotes.

Rather than implementing all of those helpers,
it's more convenient to implement just one
that prepends the namespace to the provided ref base.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.h   |  2 ++
 shared.c | 10 ++++++++++
 2 files changed, 12 insertions(+)

diff --git a/cgit.h b/cgit.h
index 93de0ea..f0d25d6 100644
--- a/cgit.h
+++ b/cgit.h
@@ -394,4 +394,6 @@ extern char *get_mimetype_for_filename(const char *filename);
 
 extern int cgit_get_sha1(const char *name, unsigned char *sha1);
 
+extern int cgit_for_each_namespaced_ref_in(const char *prefix, each_ref_fn fn, void *cb_data);
+
 #endif /* CGIT_H */
diff --git a/shared.c b/shared.c
index 81a5cd8..54940b3 100644
--- a/shared.c
+++ b/shared.c
@@ -651,3 +651,13 @@ int cgit_get_sha1(const char *name, unsigned char *sha1)
 		return get_sha1(name, sha1);
 	}
 }
+
+int cgit_for_each_namespaced_ref_in(const char *prefix, each_ref_fn fn, void *cb_data)
+{
+	struct strbuf strbuf = STRBUF_INIT;
+	int ret;
+	strbuf_addf(&strbuf, "%s%s", get_git_namespace(), prefix);
+	ret = for_each_ref_in(strbuf.buf, fn, cb_data);
+	strbuf_release(&strbuf);
+	return ret;
+}
-- 
2.9.0

[PATCH 10/17] Find the default branch based on the contents of the namespace

[richard.maw]

The find_current_ref callback does not need to be modified
to strip off the namespace prefix,
since the for_each_ref functions don't include the base ref prefix.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cgit.c b/cgit.c
index d9fe2b9..540200e 100644
--- a/cgit.c
+++ b/cgit.c
@@ -458,7 +458,7 @@ static char *find_default_branch(struct cgit_repo *repo)
 	info.req_ref = repo->defbranch;
 	info.first_ref = NULL;
 	info.match = 0;
-	for_each_branch_ref(find_current_ref, &info);
+	cgit_for_each_namespaced_ref_in("refs/heads/", find_current_ref, &info);
 	if (info.match)
 		ref = info.req_ref;
 	else
-- 
2.9.0

[PATCH 11/17] Only display refs in current namespace

[richard.maw]

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 ui-refs.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ui-refs.c b/ui-refs.c
index 5b4530e..f21797b 100644
--- a/ui-refs.c
+++ b/ui-refs.c
@@ -189,9 +189,9 @@ void cgit_print_branches(int maxcount)
 
 	list.refs = NULL;
 	list.alloc = list.count = 0;
-	for_each_branch_ref(cgit_refs_cb, &list);
+	cgit_for_each_namespaced_ref_in("refs/heads/", cgit_refs_cb, &list);
 	if (ctx.repo->enable_remote_branches)
-		for_each_remote_ref(cgit_refs_cb, &list);
+		cgit_for_each_namespaced_ref_in("refs/remotes/", cgit_refs_cb, &list);
 
 	if (maxcount == 0 || maxcount > list.count)
 		maxcount = list.count;
@@ -216,7 +216,7 @@ void cgit_print_tags(int maxcount)
 
 	list.refs = NULL;
 	list.alloc = list.count = 0;
-	for_each_tag_ref(cgit_refs_cb, &list);
+	cgit_for_each_namespaced_ref_in("refs/tags/", cgit_refs_cb, &list);
 	if (list.count == 0)
 		return;
 	qsort(list.refs, list.count, sizeof(*list.refs), cmp_tag_age);
-- 
2.9.0

[PATCH 12/17] Add namespace support to dumb-clone

[richard.maw]

This requires namespacing the HEAD symbolic ref
and the list of refs.

Sending HEAD required some tweaking,
since the file itself refers to a namespaced ref,
but we want to provide the ref with its namespace stripped off.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 ui-clone.c | 23 ++++++++++++++++++++++-
 1 file changed, 22 insertions(+), 1 deletion(-)

diff --git a/ui-clone.c b/ui-clone.c
index 5f6606a..c610ed2 100644
--- a/ui-clone.c
+++ b/ui-clone.c
@@ -86,6 +86,9 @@ void cgit_clone_info(void)
 	ctx.page.filename = "info/refs";
 	cgit_print_http_headers();
 	for_each_ref(print_ref_info, NULL);
+	/* NOTE: we pass an empty prefix because we want to enumerate everything
+	   not just all refs under $namespace/refs/ */
+	cgit_for_each_namespaced_ref_in("", print_ref_info, NULL);
 }
 
 void cgit_clone_objects(void)
@@ -105,5 +108,23 @@ void cgit_clone_objects(void)
 
 void cgit_clone_head(void)
 {
-	send_file(git_path("%s", "HEAD"));
+	if (get_git_namespace()) {
+		unsigned char unused[20];
+		char *namespaced_head = NULL;
+		const char *ref;
+		namespaced_head = mkpathdup("%sHEAD", get_git_namespace());
+		/* NOTE: RESOLVE_REF_NO_RECURSE is required to prevent it resolving HEAD
+		   into a ref outside of the namespace. */
+		ref = resolve_ref_unsafe(namespaced_head, RESOLVE_REF_NO_RECURSE, unused, NULL);
+		if (!ref) {
+			cgit_print_error_page(404, "Not found", "Not found");
+			free(namespaced_head);
+			return;
+		}
+		cgit_print_http_headers();
+		htmlf("ref: %s\n", strip_namespace(ref));
+		free(namespaced_head);
+	} else {
+		send_file(git_path("%s", "HEAD"));
+	}
 }
-- 
2.9.0

[PATCH 13/17] Display notes from namespace

[richard.maw]

We must not leave it at its default behaviour,
as that results in notes from the root namespace being used
and ignoring notes from the current namespace,
since it is not namespace aware.

We can handle this by instructing it to not load the default refs,
and providing a set of extra refs to use.

The provided extra refs are globs rather than ref names,
so we should escape them to be sure.

We get an annoying warning if the provided ref does not exist,
so we check whether the ref exists before attempting to provide it.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c | 34 +++++++++++++++++++++++++++++++++-
 1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/cgit.c b/cgit.c
index 540200e..d428019 100644
--- a/cgit.c
+++ b/cgit.c
@@ -587,7 +587,39 @@ static int prepare_repo_cmd(void)
 	 * load local configuration from the git repository, so we do them both while
 	 * the HOME variables are unset. */
 	setup_git_directory_gently(&nongit);
-	init_display_notes(NULL);
+	if (get_git_namespace()) {
+		struct display_notes_opt opt = {
+			.use_default_notes = 0,
+			.extra_notes_refs = STRING_LIST_INIT_NODUP,
+		};
+		struct strbuf namespaced_note_ref = STRBUF_INIT;
+		struct strbuf glob_escaped = STRBUF_INIT;
+
+		strbuf_add(&namespaced_note_ref, get_git_namespace(),
+		           strlen(get_git_namespace()));
+		strbuf_add(&namespaced_note_ref, "refs/notes/commits",
+		           strlen("refs/notes/commits"));
+
+		if (ref_exists(namespaced_note_ref.buf)) {
+			if (has_glob_specials(namespaced_note_ref.buf)) {
+				for (const char *c = namespaced_note_ref.buf; *c; c++) {
+					if (is_glob_special(*c))
+						strbuf_addchars(&glob_escaped, '\\', 1);
+					strbuf_addchars(&glob_escaped, *c, 1);
+				}
+				string_list_append(&opt.extra_notes_refs, glob_escaped.buf);
+			} else {
+				string_list_append(&opt.extra_notes_refs, namespaced_note_ref.buf);
+			}
+		}
+
+		init_display_notes(&opt);
+		string_list_clear(&opt.extra_notes_refs, 1);
+		strbuf_release(&glob_escaped);
+		strbuf_release(&namespaced_note_ref);
+	} else {
+		init_display_notes(NULL);
+	}
 
 	if (nongit) {
 		const char *name = ctx.repo->name;
-- 
2.9.0

[PATCH 14/17] Add documentation for repo.namespace

[richard.maw]

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgitrc.5.txt | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 2e1912d..59127f8 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -570,6 +570,10 @@ repo.max-stats::
 repo.name::
 	The value to show as repository name. Default value: <repo.url>.
 
+repo.namespace::
+	Set the git namespace, so that fetching and cloning only provide refs
+        under refs/namespaces. Default value: none.
+
 repo.owner::
 	A value used to identify the owner of the repository. Default value:
 	none.
@@ -772,6 +776,34 @@ Conversely, when a ttl value is zero, the cache is disabled for that
 particular page type, and the page type is never cached.
 
 
+NAMESPACES
+----------
+
+A namespace may be created by creating a "HEAD" symbolic ref for the namespace,
+which may be done with the following command:
+
+....
+git symbolic-ref $namespace_ref_base/HEAD $namespace_ref_base/refs/heads/master
+....
+
+and adding "repo.namespace" for the repository.
+
+For a namespace called "foo/bar", the base ref would be
+"refs/namespaces/foo/refs/namespaces/bar/",
+so the command to create the HEAD ref is:
+
+....
+git symbolic-ref refs/namespaces/foo/refs/namespaces/bar/HEAD \
+                 refs/namespaces/foo/refs/namespaces/bar/refs/heads/master
+....
+
+The config setting for this repository would be:
+
+....
+repo.namespace = foo/bar
+....
+
+
 EXAMPLE CGITRC FILE
 -------------------
 
-- 
2.9.0

[PATCH 15/17] Allow agefile to be set per-repository

[richard.maw]

This adds "repo.agefile", since namespaced repositories share the same files,
and we'd like to be able to see the ages of the refs for each namespace.

Whatever the git server uses for updating the age file must be namespace aware
and must write the age file to a path consistent with "repo.agefile".

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgit.c        | 2 ++
 cgit.h        | 1 +
 ui-repolist.c | 3 ++-
 3 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/cgit.c b/cgit.c
index d428019..a87f245 100644
--- a/cgit.c
+++ b/cgit.c
@@ -88,6 +88,8 @@ static void repo_config(struct cgit_repo *repo, const char *name, const char *va
 		repo->logo = xstrdup(value);
 	else if (!strcmp(name, "logo-link") && value != NULL)
 		repo->logo_link = xstrdup(value);
+	else if (!strcmp(name, "agefile") && value != NULL)
+		repo->agefile = xstrdup(value);
 	else if (!strcmp(name, "hide"))
 		repo->hide = atoi(value);
 	else if (!strcmp(name, "ignore"))
diff --git a/cgit.h b/cgit.h
index f0d25d6..b6994a3 100644
--- a/cgit.h
+++ b/cgit.h
@@ -88,6 +88,7 @@ struct cgit_repo {
 	char *section;
 	char *clone_url;
 	char *namespace;
+	char *agefile;
 	char *logo;
 	char *logo_link;
 	int snapshots;
diff --git a/ui-repolist.c b/ui-repolist.c
index 30915df..6b1afb6 100644
--- a/ui-repolist.c
+++ b/ui-repolist.c
@@ -42,7 +42,8 @@ static int get_repo_modtime(const struct cgit_repo *repo, time_t *mtime)
 		*mtime = repo->mtime;
 		return 1;
 	}
-	strbuf_addf(&path, "%s/%s", repo->path, ctx.cfg.agefile);
+	strbuf_addf(&path, "%s/%s", repo->path,
+	            repo->agefile ? repo->agefile : ctx.cfg.agefile);
 	if (stat(path.buf, &s) == 0) {
 		*mtime = read_agefile(path.buf);
 		if (*mtime) {
-- 
2.9.0

[PATCH 16/17] Update contrib script to update agefiles per namespace

[richard.maw]

This uses the same age file as before if it's in the root namespace,
but puts the agefile in info/web/namespaced/$GIT_NAMESPACE/last-modified.

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 contrib/hooks/post-receive.agefile | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/contrib/hooks/post-receive.agefile b/contrib/hooks/post-receive.agefile
index 2f72ae9..d0be639 100755
--- a/contrib/hooks/post-receive.agefile
+++ b/contrib/hooks/post-receive.agefile
@@ -10,10 +10,23 @@
 # each of your repositories.
 #
 
-agefile="$(git rev-parse --git-dir)"/info/web/last-modified
+agefile="$(git rev-parse --git-dir)"/info/web
+if [ -z "$GIT_NAMESPACE" ]; then
+	agefile="$agefile/last-modified"
+else
+	agefile="$agefile/namespaced/$GIT_NAMESPACE/last-modified"
+fi
+
+old_IFS="$IFS"
+IFS=/
+for comp in $GIT_NAMESPACE; do
+	nsref="${nsref}refs/namespaces/$comp/"
+done
+IFS="$old_IFS"
 
 mkdir -p "$(dirname "$agefile")" &&
 git for-each-ref \
 	--sort=-authordate --count=1 \
 	--format='%(authordate:iso8601)' \
+	$nsref \
 	>"$agefile"
-- 
2.9.0

[PATCH 17/17] Add documentation for repo.agefile

[richard.maw]

Signed-off-by: Richard Maw <richard.maw at gmail.com>
---
 cgitrc.5.txt | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 59127f8..b449338 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -468,6 +468,10 @@ repo.about-filter::
 	Override the default about-filter. Default value: none. See also:
 	"enable-filter-overrides". See also: "FILTER API".
 
+repo.agefile::
+	Override the default agefile. Default value: none. See also:
+	"agefile". See also: "NAMESPACES".
+
 repo.branch-sort::
 	Flag which, when set to "age", enables date ordering in the branch ref
 	list, and when set to "name" enables ordering by branch name. Default
@@ -803,6 +807,16 @@ The config setting for this repository would be:
 repo.namespace = foo/bar
 ....
 
+It's also recommended to set a different age file for the namespace.
+The cgit repository contains a contrib script for generating age-files
+for namespaces in a subdirectory.
+
+The corresponding config for the "foo/bar" namespace would be:
+
+....
+repo.agefile = info/web/namespaced/foo/bar/last-modified
+....
+
 
 EXAMPLE CGITRC FILE
 -------------------
-- 
2.9.0

[PATCH 00/17] Add support for namespaced repositories

[Jason]

Hi Richard,

This looks like really cool work. I'll review it this week and get
back to you. Thanks for submitting.

Jason

[PATCH 01/17] Fix qry.head leak on error

[Jason]

Good catch. Merging this.

In the process of reviewing the actual meat of the series, standby.

[PATCH 03/17] Add a wrapper for get_sha1 called cgit_get_sha1

[Jason]

Sorry for the churn here, but this doesn't apply, due to Lukas' recent
patch. Could you respin this?

By the way, feel free to alternatively submit this series with
git-request-pull if you've got it in a repo somewhere.

[PATCH 03/17] Add a wrapper for get_sha1 called cgit_get_sha1

[richard.maw]

On 12 July 2016 at 00:12, Jason A. Donenfeld <Jason at zx2c4.com> wrote:
> Sorry for the churn here, but this doesn't apply, due to Lukas' recent
> patch. Could you respin this?

Sure thing, I should hopefully have this done late wednesday evening.

> By the way, feel free to alternatively submit this series with
> git-request-pull if you've got it in a repo somewhere.

That's a new one to me. I like patches on a mailing list for feedback,
but applying by e-mail is a faff, so sticking the output of
git-request-pull in the cover letter might be the best of both worlds
for me.

If you'd prefer just the request-pull text I can do that.

[PATCH 03/17] Add a wrapper for get_sha1 called cgit_get_sha1

[Jason]

On Tue, Jul 12, 2016 at 11:16 AM, Richard <richard.maw at gmail.com> wrote:
> On 12 July 2016 at 00:12, Jason A. Donenfeld <Jason at zx2c4.com> wrote:
>> Sorry for the churn here, but this doesn't apply, due to Lukas' recent
>> patch. Could you respin this?
>
> Sure thing, I should hopefully have this done late wednesday evening.
>
>> By the way, feel free to alternatively submit this series with
>> git-request-pull if you've got it in a repo somewhere.
>
> That's a new one to me. I like patches on a mailing list for feedback,
> but applying by e-mail is a faff, so sticking the output of
> git-request-pull in the cover letter might be the best of both worlds
> for me.
>
> If you'd prefer just the request-pull text I can do that.

It's however you wish. Both forms work well for me.