[RFC] Relocate repos when using scan-path

[RFC] Relocate repos when using scan-path

[bluewind]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

Having now seen section-from-path now I think this feature is pretty
useful, but to fully utilize it I'd have to move around some repos for a
proper directory structure.

Since I want to keep URLs working for existing clones I thought about
placing symlinks in the old location, but then cgit will display each
repo twice.

Should there be an option to ignore symlinks, a (per repo) list of valid
paths for the index or something else to keep cgit's index list clean?



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.20 (GNU/Linux)

iQIcBAEBAgAGBQJRkPD9AAoJEG0WVcFM4cE+V3EP+gNrdR0mxRGrDSbqiiCBJuHR
J/zXz2RCKDT29ZzvI94aCWilxJgYYcNKMt+RuQW88PyKLDhZrx9FDNg2WPARMI7N
QCxj/J/OkXem4lzcpvZFFMjmpOMKqD3k3CWFCayjqpNn7Ubtlfyy6gB81x2iMR0U
G1TrxGIC9k15qoXDwlaAk6NdpN3p/LWECDaTtANFO9uEeaI9EeUj2OE4N0blrMve
hXu9i6e4qP+WYuDEfWUz9JAeJ6aBEUWBxv0nk5zR/wL3mVa3x0mFcrDVabn4V1aG
wPf1PgucxAhce6GeF+iAOKrkmQSmffybmgdUVSAb+t7PFcDCGJKflOeIiZLPlHPj
DK3jzRxXEfsTr2GiJAbCEEhQds80Hs3WQB6QwTvOyi0bWlcOcA4+E29Ri6lwZOxn
EF0Wwh3MFeCgS5YdS+pGTsKHhdCNkkBZ95BTw334EnhQuWWmhl0fyNqQFUK4ohRO
Tv4mxp2eql2Kg9gQv17rLbmJ+EYAIgHYEI9lvcQGAI2fiQQIJ4bXVGkOs/mCk6BV
M053OdcKvQNJ4IRgqJxEhSlQf6C3MCnh85QXeUK0fpYXuYtakclmdNIWE/ytmE2H
Fs/z+I3qFwXQ5QNGG41Anq8daFVzAPRETMO9UMcAC2IvtKUib2Ma16olTTx4C5xD
hZKLPDtcFUHSIpzqpBBj
=1SjG
-----END PGP SIGNATURE-----

[RFC] Relocate repos when using scan-path

[john]

On Mon, May 13, 2013 at 03:56:13PM +0200, Florian Pritz wrote:
> Having now seen section-from-path now I think this feature is pretty
> useful, but to fully utilize it I'd have to move around some repos for a
> proper directory structure.
> 
> Since I want to keep URLs working for existing clones I thought about
> placing symlinks in the old location, but then cgit will display each
> repo twice.
> 
> Should there be an option to ignore symlinks, a (per repo) list of valid
> paths for the index or something else to keep cgit's index list clean?

I think what you want here is to return a "301 Moved Permanently"
response on the old URLs.  I considered adding this to CGit but I'm not
sure we should add yet more configuration options when this can already
be achieved with mod_rewrite or ngx_http_rewrite_module.

[RFC] Relocate repos when using scan-path

[Jason]

John is right here. This is something you definitely want to handle
elsewhere -- using mod_rewrite or similar.

[RFC] Relocate repos when using scan-path

[bluewind]

On 18.05.2013 17:10, John Keeping wrote:
> On Mon, May 13, 2013 at 03:56:13PM +0200, Florian Pritz wrote:
>> Having now seen section-from-path now I think this feature is pretty
>> useful, but to fully utilize it I'd have to move around some repos for a
>> proper directory structure.
>> 
>> Since I want to keep URLs working for existing clones I thought about
>> placing symlinks in the old location, but then cgit will display each
>> repo twice.
>> 
>> Should there be an option to ignore symlinks, a (per repo) list of valid
>> paths for the index or something else to keep cgit's index list clean?
> 
> I think what you want here is to return a "301 Moved Permanently"
> response on the old URLs.  I considered adding this to CGit but I'm not
> sure we should add yet more configuration options when this can already
> be achieved with mod_rewrite or ngx_http_rewrite_module.
> 

The point is that I want to keep already cloned repositories (as in "git
clone git://foo" via git-daemon) working and mod_rewrite won't help with
that even though it would probably work fine with the http clones.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20130518/32ae87c5/attachment.asc>

[RFC] Relocate repos when using scan-path

[Jason]

On Sat, May 18, 2013 at 5:27 PM, Florian Pritz <bluewind at xinu.at> wrote:
> The point is that I want to keep already cloned repositories (as in "git
> clone git://foo" via git-daemon) working and mod_rewrite won't help with
> that even though it would probably work fine with the http clones.

Make a directory elsewhere on the file system.
Symlink everything you need into it.
Point git-daemon at that.

[RFC] Relocate repos when using scan-path

[john]

On Sat, May 18, 2013 at 05:46:35PM +0200, Jason A. Donenfeld wrote:
> On Sat, May 18, 2013 at 5:27 PM, Florian Pritz <bluewind at xinu.at> wrote:
> > The point is that I want to keep already cloned repositories (as in "git
> > clone git://foo" via git-daemon) working and mod_rewrite won't help with
> > that even though it would probably work fine with the http clones.
> 
> Make a directory elsewhere on the file system.
> Symlink everything you need into it.
> Point git-daemon at that.

That might still break SSH; for a proper solution I think we need
something like this (untested).

It might be good to force USE_WILDCARD=YesPlease in the makefile on top
of this patch, which will replace fnmatch with Git's wildmatch, adding
support for "**" to match any number of directory levels.

-- >8 --
Subject: [PATCH] Add scan-exclude option to filter out repositories

Signed-off-by: John Keeping <john at keeping.me.uk>
---
 cgit.c       |  4 ++++
 cgit.h       |  1 +
 cgitrc.5.txt |  7 +++++++
 scan-tree.c  | 15 +++++++++++++++
 4 files changed, 27 insertions(+)

diff --git a/cgit.c b/cgit.c
index 6f44ef2..c621439 100644
--- a/cgit.c
+++ b/cgit.c
@@ -229,6 +229,8 @@ static void config_cb(const char *name, const char *value)
 		ctx.cfg.max_commit_count = atoi(value);
 	else if (!strcmp(name, "project-list"))
 		ctx.cfg.project_list = xstrdup(expand_macros(value));
+	else if (!strcmp(name, "scan-exclude"))
+		string_list_append(&ctx.cfg.scan_exclude, expand_macros(value));
 	else if (!strcmp(name, "scan-path"))
 		if (!ctx.cfg.nocache && ctx.cfg.cache_size)
 			process_cached_repolist(expand_macros(value));
@@ -405,6 +407,8 @@ static void prepare_context(struct cgit_context *ctx)
 	ctx->page.expires = ctx->page.modified;
 	ctx->page.etag = NULL;
 	memset(&ctx->cfg.mimetypes, 0, sizeof(struct string_list));
+	memset(&ctx->cfg.scan_exclude, 0, sizeof(struct string_list));
+	ctx->cfg.scan_exclude.strdup_strings = 1;
 	if (ctx->env.script_name)
 		ctx->cfg.script_name = xstrdup(ctx->env.script_name);
 	if (ctx->env.query_string)
diff --git a/cgit.h b/cgit.h
index 850b792..ee6a545 100644
--- a/cgit.h
+++ b/cgit.h
@@ -238,6 +238,7 @@ struct cgit_config {
 	int branch_sort;
 	int commit_sort;
 	struct string_list mimetypes;
+	struct string_list scan_exclude;
 	struct cgit_filter *about_filter;
 	struct cgit_filter *commit_filter;
 	struct cgit_filter *source_filter;
diff --git a/cgitrc.5.txt b/cgitrc.5.txt
index 9b803b3..71db425 100644
--- a/cgitrc.5.txt
+++ b/cgitrc.5.txt
@@ -329,6 +329,13 @@ root-title::
 	Text printed as heading on the repository index page. Default value:
 	"Git Repository Browser".
 
+scan-exclude::
+	Specified a pattern to be used to exclude repositories found by
+	"scan-path". This option may be specified more than once, with each
+	pattern being compared to discovered repositories with fnmatch(3).
+	Any matches will cause the repository to be omitted from CGit's index.
+	This must be defined prior to scan-path. See also: scan-path.
+
 scan-hidden-path::
 	If set to "1" and scan-path is enabled, scan-path will recurse into
 	directories whose name starts with a period ('.'). Otherwise,
diff --git a/scan-tree.c b/scan-tree.c
index a1ec8fb..6ab49e5 100644
--- a/scan-tree.c
+++ b/scan-tree.c
@@ -75,6 +75,18 @@ static char *xstrrchr(char *s, char *from, int c)
 	return from < s ? NULL : from;
 }
 
+static int exclude_repo(const char *path)
+{
+	int i;
+	struct string_list *exclude = &ctx.cfg.scan_exclude;
+
+	for (i = 0; i < exclude->nr; i++)
+		if (!fnmatch(exclude->items[i].string, path, FNM_PATHNAME))
+			return 1;
+
+	return 0;
+}
+
 static void add_repo(const char *base, struct strbuf *path, repo_config_fn fn)
 {
 	struct stat st;
@@ -91,6 +103,9 @@ static void add_repo(const char *base, struct strbuf *path, repo_config_fn fn)
 		return;
 	}
 
+	if (exclude_repo(path->buf))
+		return;
+
 	strbuf_addch(path, '/');
 	pathlen = path->len;
 
-- 
1.8.3.rc2.285.gfc18c2c

[RFC] Relocate repos when using scan-path

[Jason]

On Sat, May 18, 2013 at 6:06 PM, John Keeping <john at keeping.me.uk> wrote:
> That might still break SSH; for a proper solution I think we need
> something like this (untested).

Why? What's git-daemon have anything to do with SSH?

> Subject: [PATCH] Add scan-exclude option to filter out repositories

Not really super up for merging this. Seems superfluous. There's
projects.list for this sort of thing, anyway, albeit in the opposite
direction.

[RFC] Relocate repos when using scan-path

[john]

On Sat, May 18, 2013 at 06:20:56PM +0200, Jason A. Donenfeld wrote:
> On Sat, May 18, 2013 at 6:06 PM, John Keeping <john at keeping.me.uk> wrote:
> > That might still break SSH; for a proper solution I think we need
> > something like this (untested).
> 
> Why? What's git-daemon have anything to do with SSH?

Nothing, but the same problem of moving repositories applies.

We can solve HTTP with a 301 redirect but not git:// or ssh:// and
people seem to use absolute paths with SSH if they're not using Gitolite
or an equivalent, which makes it harder to move repositories accessed in
that way.

[RFC] Relocate repos when using scan-path

[bluewind]

On 18.05.2013 18:06, John Keeping wrote:
> On Sat, May 18, 2013 at 05:46:35PM +0200, Jason A. Donenfeld wrote:
>> On Sat, May 18, 2013 at 5:27 PM, Florian Pritz <bluewind at xinu.at> wrote:
>> > The point is that I want to keep already cloned repositories (as in "git
>> > clone git://foo" via git-daemon) working and mod_rewrite won't help with
>> > that even though it would probably work fine with the http clones.
>> 
>> Make a directory elsewhere on the file system.
>> Symlink everything you need into it.
>> Point git-daemon at that.
> 
> Subject: [PATCH] Add scan-exclude option to filter out repositories

That's a global option and I don't want to update the global config each
time I add/move repos.

Maybe we could check if the repo dir is a symlink and only hide it and
redirect to the new location if the last target is within scan-path?

That would allow users to easily move repos to subdirectories while
still allowing them to get the current behaviour back by simply moving
the target somewhere outside scan-path and placing some symlinks.

It also won't need a option if documented properly and IMHO it's rather
intuitive.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20130519/a079431e/attachment.asc>