[RFC] Using Git's internal config system

[RFC] Using Git's internal config system

[cgit]

Hi,

We have been discussing the feasibility of using Git's internal config
system instead of our own configuration file parser on IRC and on the
mailing list. It seems that this is doable -- however, creating a sane
upgrade path is not that easy.

What cgit config files currently look like:

    cache-size=1000
    enable-index-links=1

    repo.url=repo1.git
    repo.path=/path/to/repo1.git/
    repo.desc=Repository 1

    repo.url=repo2.git
    repo.path=/path/to/repo2.git/
    repo.desc=Repository 2

What cgit config files might look like when using Git's internal config
system:

    [core]
        cache-size = 1000
        enable-index-links = 1
    [repo "repo1"]
        url = repo1.git
        path = /path/to/repo1.git/
        desc = Repository 1
    [repo "repo2"]
        url = repo2.git
        path = /path/to/repo2.git/
        desc = Repository 2

I was thinking of a way to support both syntaxes so that people have
enough time to migrate to the new format (the other possibility is just
switching to the new format without providing any backwards
compatibility). Since Git's config system does not allow dots inside
variable names, I suggest following implementation:

1. Use strbuf instead of fixed-size buffers to read lines from the
   configuration file (I already submitted a patch for this).

2. Allow following syntax which allows for creating subsections for each
   repository instead of using "repo.url" as a delimiter:

    repo.repo1.url=repo1.git
    repo.repo1.path=/path/to/repo1.git/
    repo.repo1.desc=Repository 1

3. Support sections and internally map each configuration variable "var"
   that is specified inside section "section" to "var.section" and each
   "var" that is specified inside section 'repo "reponame"' to
   "repo.reponame.var", respectively (that is exactly how Git does it
   internally). Prefix section-less configuration variables with "core."

4. Deprecate the old configuration format.

5. Drop support for the old configuration format, remove legacy code and
   use Git's internal config system instead in a future major release.

I prepared patches for 2 and 3. However, there are several issues with
the new syntax:

* We can no longer use "mimetype." in the core section. Solution is
  simple: Create a separate "mimetype" section.

* We need to find an alternate syntax for "repo.module-link.name =
  value". As far as I know, Git does not support nested sections. Does
  anybody have an idea how to do this? We need something like:

    [repo "foo"]
        url = foo.git
        path = /some/path/to/foo/
        desc = Foo repository
	[module-link "path1"]
            format = formatstring1
	[module-link "path2"]
            format = formatstring2

  Maybe just use "module-link = " and allow delimiters to specify pairs
  of paths and corresponding format strings?

* How do we support "section = " statements? Basically the same issue.

Ideas and suggestions welcome!

Regards,
Lukas

[RFC] Using Git's internal config system

[mailings]

On 05/06/13 12:06, Lukas Fleischer wrote:
> Hi,
> 
> We have been discussing the feasibility of using Git's internal config
> system instead of our own configuration file parser on IRC and on the
> mailing list. It seems that this is doable -- however, creating a sane
> upgrade path is not that easy.
> 
> What cgit config files currently look like:
> 
>     cache-size=1000
>     enable-index-links=1
> 
>     repo.url=repo1.git
>     repo.path=/path/to/repo1.git/
>     repo.desc=Repository 1
> 
>     repo.url=repo2.git
>     repo.path=/path/to/repo2.git/
>     repo.desc=Repository 2
> 
> What cgit config files might look like when using Git's internal config
> system:
> 
>     [core]
>         cache-size = 1000
>         enable-index-links = 1
>     [repo "repo1"]
>         url = repo1.git
>         path = /path/to/repo1.git/
>         desc = Repository 1
>     [repo "repo2"]
>         url = repo2.git
>         path = /path/to/repo2.git/
>         desc = Repository 2
> 
> I was thinking of a way to support both syntaxes so that people have
> enough time to migrate to the new format (the other possibility is just
> switching to the new format without providing any backwards
> compatibility). Since Git's config system does not allow dots inside
> variable names, I suggest following implementation:
> 
> 1. Use strbuf instead of fixed-size buffers to read lines from the
>    configuration file (I already submitted a patch for this).
> 
> 2. Allow following syntax which allows for creating subsections for each
>    repository instead of using "repo.url" as a delimiter:
> 
>     repo.repo1.url=repo1.git
>     repo.repo1.path=/path/to/repo1.git/
>     repo.repo1.desc=Repository 1
> 
> 3. Support sections and internally map each configuration variable "var"
>    that is specified inside section "section" to "var.section" and each
>    "var" that is specified inside section 'repo "reponame"' to
>    "repo.reponame.var", respectively (that is exactly how Git does it
>    internally). Prefix section-less configuration variables with "core."
> 
> 4. Deprecate the old configuration format.
> 
> 5. Drop support for the old configuration format, remove legacy code and
>    use Git's internal config system instead in a future major release.
> 
> I prepared patches for 2 and 3. However, there are several issues with
> the new syntax:
> 
> * We can no longer use "mimetype." in the core section. Solution is
>   simple: Create a separate "mimetype" section.
> 
> * We need to find an alternate syntax for "repo.module-link.name =
>   value". As far as I know, Git does not support nested sections. Does
>   anybody have an idea how to do this? We need something like:
> 
>     [repo "foo"]
>         url = foo.git
>         path = /some/path/to/foo/
>         desc = Foo repository
> 	[module-link "path1"]
>             format = formatstring1
> 	[module-link "path2"]
>             format = formatstring2
> 

How about just adding the cgit repo specific config to the actual git
repo config file?
Something like:

[cgit]
url = foo.git
path = /some/path/to/foo/
desc = Foo repository

section = ....

[cgit "module-links"]
format = formatstring1
format = formatstring2

etc..


>   Maybe just use "module-link = " and allow delimiters to specify pairs
>   of paths and corresponding format strings?
> 
> * How do we support "section = " statements? Basically the same issue.
> 
> Ideas and suggestions welcome!
> 
> Regards,
> Lukas
> _______________________________________________
> CGit mailing list
> CGit at lists.zx2c4.com
> http://lists.zx2c4.com/mailman/listinfo/cgit
> 

-- 
Ferry Huberts

[RFC] Using Git's internal config system

[john]

On Wed, Jun 05, 2013 at 12:06:58PM +0200, Lukas Fleischer wrote:
> We have been discussing the feasibility of using Git's internal config
> system instead of our own configuration file parser on IRC and on the
> mailing list. It seems that this is doable -- however, creating a sane
> upgrade path is not that easy.
> 
> What cgit config files currently look like:
> 
>     cache-size=1000
>     enable-index-links=1
> 
>     repo.url=repo1.git
>     repo.path=/path/to/repo1.git/
>     repo.desc=Repository 1
> 
>     repo.url=repo2.git
>     repo.path=/path/to/repo2.git/
>     repo.desc=Repository 2
> 
> What cgit config files might look like when using Git's internal config
> system:
> 
>     [core]
>         cache-size = 1000
>         enable-index-links = 1
>     [repo "repo1"]
>         url = repo1.git
>         path = /path/to/repo1.git/
>         desc = Repository 1
>     [repo "repo2"]
>         url = repo2.git
>         path = /path/to/repo2.git/
>         desc = Repository 2
> 
> I was thinking of a way to support both syntaxes so that people have
> enough time to migrate to the new format (the other possibility is just
> switching to the new format without providing any backwards
> compatibility). Since Git's config system does not allow dots inside
> variable names, I suggest following implementation:
> 
> 1. Use strbuf instead of fixed-size buffers to read lines from the
>    configuration file (I already submitted a patch for this).

How about dropping into the new syntax as soon as you hit a line
beginning with "["?  So we start off using the existing parser, and if
you hit "[" feed the rest of the file to Git's parser.

> 2. Allow following syntax which allows for creating subsections for each
>    repository instead of using "repo.url" as a delimiter:
> 
>     repo.repo1.url=repo1.git
>     repo.repo1.path=/path/to/repo1.git/
>     repo.repo1.desc=Repository 1
>
> 3. Support sections and internally map each configuration variable "var"
>    that is specified inside section "section" to "var.section" and each
>    "var" that is specified inside section 'repo "reponame"' to
>    "repo.reponame.var", respectively (that is exactly how Git does it
>    internally). Prefix section-less configuration variables with "core."
> 
> 4. Deprecate the old configuration format.
> 
> 5. Drop support for the old configuration format, remove legacy code and
>    use Git's internal config system instead in a future major release.
> 
> I prepared patches for 2 and 3. However, there are several issues with
> the new syntax:
> 
> * We can no longer use "mimetype." in the core section. Solution is
>   simple: Create a separate "mimetype" section.
> 
> * We need to find an alternate syntax for "repo.module-link.name =
>   value". As far as I know, Git does not support nested sections. Does
>   anybody have an idea how to do this? We need something like:
> 
>     [repo "foo"]
>         url = foo.git
>         path = /some/path/to/foo/
>         desc = Foo repository
> 	[module-link "path1"]
>             format = formatstring1
> 	[module-link "path2"]
>             format = formatstring2
> 
>   Maybe just use "module-link = " and allow delimiters to specify pairs
>   of paths and corresponding format strings?
> 
> * How do we support "section = " statements? Basically the same issue.

We could just support that using ordering like we currently do.  So when
you hit a "section.name" entry we switch section.  Since Git's parser
just uses callbacks that should be fairly easy to implement.

However, I'm not sure that Git's config syntax necessarily maps well to
CGit's needs.  Git only supports three levels of hierarchy, so the
obvious mapping of several features does not work.  Already some config
keys are not available with "enable-git-config = 1" because the key
would be a path but Git only accepts that in a section name.

[RFC] Using Git's internal config system

[john]

On Wed, Jun 05, 2013 at 12:18:35PM +0200, Ferry Huberts wrote:
> How about just adding the cgit repo specific config to the actual git
> repo config file?
> Something like:
> 
> [cgit]
> url = foo.git
> path = /some/path/to/foo/
> desc = Foo repository
> 
> section = ....
> 
> [cgit "module-links"]
> format = formatstring1
> format = formatstring2
> 
> etc..

You can already do that if you're using "scan-path" and have
"enable-git-config = 1".

[RFC] Using Git's internal config system

[cgit]

On Wed, Jun 05, 2013 at 11:26:53AM +0100, John Keeping wrote:
> On Wed, Jun 05, 2013 at 12:06:58PM +0200, Lukas Fleischer wrote:
> [...]
> > 
> > * We need to find an alternate syntax for "repo.module-link.name =
> >   value". As far as I know, Git does not support nested sections. Does
> >   anybody have an idea how to do this? We need something like:
> > 
> >     [repo "foo"]
> >         url = foo.git
> >         path = /some/path/to/foo/
> >         desc = Foo repository
> > 	[module-link "path1"]
> >             format = formatstring1
> > 	[module-link "path2"]
> >             format = formatstring2
> > 
> >   Maybe just use "module-link = " and allow delimiters to specify pairs
> >   of paths and corresponding format strings?
> > 
> > * How do we support "section = " statements? Basically the same issue.
> 
> We could just support that using ordering like we currently do.  So when
> you hit a "section.name" entry we switch section.  Since Git's parser
> just uses callbacks that should be fairly easy to implement.

Unfortunately, it is not that simple. With the new syntax, elements of
the same section are grouped together and we can't put elements from
different sections in arbitrary order, unless we want to do something
like:

    [core]
        section = section1
    [repo "repo1"]
        url = repo1.git
        path = /path/to/repo1/
    [core]
        section = section2
    [repo "repo2"]
        url = repo2.git
        path = /path/to/repo2/

What we could do is add a "section" key to each repository.

> 
> However, I'm not sure that Git's config syntax necessarily maps well to
> CGit's needs.  Git only supports three levels of hierarchy, so the
> obvious mapping of several features does not work.  Already some config
> keys are not available with "enable-git-config = 1" because the key
> would be a path but Git only accepts that in a section name.

I tend to agree. Jason, any comments?

I will rework that patch I already submitted for now. Using Git's config
system (if desirable) seems to require a lot of preparatory work...

[RFC] Using Git's internal config system

[john]

On Wed, Jun 05, 2013 at 12:52:57PM +0200, Lukas Fleischer wrote:
> On Wed, Jun 05, 2013 at 11:26:53AM +0100, John Keeping wrote:
> > On Wed, Jun 05, 2013 at 12:06:58PM +0200, Lukas Fleischer wrote:
> > [...]
> > > 
> > > * We need to find an alternate syntax for "repo.module-link.name =
> > >   value". As far as I know, Git does not support nested sections. Does
> > >   anybody have an idea how to do this? We need something like:
> > > 
> > >     [repo "foo"]
> > >         url = foo.git
> > >         path = /some/path/to/foo/
> > >         desc = Foo repository
> > > 	[module-link "path1"]
> > >             format = formatstring1
> > > 	[module-link "path2"]
> > >             format = formatstring2
> > > 
> > >   Maybe just use "module-link = " and allow delimiters to specify pairs
> > >   of paths and corresponding format strings?
> > > 
> > > * How do we support "section = " statements? Basically the same issue.
> > 
> > We could just support that using ordering like we currently do.  So when
> > you hit a "section.name" entry we switch section.  Since Git's parser
> > just uses callbacks that should be fairly easy to implement.
> 
> Unfortunately, it is not that simple. With the new syntax, elements of
> the same section are grouped together and we can't put elements from
> different sections in arbitrary order, unless we want to do something
> like:
> 
>     [core]
>         section = section1
>     [repo "repo1"]
>         url = repo1.git
>         path = /path/to/repo1/
>     [core]
>         section = section2
>     [repo "repo2"]
>         url = repo2.git
>         path = /path/to/repo2/

That was what I was suggesting, except with:

    [section]
        name = section2

instead of "core.section".  I agree that it's quite ugly though.