s6/s6-rc policy for Gentoo: config files for service scripts

s6/s6-rc policy for Gentoo: config files for service scripts

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 3917 bytes --]

Hey everybody,

yes this project is still alive and making progress - but real life is 
taking its time toll.

I want to spark another discussion to collect some arguments and 
perspectives:

As of now I have had the following structure:

- The package manager puts service source directories for both user and 
system services to /usr/lib/s6-rc/{user,system}/src/service.

- The package manager puts initial bundles to 
/etc/s6-rc/{user,system}/src/bundles;
    those can be changed by the sysadmin to decide whatever is startet when.

- Custom services can be created at /etc/s6-rc/{user,system}/src.

- The package manager puts initial per-service config files to 
/etc/s6-rc/{user,system}/config,
    those can the be changed by the sysadmin to configure system 
services and set global defaults for user services.

- The sysadmin or the user can copy a skeleton from 
/etc/s6-rc/skel/{config,src} (which is populated by the package manager 
with configs and initial bundles)
    to ${HOME}/.config/s6-rc/, this folder shall be used for 
configuration and custom user services.

That is the current state I have been running on my Desktop for ~2 Months.

After reading through Nosh (https://jdebp.uk/Softwares/nosh/), its ideas 
and its documentation,
as well as reading through some forums of both BSD and Linux, I have 
encountered an alternative idea,
which is, as far as I know, the old standard anyway:

- The package manager puts service source directories and an initial set 
of bundles
    for system services to /etc/s6-rc/system/src/{services,bundles}.

- The package manager puts service source directories and an initial set 
of bundles
    for both user and system services to /usr/share/s6-rc/{user,system} 
as a reference of the defaults.

- The sysadmin or the user can copy a skeleton from 
/etc/s6-rc/skel/{config,src} (which is populated by the package manager 
with configs and initial bundles)
    to ${HOME}/.config/s6-rc/, this folder shall be used for configuring 
existing and custom user services.

- The idea of config files is completely dropped and the "editing the 
config" part is shifted to "editing the run-script"

Following are my current thoughts about this:

Advantages:

- Simplicity: I have realized that parsing and importing the KEY=value 
pairs from the config files makes up about 80% of each service script.
                    Removing config reduces the amount of files and 
directories enormously and makes the scripts very small, hence easy to 
understand and customize.

- Flexibility: A sysadmin has way more options in changing the script 
than in changing the KEY=value pairs.

Apparent disadvantages:

- "Beginner friendliness" is way higher for simple KEY=value config files
    Counter argument: From my experience, people that would be able to 
edit configs but not scripts, do not edit service files anyways,
                                   whilst those who do are profiting 
from the finer control the script editing offers.

Disadvantages:

- User services that first source a config file at 
/etc/s6-rc/user/config and after that another one at 
${HOME}/.config/s6-rc/config
   can have global defaults set by the sysadmin from the former, this is 
not possible with a non-config approach.

- Config allow to group and expose the most relevant options in self 
explaining variable names requiring less insight by the sysadmin.
   Counter argument: The sysadmin is most likely pretty invested in the 
program the service file he is editing belongs to anyway,
                                   hence he wants to deviate from the 
default.


What do you think is better and why?

Do you suggest any alterations or even a completely different approach?


Regards,

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 2027 bytes --]

Paul Sopka schrob:
> - The sysadmin or the user can copy a skeleton from
> /etc/s6-rc/skel/{config,src} (which is populated by the package
> manager with configs and initial bundles) to ${HOME}/.config/s6-rc/,
> this folder shall be used for configuration and custom user services.

A skeleton is not config data, and should thus be in /usr/
Possibly /usr/.../examples/...

> - The package manager puts service source directories and an initial
> set of bundles for system services to
> /etc/s6-rc/system/src/{services,bundles}.
> 
> - The package manager puts service source directories and an initial
> set of bundles for both user and system services to
> /usr/share/s6-rc/{user,system} as a reference of the defaults.

If you put defaults in /usr, then prefer symlinking them into /etc,
rather than creating a copy. That'll automatically handle changing
defaults, and make it obvious what is locally customized.


> - The idea of config files is completely dropped and the "editing the
> config" part is shifted to "editing the run-script"
> [...]
> Do you suggest any alterations or even a completely different approach?

Void has a nice idiom in their run scripts:

| #!/bin/sh
| exec 2>&1
| [ -r conf ] && . ./conf
| exec acpid -f ${OPTS:=-l}

That achieves 3 things:
1) It works out of the box (without a conf file).
2) The user can create a conf file containing
        OPTS='-l -d -S'
   to customize the daemon options.
3) The user can create a conf file containing
        if [ $((`date +%s` % 100)) -eq 0 ] ; then poweroff ; fi
        exec /usr/local/bin/notreallyacpid --foo
   to completely¹ override the run file.

Note how the KEY=value pair in 2) is actually shell, hopefully
eliminating any worries about config file format.

HTH,
    Jan

¹)
The "exec 2>&1" is an artifact of how runit does logging, and should
always be done. Hence it comes before the conf entry point. If there was
a valid use for stderr, lines 2 and 3 could be switched, of course.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 3280 bytes --]

> A skeleton is not config data, and should thus be in /usr/
> Possibly /usr/.../examples/...
I agree, the reason i put it there ist that /etc/s6-linux-init/skel/ 
exists (at least on Gentoo Linux), thus for consistency.

>> - The package manager puts service source directories and an initial
>> set of bundles for system services to
>> /etc/s6-rc/system/src/{services,bundles}.
>>
>> - The package manager puts service source directories and an initial
>> set of bundles for both user and system services to
>> /usr/share/s6-rc/{user,system} as a reference of the defaults.
> If you put defaults in /usr, then prefer symlinking them into /etc,
> rather than creating a copy. That'll automatically handle changing
> defaults, and make it obvious what is locally customized.

I am not sure if I understand correctly, the files under 
/usr/share/s6-rc/{user,system}
are to be there only as a reference, not to be edited.
Are you trying to say that the non-edited files should be symlinked 
rather than copied?

> Void has a nice idiom in their run scripts:
>
> | #!/bin/sh
> | exec 2>&1
> | [ -r conf ] && . ./conf
> | exec acpid -f ${OPTS:=-l}
>
> That achieves 3 things:
> 1) It works out of the box (without a conf file).
> 2) The user can create a conf file containing
>          OPTS='-l -d -S'
>     to customize the daemon options.
> 3) The user can create a conf file containing
>          if [ $((`date +%s` % 100)) -eq 0 ] ; then poweroff ; fi
>          exec /usr/local/bin/notreallyacpid --foo
>     to completely¹ override the run file.
>
> Note how the KEY=value pair in 2) is actually shell, hopefully
> eliminating any worries about config file format.
>
> HTH,
>      Jan
>
> ¹)
> The "exec 2>&1" is an artifact of how runit does logging, and should
> always be done. Hence it comes before the conf entry point. If there was
> a valid use for stderr, lines 2 and 3 could be switched, of course.
Glorious.
Although it has to be done a bit differently when using execline (which 
I intend to use):

Take my current version of the seatd-srv:

| #!/bin/execlineb -P
|
| fdmove -c 2 1
|
| multisubstitute
| {
|     define SRV seatd
|     importas -SD /etc/s6-rc S6CONFIGDIR
| }
|
| envfile -I ${S6CONFIGDIR}/system/config/${SRV}.conf
|
| multisubstitute
| {
|     importas -SD "root" SEATD_USER
|     importas -SD "seat" SEATD_GROUP
|     importas -SD "seatd" SRV_EXEC
|     importas -sSD "" SRV_OPTS
| }
|
|
| ${SRV_EXEC}    -n3
|             -u ${SEATD_USER}
|             -g ${SEATD_GROUP}
|             ${SRV_OPTS}

Would turn into:

| #!/bin/execlineb -P
|
| fdmove -c 2 1
|
| importas -SD /etc/s6-rc S6CONFIGDIR
|
| tryexec ${S6CONFIGDIR}/system/config/seatd
|
| seatd -n3 -u root -g seatd

Allowing the sysadmin to completely override the service.
Unfortunately this also forces the sysadmin to override the service for 
every so little change,
so your

> 2) The user can create a conf file containing
>          OPTS='-l -d -S'
>     to customize the daemon options.
wont work anymore but I do not see how one could work around this in 
execline.

What do you think?


Regards,

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 1594 bytes --]

Paul Sopka schrob:
> I am not sure if I understand correctly, the files under
> /usr/share/s6-rc/{user,system}
> are to be there only as a reference, not to be edited.
> Are you trying to say that the non-edited files should be symlinked rather
> than copied?

I was indeed trying to say that.
But on second thought: you should do whatever Gentoo usually does with
such configuration files. Consistency trumps any minor advantages any
particular approach might have.

> Although it has to be done a bit differently when using execline
> [...]
> Would turn into:
> 
> | #!/bin/execlineb -P
> | fdmove -c 2 1
> | importas -SD /etc/s6-rc S6CONFIGDIR
> | tryexec ${S6CONFIGDIR}/system/config/seatd
> | seatd -n3 -u root -g seatd

That's a more reasonable size than your first example. Although this ...

> Allowing the sysadmin to completely override the service.
> Unfortunately this also forces the sysadmin to override the service for
> every so little change,

... then begs the question: what's the advantage of having the
${S6CONFIGDIR}/system/config/seatd entry point at all? How much effort
does this save the admin over creating their own my_seatd service and
disabling the one you provide?
(Honest question, I don't fully grok s6.)

Being able to easily add custom services is a necessity anyway.

So maybe don't allow any customization (besides disabling) at all, ship

| #!/bin/execlineb -P
| fdmove -c 2 1
| seatd -n3 -u root -g seatd

and teach users to create new services with the 3rd line changed to
whatever they need.

HTH,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Hoël Bézier]

[-- Attachment #1: Type: text/plain, Size: 1205 bytes --]

Am Do, Sep 19, 2024 am 07:51:50 +0200 schrieb Jan Braun:
>> Allowing the sysadmin to completely override the service.
>> Unfortunately this also forces the sysadmin to override the service for
>> every so little change,
>
>... then begs the question: what's the advantage of having the
>${S6CONFIGDIR}/system/config/seatd entry point at all? How much effort
>does this save the admin over creating their own my_seatd service and
>disabling the one you provide?
>(Honest question, I don't fully grok s6.)

It doesn’t save any effort at all.

>Being able to easily add custom services is a necessity anyway.
>
>So maybe don't allow any customization (besides disabling) at all, ship
>
>| #!/bin/execlineb -P
>| fdmove -c 2 1
>| seatd -n3 -u root -g seatd
>
>and teach users to create new services with the 3rd line changed to
>whatever they need.

For the record, I’m also in favor of this: the simpler the service scripts, the 
better in my opinion. They’re easier to understand, simpler to adapt. A short 
and direct script can be as descriptive as descriptive configuration files 
sometimes, but better because you actually know everything that takes place. :D

Hoël

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 1026 bytes --]

>> ... then begs the question: what's the advantage of having the
>> ${S6CONFIGDIR}/system/config/seatd entry point at all? How much effort
>> does this save the admin over creating their own my_seatd service and
>> disabling the one you provide?
>> (Honest question, I don't fully grok s6.)
>
> It doesn’t save any effort at all. 
I disagree, because it allows original service scripts to sit at 
/usr/lib/s6-rc/{user,system}/src/service
and be recklessly updated by the package manager.

> For the record, I’m also in favor of this: the simpler the service 
> scripts, the better in my opinion. They’re easier to understand, 
> simpler to adapt. A short and direct script can be as descriptive as 
> descriptive configuration files sometimes, but better because you 
> actually know everything that takes place. :D 
I wholeheartedly agree, this is pretty exactly my mindset too. 
Additionally they are probably faster, since they are smaller and do not 
need to parse configs.

Regards,

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]

Paul Sopka schrob:
> > > ... then begs the question: what's the advantage of having the
> > > ${S6CONFIGDIR}/system/config/seatd entry point at all? How much effort
> > > does this save the admin over creating their own my_seatd service and
> > > disabling the one you provide?
> > > (Honest question, I don't fully grok s6.)
> > 
> > It doesn’t save any effort at all.
> I disagree, because it allows original service scripts to sit at
> /usr/lib/s6-rc/{user,system}/src/service
> and be recklessly updated by the package manager.

I think there's a misunderstanding here, I'll try again:
What I proposed is putting the service definitions into /usr, never copy
or link them to /etc, never modify them (except by package updates), and
let s6 execute them directly from there.

If the local admin wants something other than the service as you ship
it, they create a completely new, independent local service and disable
the one shipped by you.

Thus you can update the definitions in /usr as you see fit, and the
admin can override things as they see fit. No machinery needed inside
the actual run scripts.

Cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 1889 bytes --]

> I think there's a misunderstanding here, I'll try again:
> What I proposed is putting the service definitions into /usr, never copy
> or link them to /etc, never modify them (except by package updates), and
> let s6 execute them directly from there.
>
> If the local admin wants something other than the service as you ship
> it, they create a completely new, independent local service and disable
> the one shipped by you.
>
> Thus you can update the definitions in /usr as you see fit, and the
> admin can override things as they see fit. No machinery needed inside
> the actual run scripts.
I do not think that providing only this option this is a good idea,
since names are unique and dependencies are declared in the dependent 
service.

Consider the example of "seatd-srv":

The sysadmin disables it and creates "seatd-new-srv",
now he has to create a "${SERVICE}-new-srv" for every "${SERVICE}-srv" 
depending on "seatd-srv"
to adjust their dependencies, do the same for every 
"${SERVICE}-new-srv", ...

Of course, this can easily be solved by only ever depending on bundles,
thus, one creates "seatd-new-srv" adds it to the bundle 
"seat-management"(everyone depends on),
removes "seatd-srv" from "seat-management" and has exactly what you 
envisioned.

Thus, a policy of "only ever depend on a bundle", which I already 
employ, is needed.
Anyway, I am not sure if all if this additionally requirement is really 
worth saving one line per service script.

A point for your approach is, of course, that the user gains full control
over the service he changed, as he is able to change the logging 
behavior, dependencies, ...

That however, requires way more insight into the entire service system, 
s6 and s6-rc
as opposed to "read the script, understand the script, adapt the script, 
put it in the right place".

Regards,

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 278 bytes --]

Paul Sopka schrob:
> I do not think that providing only this option this is a good idea,
> since names are unique and dependencies are declared in the dependent
> service.

"It simplifies dependencies" sounds like a good reason to go with the
tryexec approach.

Cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Paul Sopka]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 1786 bytes --]

 From Jan:

> That's a more reasonable size than your first example. Although this ...
>
>> Allowing the sysadmin to completely override the service.
>> Unfortunately this also forces the sysadmin to override the service for
>> every so little change,
> ... then begs the question: what's the advantage of having the
> ${S6CONFIGDIR}/system/config/seatd entry point at all? How much effort
> does this save the admin over creating their own my_seatd service and
> disabling the one you provide?
> (Honest question, I don't fully grok s6.)
It does not have anything to do with s6 itself,
it just allows original service scripts to sit at 
/usr/lib/s6-rc/{user,system}/src/service
and be recklessly updated by the package manager.

Installing them initially to /etc/s6-rc/{user,system}/src/service and 
editing them in place
will not allow that. This is in part mitigated by:

>> I am not sure if I understand correctly, the files under
>> /usr/share/s6-rc/{user,system}
>> are to be there only as a reference, not to be edited.
>> Are you trying to say that the non-edited files should be symlinked rather
>> than copied?
> I was indeed trying to say that.
> But on second thought: you should do whatever Gentoo usually does with
> such configuration files. Consistency trumps any minor advantages any
> particular approach might have.
>
But this introduces needless complexity just to save one line in every 
script.

The best would probably be for s6-rc-compile to allow for multiple 
definitions of a service,
letting later definitions override earlier ones, e.g.

s6-rc-compile ${OUTPUT_DB} ${SOURCE_1} ${SOURCE_2}

where seatd-srv in ${SOURCE_2} overrides seatd-srv in ${SOURCE_1}.

Would this be realizable Laurent?

Regards,

Paul


[-- Attachment #1.1.1.2: Type: text/html, Size: 2675 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re[2]: s6/s6-rc policy for Gentoo: config files for service scripts

[Laurent Bercot]

[-- Attachment #1: Type: text/plain, Size: 1056 bytes --]

>The best would probably be for s6-rc-compile to allow for multiple 
>definitions of a service,
>letting later definitions override earlier ones, e.g.
>
>s6-rc-compile ${OUTPUT_DB} ${SOURCE_1} ${SOURCE_2}
>
>where seatd-srv in ${SOURCE_2} overrides seatd-srv in ${SOURCE_1}.
>
>Would this be realizable Laurent?
>

  Overrides? No, this would make things confusing and error-prone. The 
behaviour
of a set of source definition directories should not be influenced by 
whether or not
you're compiling it with another set.

  Organizing services is something I don't have a lot of policy advice 
on. There's a balance
to be found between ease of use and flexibility; a sound approach seems 
to be to have
the service itself hardcoded in the system set of source definition 
directories (handled
by the package manager) but keep the run script itself pretty 
configurable via
environment definitions sourced from user-modifiable files in /etc. It's 
not the only
possible approach, but it works for most services.

--
  Laurent

Re: s6/s6-rc policy for Gentoo: config files for service scripts

[Paul Sopka]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 272 bytes --]

>  Overrides? No, this would make things confusing and error-prone. The 
> behaviour
> of a set of source definition directories should not be influenced by 
> whether or not
> you're compiling it with another set.
Thanks for letting me know!

Regards,

Paul


[-- Attachment #1.1.1.2: Type: text/html, Size: 650 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]