s6/s6-rc policy for Gentoo: XDG Base Directory Specification

s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 2084 bytes --]

To keep all the stuff a bit organized, I will open another "thread" for 
this.

I have read through 
https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html 
and to me the standards seem sensible. Most of it is the default of most 
if not all modern desktop programs anyway.

Therefore I would like to completely comply with them in this policy. 
This involves, but is not limited to, putting the live dir in 
XDG_RUNTIME_DIR. Bercot warned me in 
https://skarnet.org/lists/supervision/3114.html that this might be 
dangerous since it could be removed to quickly upon logout.

 From the start on, I planned to have all the XDG_* env's managed by the 
s6-rc system service which sets up the user supervision tree. This means 
setting the env's and, in the case of XDG_RUNTIME_DIR, creating and 
removing the directory on demand. By implementing this directly in the 
service with correct dependencies, this becomes completely safe and 
reliable.

Another suggestion was to delegate the creation and removal of 
XDG_RUNTIME_DIR to the login monitoring program (which will most likely 
be Turnstile), see https://skarnet.org/lists/supervision/3117.html. I do 
not think this is the best idea. Instead, I want to push as much basic 
functionality as possible to the s6/s6-rc services, for the following 
reasons:

1) s6/s6-rc is simple, fast and reliable

2) this allows for an easier future substitution of external programs 
used (e.g. substituting Turnstile for an alternative)

3) This allows for more fine grained control by the 
sysadmins/distribution contributors.

4) The service setting up and supervising the user tree is perfectly 
suited for this, my current, working implementation of creating and 
deleting XDG_RUNTIME_DIR containing the live dir consists of two lines 
of execline


Do you all have any objections/see any issues with the XDG Base 
Directory Specification and adhering to them?

Do you see any issues with my idea of how to handle the XDG_RUNTIME_DIR?

Any input is welcome!


Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Carlos Eduardo]

I don't think avoiding Turnstile (and similar projects) in favor of s6
services is reasonable, given the concerns you've raised in the first
block of https://skarnet.org/lists/supervision/3122.html (especially
"I'd like for the user supervision tree to only start on-demand").

Turnstile's interface is reasonably simple and easy to implement in an
alternative: backends are effectively supervised 'run' (in the
s6/runit sense) scripts for the service manager with extra arguments,
and I believe it to be pretty much the minimum viable program for
adding XDG_RUNTIME_DIR compliance to any given service manager.

Saying "this will make it easier to implement an alternative"
overlooks the fact that the complexity will be shifted into the
service set and the service manager. Is it really worth it to go
through dynamic services and events (at least, one `install -d`
oneshot per user, and a first login/last logout event source), and
probably other things required to make this more reliable, pretty much
just to make a folder?

I believe point 3 to be moot because XDG policies are very thorough
and doing something even mildly different usually requires redoing
everything from scratch.




Em sex., 5 de jul. de 2024 às 20:35, Paul Sopka <psopka@sopka.ch> escreveu:
>
> To keep all the stuff a bit organized, I will open another "thread" for
> this.
>
> I have read through
> https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
> and to me the standards seem sensible. Most of it is the default of most
> if not all modern desktop programs anyway.
>
> Therefore I would like to completely comply with them in this policy.
> This involves, but is not limited to, putting the live dir in
> XDG_RUNTIME_DIR. Bercot warned me in
> https://skarnet.org/lists/supervision/3114.html that this might be
> dangerous since it could be removed to quickly upon logout.
>
>  From the start on, I planned to have all the XDG_* env's managed by the
> s6-rc system service which sets up the user supervision tree. This means
> setting the env's and, in the case of XDG_RUNTIME_DIR, creating and
> removing the directory on demand. By implementing this directly in the
> service with correct dependencies, this becomes completely safe and
> reliable.
>
> Another suggestion was to delegate the creation and removal of
> XDG_RUNTIME_DIR to the login monitoring program (which will most likely
> be Turnstile), see https://skarnet.org/lists/supervision/3117.html. I do
> not think this is the best idea. Instead, I want to push as much basic
> functionality as possible to the s6/s6-rc services, for the following
> reasons:
>
> 1) s6/s6-rc is simple, fast and reliable
>
> 2) this allows for an easier future substitution of external programs
> used (e.g. substituting Turnstile for an alternative)
>
> 3) This allows for more fine grained control by the
> sysadmins/distribution contributors.
>
> 4) The service setting up and supervising the user tree is perfectly
> suited for this, my current, working implementation of creating and
> deleting XDG_RUNTIME_DIR containing the live dir consists of two lines
> of execline
>
>
> Do you all have any objections/see any issues with the XDG Base
> Directory Specification and adhering to them?
>
> Do you see any issues with my idea of how to handle the XDG_RUNTIME_DIR?
>
> Any input is welcome!
>
>
> Paul
>

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Paul Sopka]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 6060 bytes --]

> I don't think avoiding Turnstile (and similar projects) in favor of s6
> services is reasonable, given the concerns you've raised in the first
> block ofhttps://skarnet.org/lists/supervision/3122.html  (especially
> "I'd like for the user supervision tree to only start on-demand").

I do not want to avoid them, they are straight up required to setup 
option B) from https://skarnet.org/lists/supervision/3114.html.

Instead, I want to delegate as few functionality to them as possible. 
Why should I e.g. delegate the creation a directory to a binary, 
configured through its own config file, if it would just be one line in 
a script that loads the defining variables and is ran anyway?


> Turnstile's interface is reasonably simple and easy to implement in an
> alternative: backends are effectively supervised 'run' (in the
> s6/runit sense) scripts for the service manager with extra arguments,
> and I believe it to be pretty much the minimum viable program for
> adding XDG_RUNTIME_DIR compliance to any given service manager.
I experimented with Turnstile yesterday and hit major roadblocks.

Here is what I have understood it does:

once the user first logs in:

a1) execute "${BACKEND} run ${notification_fd}"    with an environment 
of ${USER}, ${HOME}, ...

  once ${BACKEND} writes back to ${notification_fd}:

a2) execute "${BACKEND} ready ${string}"/    without /the environment, 
but passing what was written back to ${notification_fd} beforehand as 
${string}

once the user last logs out:

b1) execute "${BACKEND} stop"

b2) kill the process started in a1) (in our case s6-svscan)

 From that structure I naturally would propose the following script, 
having ${XDG_RUNTIME_DIR} managed by Turnstile:


#!/usr/bin/env execlineb

importas directive 1
importas readinessPipe 2
importas -S XDG_RUNTIME_DIR
importas -S USER

case ${directive}
{
     run
     {
         foreground {
             install -d -o ${USER} -g ${USER} ${XDG_RUNTIME_DIR}/service
         }
         redirfd -w 3 ${readinessPipe} s6-svscan -d3 
${XDG_RUNTIME_DIR}/service
     }
     ready
     {
         foreground {
             s6-rc-init -c /home/${USER}/.local/share/s6-rc/compiled -l 
${XDG_RUNTIME_DIR}/s6-rc ${XDG_RUNTIME_DIR}/service
         }
         s6-rc -v2 -l ${XDG_RUNTIME_DIR}/s6-rc -up change default
     }
     stop
     {
         s6-rc -l ${XDG_RUNTIME_DIR}/s6-rc -bDa change
     }
}

exit 100


This does not work, since when a2) is invoked to run the "ready" part, 
we do not have ${XDG_RUNTIME_DIR} anymore. I do not see how to solve 
this without a workaround.

A possible workaround would be doubly defining XDG_RUNTIME_DIR, once in 
the Turnstile config and once in an env file which is sourced in the 
"ready" "function".


I have two major issues here:

A) ${BACKEND} is always invoked with the user's privileges, thus it can 
not create ${XDG_RUNTIME_DIR} itself.

B) a2) happens without the environment, see the issue with the script.


I think A) is a problem, since I am planning:

a)  to have a central user env file, preferably at 
/home/${USER}/.config/s6-env (${XDG_CONFIG_HOME}/s6-env), for the user 
to tweak his settings, e.g. ${DBUS_SESSION_BUS_ADDRESS}

b) to have a central sysadmin env file, preferably at /etc/s6-rc/config 
(or similar), for the system admin to tweak the settings, e.g. the 
${XDG_RUNTIME_DIR} general structure, the creation of which has to be 
delegated to Turnstile, since the script does not have enough 
permissions. This would then have to be doubly defined(once in the 
Turnstile config and once in the env-file)


> Saying "this will make it easier to implement an alternative"
> overlooks the fact that the complexity will be shifted into the
> service set and the service manager. Is it really worth it to go
> through dynamic services and events (at least, one `install -d`
> oneshot per user, and a first login/last logout event source), and
> probably other things required to make this more reliable, pretty much
> just to make a folder?
I agree about the complexity shift. But I think the solution of using 
Turnstile just as a "notifications-system" is the best: ideally it 
should do the following: if ${USER} first logs in, start the system 
service ${USER}, if he logs out of the last instance stop the system 
service ${USER}.

This yields the following advantages:

A) The added complexity is in my opinion negligible(see my current 
scripts here: https://skarnet.org/lists/supervision/3123.html).

B) My idea gives the sysadmin and the user more control over their 
system (e.g. basing the location of ${XDG_RUNTIME_DIR} on a central env 
file where other settings are set too).

C) My idea allows plugging the user supervision tree directly into the 
system supervision tree, with the following advantages

     1) Allowing to directly attach a "catch all" logger separated from 
the logger of Turnstile

     2) Keeping the chain of efficient and reliable "skarware" from the 
top of the tree until the last user service, without Turnstile in between.

D) Now if I want to replace Turnstile I only need to tell the 
Replacement the system service to start and stop. I think it is 
reasonable to plan that, since Turnstile is based on PAM and as Bercot 
said in https://skarnet.org/lists/supervision/3114.html:

>   I'm not going to write such a program in the s6 suite, because all the
> stuff around PAM and utmp is 1. very Linux-specific and 2. extremely
> brittle and badly/underspecified. At some point, however, I intend to
> take a good look at PAM and brainstorm about a replacement, because the
> current situation is just holding with massive amounts of duct tape,
> good
> will and headaches, and this makes me fearful for the future. 


Thank you for your valuable input!

Have a nice Sunday!


Paul


[-- Attachment #1.1.1.2: Type: text/html, Size: 8155 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Carlos Eduardo]

Em dom., 7 de jul. de 2024 às 06:06, Paul Sopka <psopka@sopka.ch> escreveu:

> Instead, I want to delegate as few functionality to them as possible. Why should I e.g. delegate the creation a directory to a binary, configured through its own config file, if it would just be one line in a script that loads the defining variables and is ran anyway?

Because, as you noticed, this will require root privileges in the
backend. And, to make it generic, complexity will again be shifted
into the service set; the common default "/run/user/${UID}", for
instance, implies serial substitution in an execline script [1] or
eval in a shell. Which by then, it will no longer be a matter of "just
2 lines of execline".

[1] See https://skarnet.org/software/execline/multisubstitute.html
("Security" section) https://skarnet.org/lists/skaware/2049.html.

Non daemontools-style supervisors, such as dinit (which is Turnstile's
default), also don't provide a simple "setuidgid" privilege dropper,
so they'll have little use for this functionality unless they add more
dependencies or become more error-prone.

You're making more work for yourself and the people who'd help you
maintain the service set, and making an s6 policy with more
assumptions.

>
> I experimented with Turnstile yesterday and hit major roadblocks.
>
> Here is what I have understood it does:
>
> once the user first logs in:
>
> a1) execute "${BACKEND} run ${notification_fd}"    with an environment of ${USER}, ${HOME}, ...

There's two additional arguments: "$srv_dir", a folder that's
dedicated to temporary files for the service manager (which seems to
me to be the ideal place for s6-svscan's scan directory and s6-rc's
live folder¹), and "$confdir", which is the path to where the
configuration files reside (which can serve as the storage place for
an environment file).

¹From my testing, XDG_RUNTIME_DIR also works.

> This does not work, since when a2) is invoked to run the "ready" part, we do not have ${XDG_RUNTIME_DIR} anymore. I do not see how to solve this without a workaround.
>
> A possible workaround would be doubly defining XDG_RUNTIME_DIR, once in the Turnstile config and once in an env file which is sourced in the "ready" "function".
>

See `/run/service/s6rc-fdholder-filler/run` on your machine for
another solution: have 'ready' be only

```
s6-rc -l "$string" -up change default
```

and 'run' be

```
... fdmove -c 3 1 pipeline -dw { if -nt { s6-rc-init } s6-svscanctl -t
... } fdswap 3 1 s6-svscan -d3 ...
```

Instead of just s6-svscan (i. e. feed the readiness notification to
another step instead directly sending s6-svscan's notification to
turnstile).

Given dinit has the service management and service supervision layers
in the same executable, Turnstile's authors probably expect the
service management layer being up and running by the time 'run' has
notified readiness, which is possibly why they didn't bother exporting
variables in the "ready" script.

>
> I have two major issues here:
>
> A) ${BACKEND} is always invoked with the user's privileges, thus it can not create ${XDG_RUNTIME_DIR} itself.
>
> B) a2) happens without the environment, see the issue with the script.
>
>
> I think A) is a problem, since I am planning:
>
> a)  to have a central user env file, preferably at /home/${USER}/.config/s6-env (${XDG_CONFIG_HOME}/s6-env), for the user to tweak his settings, e.g. ${DBUS_SESSION_BUS_ADDRESS}
>
>
> b) to have a central sysadmin env file, preferably at /etc/s6-rc/config (or similar), for the system admin to tweak the settings, e.g. the ${XDG_RUNTIME_DIR} general structure, the creation of which has to be delegated to Turnstile, since the script does not have enough permissions. This would then have to be doubly defined(once in the Turnstile config and once in the env-file)
>

Turnstile already answers item b: `/etc/turnstile/backend/*.conf` and
argument #4 of the run script. For a, nothing stops a backend from
sourcing one itself, and the runit backend is a precedent.

I'd rather document that the user must edit the desktop service
daemon's settings if they want to tweak how desktop services work.

>
> Saying "this will make it easier to implement an alternative"
> overlooks the fact that the complexity will be shifted into the
> service set and the service manager. Is it really worth it to go
> through dynamic services and events (at least, one `install -d`
> oneshot per user, and a first login/last logout event source), and
> probably other things required to make this more reliable, pretty much
> just to make a folder?
>
> I agree about the complexity shift. But I think the solution of using Turnstile just as a "notifications-system" is the best: ideally it should do the following: if ${USER} first logs in, start the system service ${USER}, if he logs out of the last instance stop the system service ${USER}.
>
> This yields the following advantages:
>
> A) The added complexity is in my opinion negligible(see my current scripts here: https://skarnet.org/lists/supervision/3123.html).
>

Your scripts are pretty close functionally to what I currently have on
my machine, but the complexity is small because they're not
configurable, and they don't export all environment variables that are
expected of a login session.

For instance, I once tried to make my equivalent set of scripts
"username-agnostic" and use s6's (the supervision suite's) built-in
instancing feature — with the username as the parameter — to spawn
user supervision trees. This was enough to make execline painful to
use correctly, and I had to switch to a shell script.

I'm currently writing a turnstile backend [1] that has a catch-all
logger and does s6-rc-init in the `run` step; it's refreshing to be
able to reasonably write it in execline.

[1] https://paste.artixlinux.org/view/6308e776

> B) My idea gives the sysadmin and the user more control over their system.

FWIW, Turnstile also allows changing the XDG_RUNTIME_DIR and the
DBUS_SESSION_BUS_ADDRESS, and has a few more format strings than I
expected.

> (e.g. basing the location of ${XDG_RUNTIME_DIR} on a central env file where other settings are set too).

I don't think "low number of config files" is a metric worth
optimizing for, especially when the tradeoff is encoding desktop
assumptions in low-level policy.

> C) My idea allows plugging the user supervision tree directly into the system supervision tree, with the following advantages
>
>     1) Allowing to directly attach a "catch all" logger separated from the logger of Turnstile

See s6-svscan's -X option. This can be done entirely within the user's
supervision tree, and relies on the same foundation as the catch-all
logging in s6-linux-init; it just requires the scan directory to be
initialized with a catch-all logger before s6-svscan is run.

>
>     2) Keeping the chain of efficient and reliable "skarware" from the top of the tree until the last user service, without Turnstile in between.
>

As Laurent said (in regards to graphical user services, but I believe
this applies here), the tradeoff is introducing environment data
sharing and process lifetime management across unrelated process
trees, which is neither.

>
> D) Now if I want to replace Turnstile I only need to tell the Replacement the system service to start and stop. I think it is reasonable to plan that, since Turnstile is based on PAM and as Bercot said in https://skarnet.org/lists/supervision/3114.html:
>

I'm not aware of any Turnstile replacements, and given your design
goals, I think you're better off creating a small single-job PAM
module like pam_rundir [1]. Make it invoke
s6-rc/s6-instance-{create,delete}/whatever when logging in/out instead
of managing a runtime directory.

I don't think writing such a module is worth it, though. Turnstile
allows your system policy to only worry about core machine management;
your proposal will instead muddle that with stuff that's only relevant
for the desktop services.

[1] https://gitea.artixlinux.org/artix/pam_rundir

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Paul Sopka]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 8242 bytes --]


> You're making more work for yourself and the people who'd help you
> maintain the service set, and making an s6 policy with more
> assumptions.
I disagree. If the most important work is delegated to the service 
scripts, once it works, it works. Forever. Easy to adapt and replace by 
sysadmins/users.

Extremely relying on specific functionality of an external program, 
forces the maintainer to tightly monitor the programs development and 
extensively test all its releases, in practice: forcing the possible 
Turnstile package on Gentoo to have the same maintainer(maintaining 
group) as the possible s6/s6-rc profile. It also forces more assumptions 
on everybody, since we need to comply to the exact way Turnstile solves 
things:

> Non daemontools-style supervisors, such as dinit (which is Turnstile's
> default), also don't provide a simple "setuidgid" privilege dropper,
> so they'll have little use for this functionality unless they add more
> dependencies or become more error-prone.
This shows it quite nice, as it would have me need to align more to the 
limitations of dinit, even though I do have the tools for the job. 
Running the backend as root could just be added as a config option to 
Turnstile.

But I think we are approaching philosophical/subjective grounds here.

It is good that we develop our own interpretations of "the best" 
solution, learning from and criticizing each other.


> There's two additional arguments: "$srv_dir", a folder that's
> dedicated to temporary files for the service manager (which seems to
> me to be the ideal place for s6-svscan's scan directory and s6-rc's
> live folder¹), and "$confdir", which is the path to where the
> configuration files reside (which can serve as the storage place for
> an environment file).
It seems less then ideal to me to use the filesystem as a temporary 
storage, when all that is needed are a handful environmental variables, 
which can easily be passed to the backend.


> See `/run/service/s6rc-fdholder-filler/run` on your machine for
> another solution: have 'ready' be only
>
> ```
> s6-rc -l "$string" -up change default
> ```
>
> and 'run' be
>
> ```
> ... fdmove -c 3 1 pipeline -dw { if -nt { s6-rc-init } s6-svscanctl -t
> ... } fdswap 3 1 s6-svscan -d3 ...
> ```
>
> Instead of just s6-svscan (i. e. feed the readiness notification to
> another step instead directly sending s6-svscan's notification to
> turnstile).
Thanks, will test!


> Given dinit has the service management and service supervision layers
> in the same executable, Turnstile's authors probably expect the
> service management layer being up and running by the time 'run' has
> notified readiness, which is possibly why they didn't bother exporting
> variables in the "ready" script.
Adding the ability to pass the env's to the ready function in the form 
of a config option in turnstile.conf would probably be the best way to 
allow elegant integration with s6-rc. I have opened an issue for that, 
see here: https://github.com/chimera-linux/turnstile/issues/15


> Turnstile already answers item b: `/etc/turnstile/backend/*.conf` and
> argument #4 of the run script. For a, nothing stops a backend from
> sourcing one itself, and the runit backend is a precedent.
Yet again needing tight integration with Turnstile. I do not think this 
is inherently bad, I just do not personally like such tight integration 
if I can do it differently.


> I'd rather document that the user must edit the desktop service
> daemon's settings if they want to tweak how desktop services work.
I technically agree, but some configs must be global, e.g. 
${DBUS_SESSION_BUS_ADDRESS}, since other services, e.g. Pipewire, rely 
on them.

The alternative would be a dynamically populated and sourced env 
directory, I am still thinking about this, but the issue I see is how to 
propagate the required (currently ${DBUS_SESSION_BUS_ADDRESS} and 
${XDG_RUNTIME_DIR}) to the user's login shell now that they are dynamic 
and maybe not completely set on login?


> For instance, I once tried to make my equivalent set of scripts
> "username-agnostic" and use s6's (the supervision suite's) built-in
> instancing feature — with the username as the parameter — to spawn
> user supervision trees. This was enough to make execline painful to
> use correctly, and I had to switch to a shell script.
I will try that too, because I think this would be the most elegant way 
to start user service trees, if one doesn't want to rely on Turnstile.


> I'm currently writing a turnstile backend [1] that has a catch-all
> logger and does s6-rc-init in the `run` step; it's refreshing to be
> able to reasonably write it in execline.
>
> [1]https://paste.artixlinux.org/view/6308e776
Wow, I will look into that in more detail for sure!


> I don't think "low number of config files" is a metric worth
> optimizing for, especially when the tradeoff is encoding desktop
> assumptions in low-level policy.
I need to be careful to not have two or more important config files that 
contain the same setting parsed by different essential software. This 
would require the sysadmin to keep them in sync.

Additionally I do think that "low number of config files" is an 
important metric when it comes to /related/ settings, since that makes 
life easier for users and sysadmins.


> See s6-svscan's -X option. This can be done entirely within the user's
> supervision tree, and relies on the same foundation as the catch-all
> logging in s6-linux-init; it just requires the scan directory to be
> initialized with a catch-all logger before s6-svscan is run.
Thank you, I will take a look!


> As Laurent said (in regards to graphical user services, but I believe
> this applies here), the tradeoff is introducing environment data
> sharing and process lifetime management across unrelated process
> trees, which is neither.
This might be subjective again, but I would not consider the user (and 
thus his service-tree) be unrelated to the system he uses (and thus the 
system-tree)


> I'm not aware of any Turnstile replacements, and given your design
> goals, I think you're better off creating a small single-job PAM
> module like pam_rundir [1]. Make it invoke
> s6-rc/s6-instance-{create,delete}/whatever when logging in/out instead
> of managing a runtime directory.
>
> I don't think writing such a module is worth it, though. Turnstile
> allows your system policy to only worry about core machine management;
> your proposal will instead muddle that with stuff that's only relevant
> for the desktop services.
>
> [1]https://gitea.artixlinux.org/artix/pam_rundir
Thanks for the heads up and the link.


This brings me to my last point, an idea I got while writing this: 
https://skarnet.org/lists/supervision/3131.html.

The idea is the following:

- Make a user service tree on boot (preferably using instantiated 
services) if the user has an appropriate dir in ${XDG_DATA_HOME} (most 
likely /home/${USER}/.local/.share/s6-rc/...). This service prepares all 
the basics, e.g. ${XDG_RUNTIME_DIR}, the logging dir of the user 
services, ...

Then:

- Autostart all user-services in an "autostart" bundle the user populated.

- Upon first login start a "login" bundle the user populated, stop it 
after last logout. This can be fine grained with a bundle for ssh login, 
a bundle for getty login,..., using appropriate PAM modules.

- An env directory which is also used by the user-services shall be 
sourced upon login.

- Upon starting a graphical session start a sub-user-tree with the 
s6-svscan process being a child of the wayland compositor,X server,...


This allows the following:

1) Solving all of Bercots cases 
(https://skarnet.org/lists/supervision/3114.html) A), B) and C) with 
"just" 2 user-service-trees, one being nested inside the other.

2) Not relying on external tools to do basic work.

3) Being uniform (everything based on s6-tools) and thus (hopefully) 
relatively easy to maintain.


Do you see any issues with that proposal?


Thank you for your critique!


Paul


[-- Attachment #1.1.1.2: Type: text/html, Size: 11526 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Carlos Eduardo]

[-- Attachment #1: Type: text/plain, Size: 9279 bytes --]

Em seg., 8 de jul. de 2024 às 09:55, Paul Sopka <psopka@sopka.ch> escreveu:
> You're making more work for yourself and the people who'd help you
> maintain the service set, and making an s6 policy with more
> assumptions.
>
> I disagree. If the most important work is delegated to the service
scripts, once it works, it works. Forever. Easy to adapt and replace by
sysadmins/users.

"Forever" is a strong word (see below), and for the latter of these to be
true, a lot of thought must go into packaging and design.

> Extremely relying on specific functionality of an external program,
forces the maintainer to tightly monitor the programs development and
extensively test all its releases.

This is the maintainer's job. Even s6 is not an unchanging foundation; it
has had commands being renamed and shifting from one package to another in
the past, and some use cases stopped being supported along the way.

One example I remember is setsid becoming mandatory for services, which
broke mine and a few people's "X server/compositor as a service" setup
under elogind+polkit back in the day.

> in practice: forcing the possible Turnstile package on Gentoo to have the
same maintainer(maintaining group) as the possible s6/s6-rc profile.

Not necessarily. Turnstile is designed in a way that makes it possible for
the people writing the s6-rc profile to focus on getting system daemons
right and have their mission regarding desktop services stop at creating a
Turnstile service, and to ensure the responsibility for a working user
services setup to fall entirely on the maintainers of latter, which can
test every backend without switching PID 1 – people from other profiles
could help test it (as is the case in Artix; there are OpenRC users who use
Turnstile despite there being only runit and dinit backends).

Turnstile's developers also seem to be cooperative in regards to having
backends upstreamed and coordinating changes to them.

Instead, your proposals heavily burden the s6/s6-rc profile developers with
a mostly desktop-specific problem, task them with developing and
maintaining not only a complex and specialized set of services but also
custom PAM modules, and the result relies heavily on using s6-rc with
dynamic event sources, which it is not yet perfectly suited for.

> It also forces more assumptions on everybody, since we need to comply to
the exact way Turnstile solves things:
> Non daemontools-style supervisors, such as dinit (which is Turnstile's
> default), also don't provide a simple "setuidgid" privilege dropper,
> so they'll have little use for this functionality unless they add more
> dependencies or become more error-prone.
>
> This shows it quite nice, as it would have me need to align more to the
limitations of dinit, even though I do have the tools for the job.

The limitation is not because of dinit. It's part of the project's goals.
(If they really wanted to, they could clone setuidgid, as many people have
since daemontools came out, or accept the risk that comes with root
privileges).

The s6-rc backend you're requesting in that issue would be a special
snowflake that ignores the XDG_RUNTIME_DIR setting present in
turnstiled.conf in favor of one in a idiosyncratic master environment file
instead, and assumes a specific system service manager with a specific set
of services (most of which duplicating what Turnstile already provides).
Remember Turnstile backends should not care about what PID 1 is running.

There are only multiple config files to worry about keeping in sync, and
unavoidable integration issues that require constant intervention from the
maintainers of the s6 system service set, because you want to force the
project in a different role from what it was designed for: a one stop shop
desktop service daemon.

Since you're bumping against core design decisions in Turnstile, your setup
is better off relying on its own PAM stuff.

> There's two additional arguments: "$srv_dir", a folder that's
> dedicated to temporary files for the service manager (which seems to
> me to be the ideal place for s6-svscan's scan directory and s6-rc's
> live folder¹), and "$confdir", which is the path to where the
> configuration files reside (which can serve as the storage place for
> an environment file).
>
> It seems less then ideal to me to use the filesystem as a temporary
storage, when all that is needed are a handful environmental variables,
which can easily be passed to the backend.

I'm not sure what environment variables you are referring to. If you mean
storing XDG_RUNTIME_DIR and s6-rc's live directory there for the ready
directive, see below for why I don't think it's a good idea.

> Given dinit has the service management and service supervision layers
> in the same executable, Turnstile's authors probably expect the
> service management layer being up and running by the time 'run' has
> notified readiness, which is possibly why they didn't bother exporting
> variables in the "ready" script.
>
> Adding the ability to pass the env's to the ready function in the form of
a config option in turnstile.conf would probably be the best way to allow
elegant integration with s6-rc. I have opened an issue for that, see here:
https://github.com/chimera-linux/turnstile/issues/15

I don't think this is elegant, as it doesn't allow Turnstile to recognize a
failure of s6-rc-init and try again in that case. Remember backends are
supposed to run _service managers_.

This is probably irrelevant now, since it's clear Turnstile isn't what
you're looking for.

> I'd rather document that the user must edit the desktop service
> daemon's settings if they want to tweak how desktop services work.
>
> I technically agree, but some configs must be global, e.g.
${DBUS_SESSION_BUS_ADDRESS}, since other services, e.g. Pipewire, rely on
them.
>
> The alternative would be a dynamically populated and sourced env
directory, I am still thinking about this, but the issue I see is how to
propagate the required (currently ${DBUS_SESSION_BUS_ADDRESS} and
${XDG_RUNTIME_DIR}) to the user's login shell now that they are dynamic and
maybe not completely set on login?

Turnstile's PAM module does that, synchronizing these in the login process
to be the same ones as in the supervision tree the daemon spawns, and
unless you want to make users replace their shell with something like
`/etc/execline-startup` as described in
https://skarnet.org/lists/supervision/3126.html, your proposal is going to
need a custom PAM module for that too.

> As Laurent said (in regards to graphical user services, but I believe
> this applies here), the tradeoff is introducing environment data
> sharing and process lifetime management across unrelated process
> trees, which is neither.
>
> This might be subjective again, but I would not consider the user (and
thus his service-tree) be unrelated to the system he uses (and thus the
system-tree)

Under your design, the process that authenticates the user and has all the
information provided by PAM (or an eventual replacement) will be unrelated
to the user supervision tree (in the sense that the supervision tree will
not be forked off from it). This is a fact.

Turnstile requires the service backend not to be a oneshot to ensure the
supervision tree is a child of an actual login session.

> This brings me to my last point, an idea I got while writing this:
https://skarnet.org/lists/supervision/3131.html.
>
> The idea is the following:
>
> - Make a user service tree on boot (preferably using instantiated
services) if the user has an appropriate dir in ${XDG_DATA_HOME} (most
likely /home/${USER}/.local/.share/s6-rc/...). This service prepares all
the basics, e.g. ${XDG_RUNTIME_DIR}, the logging dir of the user services,
...
>
> Then:
>
> - Autostart all user-services in an "autostart" bundle the user populated.
>
> - Upon first login start a "login" bundle the user populated, stop it
after last logout. This can be fine grained with a bundle for ssh login, a
bundle for getty login,..., using appropriate PAM modules.
>
> - An env directory which is also used by the user-services shall be
sourced upon login.
>
> - Upon starting a graphical session start a sub-user-tree with the
s6-svscan process being a child of the wayland compositor,X server,...
>

The "per-login-type bundle" idea would require counting each type of login
independently. You also probably don't want to stop bundles, only keep
track of the necessary ones and `-up change` as events come, in order to
handle overlap among them.

Obarun Linux's 66 service manager (which is the one mentioned in the closed
Turnstile issue you first commented on) also works with s6 user supervision
trees spawned directly off of the system tree. You might want to study it.

I'll limit my comments to that, since you clearly plan to add complexity to
the service set and the service management layer in the way I predicted in
my previous message, and make XDG policies a deep-rooted default. We'll
likely just not agree again.

...and you're probably tired of me shilling Turnstile :P.

Have a nice Tuesday.

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 2110 bytes --]

Thanks for the extensive answer.

Since I feel like we are more and more fighting about each others 
wording and the answers consisting of quotations of each other get 
longer and longer, I will try and summarize what we have now.

I hope you are fine with this.


My approach(see here: https://skarnet.org/lists/supervision/3139.html):

- Forks the user-tree off the system-tree.

- Allows autostart of different bundles at different times (e.g. 
directly at boot time) / login types (e.g. on ssh login, on graphical 
login, ...).

- I still need to find/decide on a good solution for getting the env's 
to the user's shell.

- Needs direct maintenance of the people responsible for the s6/s6-rc 
overlay/profile.

- Adds the need for ~3 (when using s6-rc) system services to properly 
handle user service tree creation and instantiation.


Turnstile

- Forks the user-tree off the turnstile process, directly related to the 
login session.

- Austostarts the same user services on every login type, none on boot.

- Automatically exports env's to the user's shell.

- Needs close monitoring by the people responsible for the s6/s6-rc 
overlay/profile or at least good connection between them and the 
maintainer of the Turnstile ebuild, due to its tight integration.

- Handles user service tree creation itself.


Did I miss anything important?


Anyway, I will happily to continue this discussion, I think someone 
criticizing my ideas as strongly as you do, is the best way to bring 
their issues and flaws to light, so that I can solve them or even 
realize that I am on the wrong track alltogether.


> Obarun Linux's 66 service manager (which is the one mentioned in the closed
> Turnstile issue you first commented on) also works with s6 user supervision
> trees spawned directly off of the system tree. You might want to study it.
Thank you, I will look into that!


> ...and you're probably tired of me shilling Turnstile :P.
No, not at all, I have learned a lot from your criticisms. Seriously, 
keep going!


Have a nice Thursday!


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Carlos Eduardo]

[-- Attachment #1: Type: text/plain, Size: 3504 bytes --]

Em qui., 11 de jul. de 2024 às 11:55, Paul Sopka <psopka@sopka.ch> escreveu:
> I will try and summarize what we have now.
>
> I hope you are fine with this.

Sure. Long e-mails get tiring to read and reply to, for both of us.

> https://skarnet.org/lists/supervision/3139.html

Frankly, your priorities when designing a replacement were very misguided.

- `/run/turnstiled/sessions` exposes similar data as your proposal's
`/run/session` folder (including being able to query the session type).
Take a closer look at the prior art before writing it off.
- Just the session data isn't enough, as you need to patch third party
software that relies on it [3]. Turnstile (or, really, any proposal that
doesn't assume a specific setup of the underlying system) allows sharing
this load with other distributions.
- Without a daemon's mediation, processes writing to the database have to
coordinate themselves with locks¹. This introduces risks and limitations
you don't want to have when said writes are happening without human
oversight [1] [2]. Turnstile is a daemon for a good reason.
- Expecting the user to directly edit scripts if they want different
behavior, instead providing some degree of built-in configurability, is not
reasonable for a packaged solution, as even the smallest user tweaks would
be in constant attrition with package upgrades.
- I won't repeat myself on why a system that relies on calling the current
generation of s6-rc upon receiving events, or processes meant to be running
in different contexts sharing the same supervision tree, especially one
with only a boot-time environment, are a bad idea.

¹Which are missing from your script entirely, which is dangerous when you
have in-place editing such as the counter and newline files.

[1] https://skarnet.org/software/s6-rc/s6-rc.html (-b option).
[2] https://skarnet.org/lists/supervision/0391.html
[3] https://github.com/void-linux/void-packages/pull/44676

> Turnstile
>
> - Forks the user-tree off the turnstile process, directly related to the
> login session.

Not to _the_, but to _a_ login session. Whenever turnstiled is informed of
a log-in, it loads the PAM modules in /etc/pam.d/turnstiled, and runs
`backend run ...` as the shell. You can confirm that with a pstree;
dinit/s6-svscan are children of an intermediate "turnstiled" process, not
`login`/your display manager (see [1] for why replicating this directly on
top of a supervision suite is more trouble than it's worth).

[1] https://jdebp.uk/FGA/dont-abuse-su-for-dropping-privileges.html,
section "PAM changed everything".

> - Needs close monitoring by the people responsible for the s6/s6-rc
> overlay/profile or at least good connection between them and the
> maintainer of the Turnstile ebuild, due to its tight integration.

I've already explained why this is not true unless you want to force the
project out of its scope, and how it's an infinitely more accurate
description for your proposal.

Instead of reducing scope and relying on a loosely coupled third party
solution, you're making a solution that marries session tracking to a
specific policy for a specific init system. I wonder where I've seen that
before.

> Did I miss anything important?

I believe you're correct on the other points.

Since I've made my points here also regarding the "user session tracking"
thread, I won't make a standalone reply to it.

> Have a nice Thursday!

Have a nice Saturday :D

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Paul Sopka]

[-- Attachment #1: Type: text/plain, Size: 9521 bytes --]

I am resending this unsigned, since the message is too large otherwise.


> - `/run/turnstiled/sessions` exposes similar data as your proposal's
> `/run/session` folder (including being able to query the session type).
> Take a closer look at the prior art before writing it off.
Thank you for pointing that out. To my defense, this is neither 
mentioned on the Github page 
(https://github.com/chimera-linux/turnstile) nor in the manpage. Sorry 
if I have overlooked something. Anyway this is not what made me start 
creating this alternative.

My main reason is that the way my solution currently handles it, allows:

a) for different bundles to be started for different types of logins.

b) integrates better with my approach of handling the user service trees 
with instantiated s6/s6-rc services.

c) for less overhead and is way simpler e.g. not having to have a daemon 
running all the time.


> - Just the session data isn't enough, as you need to patch third party
> software that relies on it [3]. Turnstile (or, really, any proposal that
> doesn't assume a specific setup of the underlying system) allows sharing
> this load with other distributions.
> [3]https://github.com/void-linux/void-packages/pull/44676

Those are mostly patches to Turnstile, and changes to files in 
/etc/pam.d which, as config files, are distribution/system specific 
anyway. There is one patch that is not about the above is a patch to 
polikit. I need to investigate whether this is needed for my solution 
too, since my knowledge about polkit is limited.

I still need to test all the greeters, but agetty and ssh work like a 
charm. On a sidenote: I run everything that I am seriously proposing on 
my daily system. This includes the proposed session management script.


> - Expecting the user to directly edit scripts if they want different
> behavior, instead providing some degree of built-in configurability, is not
> reasonable for a packaged solution, as even the smallest user tweaks would
> be in constant attrition with package upgrades.
I completely agree, I have stated (twice) that this is a very crude 
version. I only posted this to see whether there are any fundamental 
flaws with the idea. The statements about extensibility, e.g.

> - Possibly create a /run/session/${USER} directory. 
Were more about what I am considering to add. Of course I will refine 
the script and add configurability. I am also thinking about turning it 
into a little PAM module itself to allow it to export environment 
variables to the user's shell.


> - Without a daemon's mediation, processes writing to the database have to
> coordinate themselves with locks¹. This introduces risks and limitations
> you don't want to have when said writes are happening without human
> oversight [1] [2]. Turnstile is a daemon for a good reason.
> [1]https://skarnet.org/software/s6-rc/s6-rc.html  (-b option).
That's why the option -b is here, the example of nested invocations 
should not be able to happen here, if I understood correctly.
> [2]https://skarnet.org/lists/supervision/0391.html
Doesn't s6-rc -b perfectly solve that for this situation? If not, why?


> - I won't repeat myself on why a system that relies on calling the current
> generation of s6-rc upon receiving events, or processes meant to be running
> in different contexts sharing the same supervision tree, especially one
> with only a boot-time environment, are a bad idea.

Go ahead and actually explain, please. You will not repeat yourself, 
since, up until now, you really only stated that you are considering it 
a bad idea.

> I've already explained why this is not true unless you want to force the
> project out of its scope, and how it's an infinitely more accurate
> description for your proposal.
Neither did you actually explain that. Please do.

I will go into detail on why I think the opposite is true:

If we have a group of maintainers responsible for the s6/s6-rc profile, 
they need to monitor s6/s6-rc closely. In addition to writing and fixing 
service scripts that is their job. My proposal, in it's current form, 
adds 4 additional scripts to that, which is not exactly "out of scope".

Relying on Turnstile, which to my standards is tightly integrated, 
requires quite a few things to be done in "the Turnstile way". I am not 
saying that the latter is bad! This on the other hand naturally requires 
synchronization between the Turnstile package and whatever provides the 
user service tree defaults. Which will result in either the Turnstile 
package having the same maintainers as the s6/s6-rc profile or at least 
requiring both maintaining groups to have a good connection.

I am not saying that any of this is necessarily bad, but it is causing 
more expenditure.

That said, even if it took a bit more effort to maintain my solution, 
this could be justified by the expanded functionality of per login type 
bundles and boot time user services.


> Instead of reducing scope and relying on a loosely coupled third party
> solution, you're making a solution that marries session tracking to a
> specific policy for a specific init system.

> [...] you're making a solution that marries session tracking to a
> specific policy[...]
In that sense doing it the way Turnstile does also "enforces" the 
"Turnstile policy".

I don't see where Turnstile has the advantage here, since it's "service 
manager/init system cross compatibility" results from service manager/ 
init system specific backend scripts which, in terms of complexity and 
length, are in a similar scope as my solution.


> [...] for a specific init system.
This is wrong. The usertree can be prepared by any init system, the 
script itself is tied to s6-rc for the user services, but the same goes 
for the Turnstile backend. From what I have seen of dinit, my script in 
its current form would require exactly two lines to be changed to be 
used with dinit based user services.


Finally, I want to address another point:

> Not to_the_, but to_a_  login session. Whenever turnstiled is informed of
> a log-in, it loads the PAM modules in /etc/pam.d/turnstiled, and runs
> `backend run ...` as the shell. You can confirm that with a pstree;
> dinit/s6-svscan are children of an intermediate "turnstiled" process, not
> `login`/your display manager (see [1] for why replicating this directly on
> top of a supervision suite is more trouble than it's worth).
>
> [1]https://jdebp.uk/FGA/dont-abuse-su-for-dropping-privileges.html,
> section "PAM changed everything".
And, from the mail before 
(https://skarnet.org/lists/supervision/3138.html):

> Under your design, the process that authenticates the user and has all 
> the information provided by PAM (or an eventual replacement) will be 
> unrelated to the user supervision tree (in the sense that the 
> supervision tree will not be forked off from it). This is a fact.
>
> Turnstile requires the service backend not to be a oneshot to ensure 
> the supervision tree is a child of an actual login session.
The more I think and learn about this, the more it appears merely as a 
cosmetic point:

Why should the user supervision tree being a child of the turnstiled 
process be any better than being a child of a "dynamic instance s6-svscan"?

Also, from the mail before 
(https://skarnet.org/lists/supervision/3138.html):

> Turnstile's PAM module does that, synchronizing these in the login 
> process to be the same ones as in the supervision tree the daemon 
> spawns, and unless you want to make users replace their shell with 
> something like `/etc/execline-startup` as described in 
> https://skarnet.org/lists/supervision/3126.html, your proposal is 
> going to need a custom PAM module for that too.

This is wrong. Turnstile only exports a couple of environment variables:

> The backend is a little helper program that can be written in any 
> language, it can e.g. be a shell script. It is started with a clean 
> environment with many of the common environment variables, such as 
> |HOME|, |USER|, |LOGNAME|, |SHELL|, |PATH| and others, freshly 
> initialized.
(from the Turnstile Github here: 
https://github.com/chimera-linux/turnstile)

As well as possibly ${XDG_RUNTIME_DIR} and ${DBUS_SESSION_BUS_ADDRESS}. 
One is once again limited to what Turnstile offers by default. Except by 
sourcing an env dir/file in the backend script but this causes exactly 
the synchronization needs that I want to avoid and that you criticized.

My proposal on the other hand, starts the user service tree from a 
simple s6 run script, being able to use the same env file/dir for the 
creation of e.g. ${XDG_RUNTIME_DIR} and as an environment passed to the 
user services as well as by a potential pam module that exports it's 
content to the users shell. This env file/dir can be freely changed and 
adapted by the sysadmin, since it is sourced as a whole.

> You can confirm that with a pstree;
> dinit/s6-svscan are children of an intermediate "turnstiled" process, not
> `login`/your display manager (see [1] for why replicating this directly on
> top of a supervision suite is more trouble than it's worth).
>
> [1]https://jdebp.uk/FGA/dont-abuse-su-for-dropping-privileges.html,
> section "PAM changed everything".
I don't need to replicate that, because I am already doing the same, 
just that the parent is "dynamic instance s6-svscan" instead of 
"turnstiled".

Sorry for citing the old mail, but these points are so closely related 
that I found it to be more complete in this way.


Have a nice week!


Paul

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 853 bytes --]

Hey,
I don't want to pick a side in the turnstile-or-not fight, so I
found something you seemed to agree on, to disagree with:

Paul Sopka schrob:
> > [...] and unless you want to make users replace their shell with
> > something like `/etc/execline-startup` as described in
> > https://skarnet.org/lists/supervision/3126.html, your proposal is going
> > to need a custom PAM module for that too.
> [...]
> My proposal [...]
> by a potential pam module that exports it's content to the users shell.

There's already pam_env to set environment variables. And from
pam_env.conf it can expand @{HOME} and presumably ${USER}, which should
be all you need to accomodate desktop stuff that expects XDG_* and
DBUS_SESSION_BUS_ADDRESS.
Everything else can and should be done via conventional places in the
file system, the way DJB intended. ;)

cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 488 bytes --]

> There's already pam_env to set environment variables. And from
> pam_env.conf it can expand @{HOME} and presumably ${USER}, which should
> be all you need to accomodate desktop stuff that expects XDG_* and

Thank you, this looks great!

Since it can also source a plain key=value file, it can source the same 
"config file" I source in the script setting up the user-service tree. 
Thus not creating the need to synchronize two config files.


Have a nice Monday!

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Carlos Eduardo]

[-- Attachment #1: Type: text/plain, Size: 909 bytes --]

Is there an equivalent chainloader ("envfile"-style) that can be plugged
into the instanced user service directories' command line?

Having the same configuration file be read by tools with different
behaviors (for instance, it appears the pam_env's quoting support is far
more limited than envfile's) may not lead to the same results.

Em seg., 15 de jul. de 2024 06:35, Paul Sopka <psopka@sopka.ch> escreveu:

> > There's already pam_env to set environment variables. And from
> > pam_env.conf it can expand @{HOME} and presumably ${USER}, which should
> > be all you need to accomodate desktop stuff that expects XDG_* and
>
> Thank you, this looks great!
>
> Since it can also source a plain key=value file, it can source the same
> "config file" I source in the script setting up the user-service tree.
> Thus not creating the need to synchronize two config files.
>
>
> Have a nice Monday!
>
> Paul
>
>

Re: s6/s6-rc policy for Gentoo: XDG Base Directory Specification

[Laurent Bercot]

>Is there an equivalent chainloader ("envfile"-style) that can be plugged
>into the instanced user service directories' command line?
>
>Having the same configuration file be read by tools with different
>behaviors (for instance, it appears the pam_env's quoting support is far
>more limited than envfile's) may not lead to the same results.

  I wanted envfile to be compatible with intuitive and expected
behaviour. If you write your files conservatively wrt what other tools
accept, you should be able to read them with envfile.

--
  Laurent