s6-rc user services on Gentoo

s6-rc user services on Gentoo

[Alexis]

Hi all,

Laurent has suggested i ask on IRC about this, but since (a) i'm 
in UTC+10, (b) i'm constantly moving between a few different 
networks, and (c) my bouncer is currently out of action (it's a 
Story), i thought i'd ask on these lists instead, to facilitate 
asynchronous replies. :-)

When i was on Void, a few years ago, i used s6+66 in place of 
runit. Since moving to Gentoo, i've been using OpenRC.

OpenRC currently doesn't have user services. They're gradually 
being worked on:

  https://github.com/OpenRC/openrc/pull/573

but the current state of things is that the contributor is 
rewriting the PR:

  https://github.com/OpenRC/openrc/pull/573#issuecomment-2007766162

i've had a lot on my plate for a while, but in more recent times 
i've had a bit more space to faff around with lower-priority 
stuff, and so i'm exploring setting up s6-rc to manage the user 
services i need. Once i've done so, i can:

* write a guide on the Gentoo wiki[a]; and

* provide service definitions for others to use via my personal 
  overlay[b].

which might help more people to become familiar with the s6 
ecosystem in general, and with s6-rc in particular (rather than 
trying to use s6 for service management as well as supervision).

However, i've reached an impasse, and i'm hoping some can tell me 
what i'm missing / what i'm doing wrong / what i'm 
misunderstanding.

Here's the situation:

* 
  S6RC_SERVICE_REPO=${HOME}/src/srht/gentoo-overlay/sys-apps/s6-rc-services/files/

* The above directory contains two service directories:
** dbus-session-bus/, containing:
*** type: longrun
*** notification-fd: 3
*** producer-for: dbus-session-bus-log
*** run:
    #!/usr/bin/execlineb -P
    importas HOME HOME
    redirfd -w 4 ${HOME}/.env.d/DBUS_SESSION_BUS_ADDRESS
    dbus-daemon --session --fork --nopidfile --print-address=4
*** finish:
    #!/usr/bin/execlineb -P
    importas HOME HOME
    rm -f ${HOME}/.env.d/DBUS_SESSION_BUS_ADDRESS
** dbus-session-bus-log/, containing:
*** type: longrun
*** notification-fd: 3
*** consumer-for: dbus-session-bus
*** pipeline-name: dbus-session-bus-pipeline
*** run:
    #!/usr/bin/execlineb -P
    importas HOME HOME
    exec -c
    s6-log -d3 -- t ${HOME}/.logs/s6-rc-services/dbus-session-bus

* SCANDIR="${XDG_DATA_HOME}/s6-rc/services"

* The above directory contains symlinks to

  ${S6RC_SERVICE_REPO}/{dbus-session-bus,dbus-session-bus-log}

* S6RC_COMPILED="${XDG_STATE_HOME}/s6-rc/compiled"

* Running:

  s6-rc-compile "${S6RC_COMPILED}" "${SCANDIR}"

  completes successfully.

* Running

  s6-svscan -- "${SCANDIR}"

  succeeds, with two s6-supervise processes created.

* HOWEVER, running

  s6-rc-init -c "${S6RC_COMPILED}" -l "${XDG_RUNTIME_DIR}/s6-rc" 
  "${SCANDIR}"

  fails, with the error message:

  unable to supervise service directories in 
  /run/user/1000/s6-rc/servicedirs: File exists

i've taken a look at s6rc_livedir_create.c:

  https://git.skarnet.org/cgi-bin/cgit.cgi/s6-rc/tree/src/libs6rc/s6rc_livedir_create.c

but nothing is leaping out to me as explaining what's happening.

Thoughts?


Alexis.

[a] The current s6 and s6-rc pages on the wiki have lot of detail, 
without any "quickstart" tutorials that might make it easy for 
people to get on board. A while ago i did a big restructure of the 
s6 article:

   https://wiki.gentoo.org/wiki/User:Flexibeast/drafts/S6

which i feel improves the current page:

   https://wiki.gentoo.org/wiki/S6

but which has been awaiting review by a more senior wiki editor 
than me.

The s6-rc page is similarly detailed and similarly lacking a 
"quickstart":

   https://wiki.gentoo.org/wiki/S6-rc

[b] Various people are providing s6 service definitions, 
e.g. https://codeberg.org/alecStewart1/gentoo-s6-scripts, but i'm 
not
currently aware of any providing s6-rc service definitions - 
pointers welcome. :-)

Re: s6-rc user services on Gentoo

[Hoël Bézier]

[-- Attachment #1: Type: text/plain, Size: 5160 bytes --]

Hi,

(resending this to the list as I only replied to Alexis, sorry Alexis for the 
double send.)

Am Tue, Apr 02, 2024 at 03:42:06PM +1100 schrieb Alexis:
>
>Hi all,
>
>Laurent has suggested i ask on IRC about this, but since (a) i'm
>in UTC+10, (b) i'm constantly moving between a few different
>networks, and (c) my bouncer is currently out of action (it's a
>Story), i thought i'd ask on these lists instead, to facilitate
>asynchronous replies. :-)

I feel you. IRC without bouncer is not always great for getting help on some 
topics. ^^’

> [snip]
>
>i've had a lot on my plate for a while, but in more recent times
>i've had a bit more space to faff around with lower-priority
>stuff, and so i'm exploring setting up s6-rc to manage the user
>services i need. Once i've done so, i can:
>
>* write a guide on the Gentoo wiki[a]; and
>
>* provide service definitions for others to use via my personal
>  overlay[b].
>
>which might help more people to become familiar with the s6
>ecosystem in general, and with s6-rc in particular (rather than
>trying to use s6 for service management as well as supervision).

Are you planning on having your user services supervisor itself supervised? And 
if so, by OpenRC or another instance of s6-svscan? Just asking out of 
curiosity.

>However, i've reached an impasse, and i'm hoping some can tell me
>what i'm missing / what i'm doing wrong / what i'm
>misunderstanding.
>
> [snip]
>
>* SCANDIR="${XDG_DATA_HOME}/s6-rc/services"
>
>* The above directory contains symlinks to
>
>  ${S6RC_SERVICE_REPO}/{dbus-session-bus,dbus-session-bus-log}
>
>* S6RC_COMPILED="${XDG_STATE_HOME}/s6-rc/compiled"
>
>* Running:
>
>  s6-rc-compile "${S6RC_COMPILED}" "${SCANDIR}"
>
>  completes successfully.
>
>* Running
>
>  s6-svscan -- "${SCANDIR}"
>
>  succeeds, with two s6-supervise processes created.
>
>* HOWEVER, running
>
>  s6-rc-init -c "${S6RC_COMPILED}" -l "${XDG_RUNTIME_DIR}/s6-rc"
>  "${SCANDIR}"
>
>  fails, with the error message:
>
>  unable to supervise service directories in
>  /run/user/1000/s6-rc/servicedirs: File exists

Your scandir and your service definitions are the same directory, which they 
should not be.

You currently have:
* service_definitions="${xdg_data_home}/s6-rc/services"
* compiled_services="${xdg_state_home}/s6-rc/compiled"
* scandir="${xdg_data_home}/s6-rc/services"
* livedir="${xdg_runtime_dir}/s6-rc"

However service definitions can not be used as a scandir: if your services are 
to be managed by s6-rc they should not be present at start in your scandir but 
should be added there by s6-rc-init.

That’s why s6-rc-init complains about files already existing: it tries to 
create a symlink in the scandir pointing to the services it’s managing but it 
can’t because the service are already present in the scandir.

Moreover, had you had any oneshots in your service definition directory, I 
think s6-svscan would have simply failed to start, or complained about 
directories containing no service (oneshots are not supervised by s6-svscan, 
because there’s nothing to supervise in them, so it doesn’t know about them).

What you should have is:
* service_definitions="${xdg_data_home}/s6-rc/services"
* compiled_services="${xdg_state_home}/s6-rc/compiled"
* scandir="${xdg_runtime_dir}/services"
* livedir="${xdg_runtime_dir}/s6-rc"

Your scandir should be in your runtime dir because it does not contain data but 
the current running configuration of your system. It can be an empty directory 
when you start s6-svscan, there is nothing wrong with that. s6-rc will populate 
it according to the compiled services you feed him.

>i've taken a look at s6rc_livedir_create.c:
>
>  https://git.skarnet.org/cgi-bin/cgit.cgi/s6-rc/tree/src/libs6rc/s6rc_livedir_create.c
>
>but nothing is leaping out to me as explaining what's happening.
>
>Thoughts?
>
>
>Alexis.
>
>[a] The current s6 and s6-rc pages on the wiki have lot of detail,
>without any "quickstart" tutorials that might make it easy for
>people to get on board. A while ago i did a big restructure of the
>s6 article:
>
>   https://wiki.gentoo.org/wiki/User:Flexibeast/drafts/S6
>
>which i feel improves the current page:
>
>   https://wiki.gentoo.org/wiki/S6
>
>but which has been awaiting review by a more senior wiki editor
>than me.
>
>The s6-rc page is similarly detailed and similarly lacking a
>"quickstart":
>
>   https://wiki.gentoo.org/wiki/S6-rc

Yeah, these definitely don’t help as much as they should, it’s a good thing 
someone decides on improving them, thanks for that. :)

>[b] Various people are providing s6 service definitions,
>e.g. https://codeberg.org/alecStewart1/gentoo-s6-scripts, but i'm
>not
>currently aware of any providing s6-rc service definitions -
>pointers welcome. :-)

You can see my own outdated s6-rc system service definitions at:
* https://forge.dotslashplay.it/s6-rc/system-services
and my (also outdated) s6-rc user tree service definitions at:
* https://forge.dotslashplay.it/s6-rc/personal-services

Hoël

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: s6-rc user services on Gentoo

[Guillermo]

Hello,

El mar, 2 abr 2024 a las 1:42, Alexis escribió:
>
>   S6RC_SERVICE_REPO=${HOME}/src/srht/gentoo-overlay/sys-apps/s6-rc-services/files/
>
> * The above directory contains two service directories:
> ** dbus-session-bus/, containing:
> *** type: longrun
> *** notification-fd: 3
> *** producer-for: dbus-session-bus-log
> *** run:
>     #!/usr/bin/execlineb -P
>     importas HOME HOME
>     redirfd -w 4 ${HOME}/.env.d/DBUS_SESSION_BUS_ADDRESS
>     dbus-daemon --session --fork --nopidfile --print-address=4

You already got an answer about why s6-rc-init doesn't work (the scan
directory and the directory of service definitions given to
s6-rc-compile should not be the same; let s6-rc-init populate the
service directory). I'd like to comment on what you didn't ask :)

1) Why are you telling dbus-daemon to --fork? That defeats the purpose
of service supervision. The service definition for the system-wide
message bus that I have on a Gentoo VM of mine with s6 + s6-rc +
s6-linux-init uses --nofork.

2) The presence of a notification-fd file tells s6 that dbus-daemon
can be somehow coerced into producing an s6-style readiness
notification using file descriptor 3 without changing its code, are
you sure that's the case with this script? My service definition for
the system-wide message bus polls for readiness using s6-notifyoncheck
and a dbus-send command...

> [a] The current s6 and s6-rc pages on the wiki have lot of detail,
> without any "quickstart" tutorials that might make it easy for
> people to get on board.

Most of the content is from around 2017, a time in which the s6 suite
was less known, information in places other than the skarnet.org
website was lacking and inaccurate —despite official documentation,
while being short and to the point, has always been quite good and
complete IMO; it seemed people just didn't bother reading it? [1]—,
and there weren't many usage examples around.

[1] I think I never said it, but what made me look at s6 for the first
time (s6-rc and s6-linux-init didn't exist back then), after seeing a
post in the Gentoo forums, was its documentation ("OK, nice
explanations, let's try this").

Re: s6-rc user services on Gentoo

[Alexis]

Hi Hoël,

Thanks for your reply!

Hoël Bézier <hoelbezier@riseup.net> writes:

> I feel you. IRC without bouncer is not always great for getting 
> help
> on some topics. ^^’

:-) Yeah, most of the time that i'm on IRC - which is rarely 
nowadays, due to the aforementioned "lots on my plate" issue - 
it's usually to try to offer help to others, rather than asking 
for help myself, where having a working bouncer is much less of an 
issue.

> Are you planning on having your user services supervisor itself
> supervised? And if so, by OpenRC or another instance of 
> s6-svscan?
> Just asking out of curiosity.

i hadn't really thought much about that, as i've been focusing on 
simply trying to understand the basics and get them working.

My initial thinking is that i'd start by setting up the user 
services tree from .zlogin, i.e. the s6-svscan process itself 
would be unsupervised. Part of the reason for doing so is that, at 
this point, i primarily want to be using s6-rc to be able to 
manually restart user services in various situations (e.g. config 
changes), with service availability being secondary to that. But 
another part of the reason is that, in these contexts, i generally 
work best from the bottom-up, starting from what i consider to be 
the 'core' and then 'working outwards' from that.

(Yes, i know that's not always the most appropriate approach, 
particularly in contexts where security topics are inherently part 
of the 'core', and can't just be "bolted on" afterwards. i don't 
feel this is particularly the case here.)

i guess i'd probably want OpenRC supervising the user s6-svscan 
process, mainly because, at this point, i'd like to keep using 
OpenRC on my Gentoo system overall: since a lot of my volunteer 
tech work involves providing support to others and 
writing/maintaining documentation, it's useful to keep my own 
system fundamentally similar to what others are running.

> Your scandir and your service definitions are the same 
> directory,
> which they should not be.

> You currently have:
> * service_definitions="${xdg_data_home}/s6-rc/services"
> * compiled_services="${xdg_state_home}/s6-rc/compiled"
> * scandir="${xdg_data_home}/s6-rc/services"
> * livedir="${xdg_runtime_dir}/s6-rc"
>
> However service definitions can not be used as a scandir: if 
> your
> services are to be managed by s6-rc they should not be present 
> at
> start in your scandir but should be added there by s6-rc-init.
>
> That’s why s6-rc-init complains about files already existing: it 
> tries
> to create a symlink in the scandir pointing to the services it’s
> managing but it can’t because the service are already present in 
> the
> scandir.

Ah! Now i get it. Thank you. :-) i'm not sure how my reading of 
the s6-rc-init documentation led me to this; i'll need to review 
it, so that i can perhaps offer some changes/additions. :-)

> Moreover, had you had any oneshots in your service definition
> directory, I think s6-svscan would have simply failed to start, 
> or
> complained about directories containing no service (oneshots are 
> not
> supervised by s6-svscan, because there’s nothing to supervise in 
> them,
> so it doesn’t know about them).

*nod*

> What you should have is:
> * service_definitions="${xdg_data_home}/s6-rc/services"
> * compiled_services="${xdg_state_home}/s6-rc/compiled"
> * scandir="${xdg_runtime_dir}/services"
> * livedir="${xdg_runtime_dir}/s6-rc"
>
> Your scandir should be in your runtime dir because it does not 
> contain
> data but the current running configuration of your system. It 
> can be
> an empty directory when you start s6-svscan, there is nothing 
> wrong
> with that. s6-rc will populate it according to the compiled 
> services
> you feed him.

*nod*

Thanks for all this, it's been very helpful!

> You can see my own outdated s6-rc system service definitions at:
> * https://forge.dotslashplay.it/s6-rc/system-services
> and my (also outdated) s6-rc user tree service definitions at:
> * https://forge.dotslashplay.it/s6-rc/personal-services

i'll definitely take a look!

> (resending this to the list as I only replied to Alexis, sorry 
> Alexis for the double send.)

No worries. :-)


Alexis.

Re: s6-rc user services on Gentoo

[Alexis]

Guillermo <gdiazhartusch@gmail.com> writes:

> 1) Why are you telling dbus-daemon to --fork? That defeats the 
> purpose
> of service supervision. The service definition for the 
> system-wide
> message bus that I have on a Gentoo VM of mine with s6 + s6-rc +
> s6-linux-init uses --nofork.

> 2) The presence of a notification-fd file tells s6 that 
> dbus-daemon
> can be somehow coerced into producing an s6-style readiness
> notification using file descriptor 3 without changing its code, 
> are
> you sure that's the case with this script? My service definition 
> for
> the system-wide message bus polls for readiness using 
> s6-notifyoncheck
> and a dbus-send command...

These are both very reasonable questions, with the same answer: i 
hadn't got as far as actually properly setting up the services 
yet, but was merely creating 'placeholders' to (as i described in 
my email to Hoël) get the basic overall setup working. So the 
`dbus-daemon` command is literally just copy-and-pasted from what 
i currently use in my .zlogin to start a session bus. My 
apologies; i should have made this clear in my original post.

Re. 1, i'm certainly aware that forking is undesirable in this 
context; cf. a comment of mine i posted yesterday, the second one 
in this comments section: 
https://utcc.utoronto.ca/~cks/space/blog/linux/SystemdSocketActivationThoughts?showcomments#comments

Re. 2, off the top of my head, i imagine you're correct; i haven't 
yet thought about it / looked at it in any detail.

> Most of the content is from around 2017, a time in which the s6 
> suite
> was less known, information in places other than the skarnet.org
> website was lacking and inaccurate —despite official 
> documentation,
> while being short and to the point, has always been quite good 
> and
> complete IMO; it seemed people just didn't bother reading it? 
> [1]—,
> and there weren't many usage examples around.
>
> [1] I think I never said it, but what made me look at s6 for the 
> first
> time (s6-rc and s6-linux-init didn't exist back then), after 
> seeing a
> post in the Gentoo forums, was its documentation ("OK, nice
> explanations, let's try this").

*nod* Fair enough! Yeah, as i wrote in the first comment in the 
 comments section i linked to above:

> [A]s someone who spends a lot of time writing and maintaining 
> documentation, i know that people apparently don't like reading 
> the documentation. :-)

At the same time, i know that many people (including myself!) can 
sometimes struggle to "put all the pieces" together (either 
properly, or at all), even with excellent reference docs (as in 
the case of s6), in the absence of some basic, concrete, "actual 
implementation" examples. Like, the mdoc(7) man page is excellent 
and comprehensive, but to help people to get a sense of how easy 
it can be to use mdoc(7) in practice, and hopefully encourage them 
to use it, i wrote a quickstart guide:

  https://github.com/flexibeast/guides/blob/master/mdoc-quickstart.md

Which is the sort of thing i'm hoping to do for s6-rc. :-)


Alexis.

Re: s6-rc user services on Gentoo

[Laurent Bercot]

>2) The presence of a notification-fd file tells s6 that dbus-daemon
>can be somehow coerced into producing an s6-style readiness
>notification using file descriptor 3 without changing its code, are
>you sure that's the case with this script? My service definition for
>the system-wide message bus polls for readiness using s6-notifyoncheck
>and a dbus-send command...

  "dbus-daemon --print-address=3" should produce a suitable notification.
The address of the bus can only be printed once the bus exists. :)

--
  Laurent

Re: s6-rc user services on Gentoo

[Guillermo]

El mié, 3 abr 2024 a las 8:37, Laurent Bercot escribió:
>
> >2) The presence of a notification-fd file tells s6 that dbus-daemon
> >can be somehow coerced into producing an s6-style readiness
> >notification using file descriptor 3 without changing its code, are
> >you sure that's the case with this script? My service definition for
> >the system-wide message bus polls for readiness using s6-notifyoncheck
> >and a dbus-send command...
>
>   "dbus-daemon --print-address=3" should produce a suitable notification.
> The address of the bus can only be printed once the bus exists. :)

But then, there is a problem if one actually wants the server address
information that --print-address provides. Alexis' 'run' script for
example wants to save that to a file (apparently in a directory
suitable for s6-envdir). If the output is sent to the notification
pipe instead, s6-supervise will 'eat' it while waiting for the final
newline character, and then the information is lost.

And more generally, there's also the question about how 'ready'
dbus-daemon actually is when the point in its code that prints the
server address is reached. I can't really say without looking at the
code; dbus-daemon has many duties once its UNIX domain socket is
bound.

G.

Re: s6-rc user services on Gentoo

[Laurent Bercot]

>But then, there is a problem if one actually wants the server address
>information that --print-address provides. Alexis' 'run' script for
>example wants to save that to a file (apparently in a directory
>suitable for s6-envdir). If the output is sent to the notification
>pipe instead, s6-supervise will 'eat' it while waiting for the final
>newline character, and then the information is lost.

  That is true. The option is only usable as readiness notification if 
you
don't otherwise need the information.


>And more generally, there's also the question about how 'ready'
>dbus-daemon actually is when the point in its code that prints the
>server address is reached. I can't really say without looking at the
>code; dbus-daemon has many duties once its UNIX domain socket is
>bound.

  That doesn't matter. The important thing with readiness is that clients
need to be able to send messages to the daemon. As soon as the socket is
bound, it's okay; if the daemon hasn't finished initializing yet, then
that's what kernel socket buffers are for.

  Of course, if the daemon still needs to allocate resources and can 
*fail*
after binding the socket and printing the line, that's not good, but 
well,
it's a balance between reliability and simplicity.

--
  Laurent

Re: s6-rc user services on Gentoo

[Muhammad Mahendra Subrata]

[-- Attachment #1: Type: text/plain, Size: 1136 bytes --]

On Sat, Apr 6, 2024, 19:43 Guillermo <gdiazhartusch@gmail.com> wrote:

> But then, there is a problem if one actually wants the server address
> information that --print-address provides. Alexis' 'run' script for
> example wants to save that to a file (apparently in a directory
> suitable for s6-envdir). If the output is sent to the notification
> pipe instead, s6-supervise will 'eat' it while waiting for the final
> newline character, and then the information is lost.
>
> And more generally, there's also the question about how 'ready'
> dbus-daemon actually is when the point in its code that prints the
> server address is reached. I can't really say without looking at the
> code; dbus-daemon has many duties once its UNIX domain socket is
> bound.
>
> G.
>

I have been using `--print-pid=3` as readiness notification for dbus-daemon
for quite a while now on my user services and I haven't had any problems
with it so far. IIRC, looking at dbus-daemon code, it actually prints the
socket address first then its pid. So, I use the `--print-address=` option
to save the socket address to a file for consumption by s6-envdir.

>

Re: s6-rc user services on Gentoo

[Laurent Bercot]

>I have been using `--print-pid=3` as readiness notification for dbus-daemon
>for quite a while now on my user services and I haven't had any problems
>with it so far. IIRC, looking at dbus-daemon code, it actually prints the
>socket address first then its pid. So, I use the `--print-address=` option
>to save the socket address to a file for consumption by s6-envdir.

  Interesting. But it's also an implementation detail, because 
dbus-daemon
could print its pid at any time, no matter its readiness state. That's 
why
I chose to go with --print-address - but of course reading the code 
instead
of speculating is the right thing to do :)

--
  Laurent