Have an external script wait for a oneshot service

Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 457 bytes --]

Hey everyone

My question is basically what the subject says:
Is there a way for an external script to know when an s6-rc oneshot 
finished and is in "up" state?
It is simple enough of course to directly script this with s6-fifodirs,
but I wondered if there is already similar functionality (I could not 
find) in s6-rc.
If not, it would be a nice feature for s6-rc to also have status files 
and event fifodirs for oneshots.

Regards

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Charles Cazabon via supervision]

Paul Sopka <psopka@sopka.ch> wrote:
>
> If not, it would be a nice feature for s6-rc to also have status files and
> event fifodirs for oneshots.

I could be mistaken, but I thought one-shots were specifically designed for
fire-and-forget use, and that if you want to monitor it that way, you should
use a regular service.

Charles




-- 
------------------------------------------------------------------
Charles Cazabon       <charlesc-supervision-skarnet.org@pyropus.ca>
Software, consulting, and services available at http://pyropus.ca/
------------------------------------------------------------------

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 1120 bytes --]

Thank you for your answer!
> I could be mistaken, but I thought one-shots were specifically designed for
> fire-and-forget use,

I would argue that the following speaks against this:

- s6-rc tracks the state of oneshots already in that you can not run
    "s6-rc start <oneshot>" multiple times without "s6-rc stop 
<oneshot>" in between.

- Longrun service A can depend on a oneshot B in that it is only ran 
once B is finished.

- It is useful to know whether and when a oneshot task is finished.

By the above I would argue that oneshots are on equal footing with
and of equal importance as longruns, not just "fire-and-forget" and
therefore would profit from an equally powerful readiness api.

> and that if you want to monitor it that way, you should
> use a regular service.
This is not possible, since what I depend on is a script that does its 
thing and exits.
I would need to add a "sleep infinity" or better, a posix compatible 
alternative at its end,
which would cause a useless dangling longrun being supervised and logged 
- a waste of resources.

Regards

Paul Sopka


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Brett Neumeier via supervision]

On Wed, 4 Dec 2024 16:28:01 +0100
Paul Sopka <psopka@sopka.ch> wrote:
> My question is basically what the subject says:
> Is there a way for an external script to know when an s6-rc oneshot 
> finished and is in "up" state?

You could use `s6-rc -a list`, perhaps? It lists all services that are in the active state.

For example, I have a oneshot service called `swap` that turns on or off swap volumes.

# s6-rc -a list | grep swap
swap
# s6-rc stop swap
# s6-rc -a list | grep swap
#

I could run `grep -q swap` and then the exit status would tell me if it's running or not.

Does that give you what you want?

-- 
Brett Neumeier <random@freesa.org>

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 587 bytes --]

> You could use `s6-rc -a list`, perhaps? It lists all services that are in the active state.
>
> For example, I have a oneshot service called `swap` that turns on or off swap volumes.
>
> # s6-rc -a list | grep swap
> swap
> # s6-rc stop swap
> # s6-rc -a list | grep swap
> #
>
> I could run `grep -q swap` and then the exit status would tell me if it's running or not.
>
> Does that give you what you want?
Unfortunately not, since I specifically need to *wait*
and waiting based on "state-info" would require polling,
which I would rather avoid.

Regards

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Tanuj Bagaria]

unsubscribe

> On 5 Dec 2024, at 00:00, Brett Neumeier via supervision <supervision@list.skarnet.org> wrote:
> 
> On Wed, 4 Dec 2024 16:28:01 +0100
> Paul Sopka <psopka@sopka.ch> wrote:
>> My question is basically what the subject says:
>> Is there a way for an external script to know when an s6-rc oneshot
>> finished and is in "up" state?
> 
> You could use `s6-rc -a list`, perhaps? It lists all services that are in the active state.
> 
> For example, I have a oneshot service called `swap` that turns on or off swap volumes.
> 
> # s6-rc -a list | grep swap
> swap
> # s6-rc stop swap
> # s6-rc -a list | grep swap
> #
> 
> I could run `grep -q swap` and then the exit status would tell me if it's running or not.
> 
> Does that give you what you want?
> 
> --
> Brett Neumeier <random@freesa.org>

Re: Have an external script wait for a oneshot service

[Brett Neumeier via supervision]

On Wed, 4 Dec 2024 21:05:45 +0100
Paul Sopka <psopka@sopka.ch> wrote:
> > You could use `s6-rc -a list`, perhaps? It lists all services that are in the active state.
> >[...]
> > Does that give you what you want?
> Unfortunately not, since I specifically need to *wait*
> and waiting based on "state-info" would require polling,
> which I would rather avoid.

Ah, I see: from your external script, you do not want to know *whether* a oneshot service has completed; you want to block *unless or until* it has completed?

In that case, I would use `s6-ftrig-notify` in the oneshot service, and both `s6-rc -a list` and `s6-ftrig-wait` in the external script:

in the oneshot, as the last action taken before quitting, use `s6-ftrig-notify` to send an event to a fifodir;
in the external script, `s6-rc -a list | grep -q $(oneshot_service_name)` to proceed if the oneshot is already up, and then if it is not up, `s6-ftrig-wait` to block until it is.

Or, as you say, you can use fifodirs directly without the convenience programs that interact with them already.

There might be a race condition there, but that's something you can surely work around if it's a concern.

-- 
Brett Neumeier <random@freesa.org>

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 1043 bytes --]

> Ah, I see: from your external script, you do not want to know*whether* a oneshot service has completed; you want to block*unless or until* it has completed?
>
> In that case, I would use `s6-ftrig-notify` in the oneshot service, and both `s6-rc -a list` and `s6-ftrig-wait` in the external script:
>
> in the oneshot, as the last action taken before quitting, use `s6-ftrig-notify` to send an event to a fifodir;
> in the external script, `s6-rc -a list | grep -q $(oneshot_service_name)` to proceed if the oneshot is already up, and then if it is not up, `s6-ftrig-wait` to block until it is.
>
> Or, as you say, you can use fifodirs directly without the convenience programs that interact with them already.
>
> There might be a race condition there, but that's something you can surely work around if it's a concern.

Exactly!

Although it would be nicer to have a standard way to do this,
consistent with longruns, as described in my initial Mail.

Let's see what Laurent thinks of that.

Thanks and regards

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Hoël Bézier]

[-- Attachment #1: Type: text/plain, Size: 486 bytes --]

Am Mi, Dez 04, 2024 am 06:12:07 +0100 schrieb Paul Sopka:
>> and that if you want to monitor it that way, you should
>> use a regular service.
>This is not possible, since what I depend on is a script that does its
>thing and exits.
>I would need to add a "sleep infinity" or better, a posix compatible
>alternative at its end,
>which would cause a useless dangling longrun being supervised and logged
>- a waste of resources.
Why don’t you use another oneshot? ^^

Hoël

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re[2]: Have an external script wait for a oneshot service

[Laurent Bercot]

>Although it would be nicer to have a standard way to do this

  There is a standard way to block until a oneshot has completed:
depend on that oneshot.

  Duh.

  The thing is, s6-rc wasn't designed to have external scripts
interacting in this way with this engine, because if you have programs
depending on intermediary s6-rc states, they should be managed by s6-rc
as well. The point is to have a predictable machine state when s6-rc
exits; if you have random other stuff doing things in parallel, that
defeats the purpose.

  And with an s6-linux-init setup, at boot time there is no other
process than s6-rc managing your services. You have the supervision
tree, and the stage 2 script, and that's it. And if you're going to
spawn stuff in your rc.init that waits for oneshot X in the s6-rc db,
then... why? Why not have said stuff as another service?

  If you cannot, for whatever reason, maybe bring your services up in
two steps?
      s6-rc change X && stuff & s6-rc change default

  And if all else fails, the generic solution would be to have a oneshot
Y depending on X (so, that would run once X is done) that sends a
notification to a place of your choice via a mechanism of your choice,
you could use a fifodir and s6-ftrig-notify for this. You would still
have to deal with the race conditions around setting up the listener 
etc.

  The cleanest way to achieve what you want, without support in the
service set itself, is to s6-rc change X first, then s6-rc change the
rest of your services. That way, you avoid breaking abstraction with
hacks of questionable aesthetic value.

--
  Laurent

Re: Have an external script wait for a oneshot service

[Tanuj Bagaria]

unsubscribe

> On 5 Dec 2024, at 01:58, Laurent Bercot <ska-supervision@skarnet.org> wrote:
> 
> 
>> 
>> Although it would be nicer to have a standard way to do this
> 
> There is a standard way to block until a oneshot has completed:
> depend on that oneshot.
> 
> Duh.
> 
> The thing is, s6-rc wasn't designed to have external scripts
> interacting in this way with this engine, because if you have programs
> depending on intermediary s6-rc states, they should be managed by s6-rc
> as well. The point is to have a predictable machine state when s6-rc
> exits; if you have random other stuff doing things in parallel, that
> defeats the purpose.
> 
> And with an s6-linux-init setup, at boot time there is no other
> process than s6-rc managing your services. You have the supervision
> tree, and the stage 2 script, and that's it. And if you're going to
> spawn stuff in your rc.init that waits for oneshot X in the s6-rc db,
> then... why? Why not have said stuff as another service?
> 
> If you cannot, for whatever reason, maybe bring your services up in
> two steps?
>     s6-rc change X && stuff & s6-rc change default
> 
> And if all else fails, the generic solution would be to have a oneshot
> Y depending on X (so, that would run once X is done) that sends a
> notification to a place of your choice via a mechanism of your choice,
> you could use a fifodir and s6-ftrig-notify for this. You would still
> have to deal with the race conditions around setting up the listener etc.
> 
> The cleanest way to achieve what you want, without support in the
> service set itself, is to s6-rc change X first, then s6-rc change the
> rest of your services. That way, you avoid breaking abstraction with
> hacks of questionable aesthetic value.
> 
> --
> Laurent
> 

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 3621 bytes --]

>>> and that if you want to monitor it that way, you should
>>> use a regular service.
>> This is not possible, since what I depend on is a script that does its
>> thing and exits.
>> I would need to add a "sleep infinity" or better, a posix compatible
>> alternative at its end,
>> which would cause a useless dangling longrun being supervised and logged
>> - a waste of resources.
> Why don’t you use another oneshot? ^^
>
> Hoël 

>  The thing is, s6-rc wasn't designed to have external scripts
> interacting in this way with this engine, because if you have programs
> depending on intermediary s6-rc states, they should be managed by s6-rc
> as well. The point is to have a predictable machine state when s6-rc
> exits; if you have random other stuff doing things in parallel, that
> defeats the purpose.
>
>  And with an s6-linux-init setup, at boot time there is no other
> process than s6-rc managing your services. You have the supervision
> tree, and the stage 2 script, and that's it. And if you're going to
> spawn stuff in your rc.init that waits for oneshot X in the s6-rc db,
> then... why? Why not have said stuff as another service?
>
>  If you cannot, for whatever reason, maybe bring your services up in
> two steps?
>      s6-rc change X && stuff & s6-rc change default
>
>  And if all else fails, the generic solution would be to have a oneshot
> Y depending on X (so, that would run once X is done) that sends a
> notification to a place of your choice via a mechanism of your choice,
> you could use a fifodir and s6-ftrig-notify for this. You would still
> have to deal with the race conditions around setting up the listener etc.
>
>  The cleanest way to achieve what you want, without support in the
> service set itself, is to s6-rc change X first, then s6-rc change the
> rest of your services. That way, you avoid breaking abstraction with
> hacks of questionable aesthetic value. 
I wholeheartedly agree to this in respect to init.
But due to my foolish attempt to keep the discussion more general,
there is a confusion here. This is not about anything close to init,
But software that can not be part of the supervision tree:
software that's timing is controlled by the user and that is part of the 
interface.
To be more precise, I have the following setup:

- A longrun L setting up s6-svscan for a user-service-tree.

- A oneshot O setting up "s6-rc-init" and "s6-rc start default".

(two allow users to have services running on boot without login)

- A script run on login (bee it by PAM, .profile or another way),
   that starts user services only useful after login, e.g. pipewire.

Now of course the script should only be ran after the the "s6-rc-init" 
oneshot.
The easy way out of this is to have all "interface-software",
e.g. agetty, greetd, sshd, etc. depend on a bundle containing L and O of 
each user.
But I would argue that this is not an elegant solution:

- A user might not have setup user-services,
   why should he have to wait for all the other user-service-trees to be 
prepared?

- Even a user with user-services set up would have to wait
   for all the other user-trees to be prepared too, which might take 
longer to start than his.

Now if the script had a nice way to wait for O of the user logging in
to be started, the login software would not need to wait for O itself and
the waiting time would only affect the user starting slow starting 
user-services.

I argue that this is more elegant.

A readiness api for oneshots would help with this.

Regards

Paul


[-- Attachment #1.1.1.2: Type: text/html, Size: 5291 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re[2]: Have an external script wait for a oneshot service

[Laurent Bercot]

>To be more precise, I have the following setup:
>- A longrun L setting up s6-svscan for a user-service-tree.
>
>- A oneshot O setting up "s6-rc-init" and "s6-rc start default".
>
>(two allow users to have services running on boot without login)
>
>- A script run on login (bee it by PAM, .profile or another way),
>   that starts user services only useful after login, e.g. pipewire.
>
>Now of course the script should only be ran after the the "s6-rc-init" 
>oneshot.

  I get the feeling there's an XY problem here, or some confusion (but
maybe just in my own brain) because user services are an underspecified
concept. Can you please describe the details of your setup: at what
time do you want to start up a supervision tree and an s6-rc-init
process for a given user?
  - at boot time? if so, how do you decide what users to start a
tree for?
  - at login time? if so, you have a script that you run at login time,
I don't see the problem here: start the tree and the s6-rc-init in that
script, not as a part of a set of s6-rc services.

--
  Laurent

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 5086 bytes --]

On 05.12.24 09:05, Laurent Bercot wrote:
>
>> To be more precise, I have the following setup:
>> - A longrun L setting up s6-svscan for a user-service-tree.
>>
>> - A oneshot O setting up "s6-rc-init" and "s6-rc start default".
>>
>> (two allow users to have services running on boot without login)
>>
>> - A script run on login (bee it by PAM, .profile or another way),
>>   that starts user services only useful after login, e.g. pipewire.
>>
>> Now of course the script should only be ran after the the 
>> "s6-rc-init" oneshot.
>
>  I get the feeling there's an XY problem here, or some confusion (but
> maybe just in my own brain) because user services are an underspecified
> concept. Can you please describe the details of your setup: at what
> time do you want to start up a supervision tree and an s6-rc-init
> process for a given user?
>  - at boot time? if so, how do you decide what users to start a
> tree for?
>  - at login time? if so, you have a script that you run at login time,
> I don't see the problem here: start the tree and the s6-rc-init in that
> script, not as a part of a set of s6-rc services.
>
> -- 
>  Laurent
>
After the discussions about user services conducted here a couple of 
months ago,
I came to the conclusion that the following would be the best solution:

- All user-trees are started at boot time.

- The user-trees are be prepared in two ways:

     1    One system-longrun service per user:
     - 1a     Have the admin create one system-longrun running 
"s6-svscan" per user (e.g. automated via a script).
     - 1b     Have the admin create one system-oneshot per user to:
         - 1bi     Prepare the live directory for the user-tree 
(preferably under /run/user/${USER}).
         - 1bii     Start "s6-rc-init" and "s6-rc start default".

     2    One instantiated system-longrun for all users:
     - 2a    Have a system-longrun which is an instantiated "s6-svscan", 
with one instance per user
               and a (root controlled) config C specifying which users 
to instantiate for.
     - 2b    This config C shall then be parsed by an additional 
system-oneshot to do the actual instantiation.
     - 2c    A third system-oneshot does:
         - 2ci    Prepare the live directory for the user-tree for each 
user specified in config C (preferably under /run/user/${USER}).
         - 2cii    Run "s6-rc-init" and "s6-rc start default" for each 
user specified in config C.
       (this can be merged with the instantiation system-oneshot if 
desired.)

- Upon login, a login script L shall invoke an "s6-rc start login" with 
("login" being a bundle) for the user logging in.
   (this can be further extended using PAM to run "s6-rc start 
${PAM_TYPE}" to differentiate different types of logins)
   The script L can be started by PAM, .profile or in some other way.

- Upon last logout, meaning the logout that causes no logged in sessions 
for the user to be left,
   the script invokes "s6-rc -pu change default" to stop all the login 
services.

This has the following advantages:

- A user can have always running user services (e.g. snooze based timed 
services, calculations, ...)
   no matter whether the user is logged in.

- Services only required during login (e.g. pipewire, mpd, ...) only run 
when the user is logged in.

- Who gets to have a user-tree is decided by root creating the 
user-services as in 1 or instantiation config C as in 2,
   which feels natural, since root also decides who gets a user account 
in the first place.

And the following implications:

- The login script needs to wait for "s6-lrc-init" to be finished before 
it can call "s6-rc start login".
   It hence needs to wait for the corresponding system-oneshot 1b/2c.

Which (as far as I have found) can be solved in one of the following ways:

- S1    Script s6-fifodir functionality in bot the system-oneshot 1b/2c 
and the script L.

- S2    Add a readiness api for s6-rc oneshots like s6 longruns have and 
wait for that of system-oneshot 1b/2c in the script.

- S3    Have the script L do the same as system-oneshot 1b/2c:
         - 1bi/2ci    Prepare the live directory for the user-tree 
(preferably under /run/user/${USER}).
         - 1bii/2cii    Start "s6-rc-init -b" and "s6-rc start login".
   Utilizing blocking locks and tests to work parallel to system-oneshot 
1b/2c.
   This can only be done if the script is ran through PAM, since 1bi/2ci 
requires root privileges.

Since I find solution S2 to be the most elegant,
I wanted to ask whether you are willing to add a proper readiness api to 
oneshots too.
This would generally be a nice addition, as I am sure there are other 
applications to this feature
I can currently not imagine.
Further, I argue that it would make the system more consistent, since 
longruns have this functionality too.

Regards

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re[2]: Have an external script wait for a oneshot service

[Laurent Bercot]

>     - 2c    A third system-oneshot does:
>         - 2ci    Prepare the live directory for the user-tree for each user specified in config C (preferably under /run/user/${USER}).
>         - 2cii    Run "s6-rc-init" and "s6-rc start default" for each user specified in config C.
>       (this can be merged with the instantiation system-oneshot if desired.)

  Okay. My point is, why do you need that "default" bundle here? Why do
you want an s6-rc infrastructure active when the user isn't logged in?

  I can understand a permanent supervision tree per user, but a permanent
service set that includes oneshots feels overengineered to me. What
problem is this supposed to solve?


>- Upon login, a login script L shall invoke an "s6-rc start login" with ("login" being a bundle) for the user logging in.
>   (this can be further extended using PAM to run "s6-rc start ${PAM_TYPE}" to differentiate different types of logins)
>   The script L can be started by PAM, .profile or in some other way.

  My advice is to do your whole 2c point at login time. The script L can
perform XDG stuff, s6-rc-init, and s6-rc start default. On last logout,
you stop everything with "s6-rc -da change" and you delete the livedir.
Only the supervision tree remains, maintaining whatever services the
user has manually added.

  I don't think there is a good argument for keeping state (which is
what oneshots do) in user services when the user is logged out. Feel
free to give counterexamples though.

--
  Laurent

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 3861 bytes --]

>>     - 2c    A third system-oneshot does:
>>         - 2ci    Prepare the live directory for the user-tree for 
>> each user specified in config C (preferably under /run/user/${USER}).
>>         - 2cii    Run "s6-rc-init" and "s6-rc start default" for each 
>> user specified in config C.
>>       (this can be merged with the instantiation system-oneshot if 
>> desired.)
>
>  Okay. My point is, why do you need that "default" bundle here? Why do
> you want an s6-rc infrastructure active when the user isn't logged in?
>
>  I can understand a permanent supervision tree per user, but a permanent
> service set that includes oneshots feels overengineered to me. What
> problem is this supposed to solve?
>
>
>> - Upon login, a login script L shall invoke an "s6-rc start login" 
>> with ("login" being a bundle) for the user logging in.
>>   (this can be further extended using PAM to run "s6-rc start 
>> ${PAM_TYPE}" to differentiate different types of logins)
>>   The script L can be started by PAM, .profile or in some other way.
>
>  My advice is to do your whole 2c point at login time. The script L can
> perform XDG stuff, s6-rc-init, and s6-rc start default. On last logout,
> you stop everything with "s6-rc -da change" and you delete the livedir.
> Only the supervision tree remains, maintaining whatever services the
> user has manually added.
>
>  I don't think there is a good argument for keeping state (which is
> what oneshots do) in user services when the user is logged out. Feel
> free to give counterexamples though.
>
> -- 
>  Laurent
>
I find two problems with this:

- A     Users might want to achieve a state of the user-tree on boot, 
which is impossible without s6-rc
           Though I have to admit that I can not find concrete examples 
of application.

- B     Imagine the following scenario:

           A user wants to run a script every day at 12:00 and sets up a 
snooze user-longrun for that,
           which he installs and starts using s6-rc-init and s6-rc 
respectively while logged in,
           but logically does not stop on logout.
           Now the server reboots, brings up the user-tree again, but 
without the snooze user-service of course.
           The system-longrun 1a/2a would thus need an additional 
mechanism to copy s6 service definitions
           from a user writable directory to the user's service directory.

           This solution has the following implications:

            - B1    The user needs to, additionally to s6-rc source 
definitions, learn s6 service definitions.

            - B2    Distribution maintainers would need to redundantly 
create
                       both s6-rc source definitions and s6 service 
definitions for services,
                       so that users can use the latter at boot time.

All in all this would make things more complicated,
in that users have to learn and consider more things while distribution 
maintainers have more work.

Additionally having s6-rc available sometimes (while logged in) and 
sometimes not (while logged out)
will ultimately lead to confusion.

So while I agree to you, that until I find a concrete examples for A,
s6-rc during "non-login" time is technically not necessary,
lacking it makes many things more inconvenient, complex and inconsistent.

On the other side, why not have it? It is a feature that is probably 
going be useful at some point and comes with little to no cost.

Finally, I want to say that I really admire you being very conservative 
on adding features,
it is great to know that s6/s6-rc will stay sleek and efficient.

Have a nice evening

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 373 bytes --]

Laurent Bercot schrob:
>  I don't think there is a good argument for keeping state (which is
> what oneshots do) in user services when the user is logged out. Feel
> free to give counterexamples though.

fusermounting some network/encrypted filesystem comes to mind.
e.g. mount a NAS share so that a longrun snooze "cronjob" can record a
livestream to it.

cheers,
    Jan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Have an external script wait for a oneshot service

[Brett Neumeier via supervision]

On Thu, 5 Dec 2024 22:10:17 +0100
Paul Sopka <psopka@sopka.ch> wrote:
> - B     Imagine the following scenario:
> 
>            A user wants to run a script every day at 12:00 and sets up a 
> snooze user-longrun for that,
>            which he installs and starts using s6-rc-init and s6-rc 
> respectively while logged in,
>            but logically does not stop on logout.

Is there a reason that the user would not simply set up a cron job to run the script every day at noon?

I mean, obviously, s6-rc services are way sexier than crontab entries, but cron jobs have been working fine for decades and if you want to run a script on a specific schedule, why not just ... use the tools you have?

-- 
Brett Neumeier <random@freesa.org>

Re[2]: Have an external script wait for a oneshot service

[Laurent Bercot]

>- A     Users might want to achieve a state of the user-tree on boot, which is impossible without s6-rc
>           Though I have to admit that I can not find concrete examples of application.

  Exactly. That's the YAGNI principle: don't try to solve problems you
haven't encountered yet. Chances are you'll actually never encounter
them. Adding complexity in the name of "just in case" is overengineering
and it will only worsen your design.

  And if I have learned anything about users, it's that most of your
users will have trouble understanding what a supervised service is,
and if you start talking about them about s6-rc services, their brain
will start melting and they'll run away from your system as fast as
they can. And they won't be wrong.


>- B     Imagine the following scenario:
>
>           A user wants to run a script every day at 12:00 and sets up a snooze user-longrun for that,

  So far so good (I understand Brett's point, but that should not be
the basis for an argument, because there are other things that can
only be achieved via a longrun, crontab is only an example here)


>           which he installs and starts using s6-rc-init and s6-rc respectively while logged in,

  ... and here you lose me.

  The user has a supervision tree. They can install their longrun 
directly
in it. They don't need s6-rc for that. Pure s6 is enough.
  Their service will run as long as they keep it there. They can stop
it with s6-svc. Or s6-svunlink.

  s6-rc is good when you have an intricate mix of longruns and oneshots.
This is the case in a boot sequence. This *might* be the case for a
user login sequence. And this is almost certainly *not* the case for
a set of user services that you would hypothetically start at boot time
and do stuff with before a user even logs in, wtf.

  You want to run s6-rc at login time, sure, I can totally see where i
can be beneficial there. But earlier? Even systemd doesn't run user
stuff before a user logs in. And if *they* don't do it, it means that
there are ZERO use cases.


|          Now the server reboots, brings up the user-tree again, but 
without the snooze user-service of course.

  And what is preventing that? The fact that your default bundle does not
include the services that the user has defined themself. This should 
give
you a hint that this design goes against what supervision is supposed to
be doing, and thus, that it's not a good design. Whatever longrun the
user wants to have survive a reboot should survive a reboot and the
user shouldn't need a PhD to accomplish that.

  If the user has installed their service in the pure s6 way with e.g.
one (1) call to s6-svlink, it works: after a reboot, the snooze will
be back up.


>All in all this would make things more complicated,
>in that users have to learn and consider more things while distribution maintainers have more work.

  Exactly. You're *so close*.


>So while I agree to you, that until I find a concrete examples for A,
>s6-rc during "non-login" time is technically not necessary,
>lacking it makes many things more inconvenient, complex and inconsistent.

  It's more inconvenient, complex and inconsistent because you want to
keep the framework that makes it so. Remove a little more and you'll
have something elegant again.


>On the other side, why not have it? It is a feature that is probably going be
>useful at some point and comes with little to no cost.

  Little to no cost? This whole thread is about the costs! And from my
perspective, "probably going to be useful at some point" is a pretty
light argument, it's definitely not going to outweigh "too complex to
make work properly".


>Finally, I want to say that I really admire you being very conservative on adding features,
>it is great to know that s6/s6-rc will stay sleek and efficient.

  Thank you. Now take inspiration from that for your own designs. ;)

--
  Laurent

Re[2]: Have an external script wait for a oneshot service

[Laurent Bercot]

>fusermounting some network/encrypted filesystem comes to mind.
>e.g. mount a NAS share so that a longrun snooze "cronjob" can record a
>livestream to it.

  Okay. I'd argue it's still under the complexity line where doing
this manually - fuser(un)mounting at snooze service (de)installation
time - is easier and faster than setting up an s6-rc infrastructure.

--
  Laurent

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 660 bytes --]

> Is there a reason that the user would not simply set up a cron job to run the script every day at noon?
>
> I mean, obviously, s6-rc services are way sexier than crontab entries, but cron jobs have been working fine for decades and if you want to run a script on a specific schedule, why not just ... use the tools you have?

See here for for the advantages of snooze over cron: 
https://github.com/leahneukirchen/snooze

tldr; the main benefit is that each user can set it up for themself,
there is no complex security system required as when all
users use one centralized daemon.

Additionally it can make use of s6-log.

Regards

Paul



[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1: Type: text/plain, Size: 4676 bytes --]

>  Exactly. That's the YAGNI principle: don't try to solve problems you
> haven't encountered yet. Chances are you'll actually never encounter
> them. Adding complexity in the name of "just in case" is overengineering
> and it will only worsen your design.
We now have two examples of application from the other mails.

>  And if I have learned anything about users, it's that most of your
> users will have trouble understanding what a supervised service is,
> and if you start talking about them about s6-rc services, their brain
> will start melting and they'll run away from your system as fast as
> they can. And they won't be wrong. 
When I talked about "users", I mean people actually *using* these tools,
meaning s6, s6rc, execline, POSIX tools, ... hence power users and 
distribution maintainers.
People using their computers only to consume media or write documents 
(whom I will refer to as consumers),
will use Windows, MacOS or, in the best case, a highly managed Linux 
distro anyway.
Thus I do not consider them, since they will never interact with
service management directly anyway.

I therefore try to make life as easy as possible for power users
and distribution maintainers, who manage their own supervision trees
or setup the presets for said managed distros respectively.

>  The user has a supervision tree. They can install their longrun directly
> in it. They don't need s6-rc for that. Pure s6 is enough.
>  Their service will run as long as they keep it there. They can stop
> it with s6-svc. Or s6-svunlink. 

Assuming s6-rc is used for the main supervision tree and on login,
which is fair, since the very thing we discuss about is part of my project
doing all of that with s6-rc,
this would cause users to need to learn s6 additionally to s6-rc,
when s6-rc can be used for all cases.

>  You want to run s6-rc at login time, sure, I can totally see where i
> can be beneficial there. But earlier? Even systemd doesn't run user
> stuff before a user logs in. And if *they* don't do it, it means that
> there are ZERO use cases.
This is an argumentum ad verecundiam.
Systemd is fundamentally flawed in other places too.

>  And what is preventing that? The fact that your default bundle does not
> include the services that the user has defined themself. This should give
> you a hint that this design goes against what supervision is supposed to
> be doing, and thus, that it's not a good design. Whatever longrun the
> user wants to have survive a reboot should survive a reboot and the
> user shouldn't need a PhD to accomplish that. 
>  If the user has installed their service in the pure s6 way with e.g.
> one (1) call to s6-svlink, it works: after a reboot, the snooze will
> be back up. 
We have, due to my poor wording, a misunderstanding here:
Said "default" bundle is part of the user's user-tree,
it is entirely under his control and offers the user a way to have
user-services started at boot time.

I have assumed that the scan-directory of the user-tree is mounted tmpfs.
This would require additional support in 1a/2a
to save the state of the scan-directory on shutdown and load it on boot.
Or let s6-rc-init and s6-rc do this work, by adding the service to the 
user-tree bundle "default"
and running s6-rc for each user-tree at boot.

>  It's more inconvenient, complex and inconsistent because you want to
> keep the framework that makes it so. Remove a little more and you'll
> have something elegant again.
What is the "little more" you are talking about?

>  Little to no cost? This whole thread is about the costs! And from my
> perspective, "probably going to be useful at some point" is a pretty
> light argument, it's definitely not going to outweigh "too complex to
> make work properly". 
The cost is, quoting myself, one(mutually exclusive) of the following:

> - S1    Script s6-fifodir functionality in bot the system-oneshot 
> 1b/2c and the script L.
>
> - S2    Add a readiness api for s6-rc oneshots like s6 longruns have 
> and wait for that of system-oneshot 1b/2c in the script.
>
> - S3    Have the script L do the same as system-oneshot 1b/2c:
>         - 1bi/2ci    Prepare the live directory for the user-tree 
> (preferably under /run/user/${USER}).
>         - 1bii/2cii    Start "s6-rc-init -b" and "s6-rc start login".
>   Utilizing blocking locks and tests to work parallel to 
> system-oneshot 1b/2c.
>   This can only be done if the script is ran through PAM, 
> since 1bi/2ci requires root privileges. 
I do not see where this exceeds "little".

Regards

Paul


[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re[2]: Have an external script wait for a oneshot service

[Laurent Bercot]

>We now have two examples of application from the other mails.

  And both can be solved without s6-rc in a simpler way than with it.

>When I talked about "users", I mean people actually *using* these tools,
>meaning s6, s6rc, execline, POSIX tools, ... hence power users and distribution maintainers.

  Yeah. So do I. Do you have any idea how difficult it is to make
distribution maintainers adopt the s6 paradigms? Change their habits
even a little bit? Do you have any idea of the inertia I've had to
bump against, again and again? When I tell you "make this simpler",
it's not because Joe Schmoe won't understand your stuff. It's because
Power Maintainer Dan J. Hacker won't want to jump through two hoops
so you better make sure there's only one.


>I therefore try to make life as easy as possible for power users
>and distribution maintainers, who manage their own supervision trees
>or setup the presets for said managed distros respectively.

  If that really was what you're trying to do, you'd listen to me.

  Look, I *like* s6-rc. I'm happy that you like it too. I'm happy that
you want to use it. I'm happy when people use my stuff. Don't get me
wrong. But what I like even more is when the right tool is used for a
job, in the right place, with the right glue. That's what makes life
simpler for everyone, truly.


>Assuming s6-rc is used for the main supervision tree and on login,
>which is fair, since the very thing we discuss about is part of my project
>doing all of that with s6-rc,
>this would cause users to need to learn s6 additionally to s6-rc,
>when s6-rc can be used for all cases.

  The vast majority of services a user will want to have are longruns.
And longruns are run under s6; longrun service definition directories
are pretty much s6 service directories. I don't think it's honest to
say "you only need to learn s6-rc, not s6"; the s6-rc learning curve
*includes* learning at least the fundamentals of s6.


>This is an argumentum ad verecundiam.

  Not at all. I never refer to systemd as an authority (lol).
  It's just that systemd is maximalist: every use case they can think of,
they include. (Since the unit file format is extensive, they have to.)
They include way, way more than they should; they are by no means an
example to follow. My point is that if a use case isn't planned for in
systemd, there's a pretty good chance it's not a real one.

  Anyway. You obviously think your design is superior, so who am I to
say otherwise. We're not going to agree on this, so, since it's the
person who does the work who has the final say, you're right. Write your
project however you want to. But I'm not adding hooks to s6-rc to
support notification of intermediary states to external programs, 
because
I simply don't believe it's how it should be used.

--
  Laurent

Re: Have an external script wait for a oneshot service

[Hoël Bézier]

[-- Attachment #1: Type: text/plain, Size: 1317 bytes --]

>>Assuming s6-rc is used for the main supervision tree and on login,
>>which is fair, since the very thing we discuss about is part of my project
>>doing all of that with s6-rc,
>>this would cause users to need to learn s6 additionally to s6-rc,
>>when s6-rc can be used for all cases.
>
>  The vast majority of services a user will want to have are longruns.
>And longruns are run under s6; longrun service definition directories
>are pretty much s6 service directories. I don't think it's honest to
>say "you only need to learn s6-rc, not s6"; the s6-rc learning curve
>*includes* learning at least the fundamentals of s6.

As a matter of fact, I’ve setup “user services” on my computers (services for 
one user only actually, I do not care about edge cases since I’m both root and 
the regular user) a few months ago with s6-rc, and up til now I only have 
longruns.

I do have though dependencies between these longruns, mostly between the user 
dbus service and other user-specific services that depend on that thing, like 
pipewire.

Not sure it would justify the use of s6-rc, because one could just let pipewire 
start and die until dbus is run. Or consider these services do not make sense 
when the user is not logged in, so that they will be run by s6-rc on login.

Hoël

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Have an external script wait for a oneshot service

[Paul Sopka]

[-- Attachment #1.1.1.1: Type: text/plain, Size: 4748 bytes --]

First of all, if you get tired of this discussion, just tell me and I 
will stop bothering you.
I, for my part, experience great joy from this discussion!

If you are still interested, here you go:

>  And both can be solved without s6-rc in a simpler way than with it.
Why do you think it is simpler?
It is one oneshot less, at the cost of running s6 only and s6-rc 
services side by side
and lacking dependency management (see Hoël's mail).

Besides, I suppose one being this:

>> fusermounting some network/encrypted filesystem comes to mind.
>> e.g. mount a NAS share so that a longrun snooze "cronjob" can record a
>> livestream to it.
>
>  Okay. I'd argue it's still under the complexity line where doing
> this manually - fuser(un)mounting at snooze service (de)installation
> time - is easier and faster than setting up an s6-rc infrastructure. 
But this to me seems like saying "why not mount devfs in the longrun 
starting mdevd/udevd?"

>  Yeah. So do I. Do you have any idea how difficult it is to make
> distribution maintainers adopt the s6 paradigms? Change their habits
> even a little bit? Do you have any idea of the inertia I've had to
> bump against, again and again? When I tell you "make this simpler",
> it's not because Joe Schmoe won't understand your stuff. It's because
> Power Maintainer Dan J. Hacker won't want to jump through two hoops
> so you better make sure there's only one.
I can only imagine. But that's why I am doing this,
so that everybody like minded can have a working base structure to use 
s6/s6-rc.

>  If that really was what you're trying to do, you'd listen to me.
I listen to you and I really try to understand your point,
I am just not convinced (yet?).

>  Look, I *like* s6-rc. I'm happy that you like it too. I'm happy that
> you want to use it. I'm happy when people use my stuff. Don't get me
> wrong. But what I like even more is when the right tool is used for a
> job, in the right place, with the right glue. That's what makes life
> simpler for everyone, truly. 
I wholeheartedly agree, our disagreement lies somewhere else.

>  The vast majority of services a user will want to have are longruns.
> And longruns are run under s6; longrun service definition directories
> are pretty much s6 service directories. I don't think it's honest to
> say "you only need to learn s6-rc, not s6"; the s6-rc learning curve
> *includes* learning at least the fundamentals of s6.
I fully agree, scrap what I have written concerning this.
I want to mention though what always using s6-rc makes the experience 
more consistent.

> But I'm not adding hooks to s6-rc to
> support notification of intermediary states to external programs, because
> I simply don't believe it's how it should be used. 
I respect that.


I would be interested in what you think about the other points in my 
previous mail, especially:

> I have assumed that the scan-directory of the user-tree is mounted tmpfs.
> This would require additional support in 1a/2a
> to save the state of the scan-directory on shutdown and load it on boot.
> Or let s6-rc-init and s6-rc do this work, by adding the service to the 
> user-tree bundle "default"
> and running s6-rc for each user-tree at boot. 

> What is the "little more" you are talking about? 

> I do not see where this exceeds "little".
and whether:

> I have assumed that the scan-directory of the user-tree is mounted tmpfs. 
Is a good idea in the first place.


Finally, I believe to have found (thanks to our discussion making me 
think) an even better solution:
(note the paths / filenames being only exemplary)

1.    Have a system-longrun create/mount /run/user/${USER} tmpfs and 
start s6-svscan on /run/user/${USER}/service

2.    Have an (optional) oneshot do the following:

        a) Take a lock on /run/user/${USER}/s6-rc-lock

        b) If { /run/user/${USER}/s6-rc does not exist } s6-rc-init

        c) s6-rc start default

3.    Have the login script

        a) Take a lock on /run/user/${USER}/s6-rc-lock

        b) If { /run/user/${USER}/s6-rc does not exist } s6-rc-init

        c) s6-rc start login

This does not require waiting for a system-oneshot,
it allows people like you to not have s6-rc run for users on boot
and allows people like me to manage it everything with s6-rc.
Additionally it cleanly decouples boot user-s6-rc from login user-s6-rc,
in that the latter do not require the former to be prepared.


> so who am I to say otherwise.
You are the creator of the best pieces of software I could discover on 
the internet so far.
Thank you for writing it!

Regards

Paul


[-- Attachment #1.1.1.2: Type: text/html, Size: 7200 bytes --]

[-- Attachment #1.1.2: OpenPGP public key --]
[-- Type: application/pgp-keys, Size: 3195 bytes --]

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 840 bytes --]

Re: Have an external script wait for a oneshot service

[Jan Braun]

[-- Attachment #1: Type: text/plain, Size: 1984 bytes --]

Laurent Bercot schrob:
[restored quote for context]
> > > You want to run s6-rc at login time, sure, I can totally see where i
> > > can be beneficial there. But earlier? Even systemd doesn't run user
> > > stuff before a user logs in. And if *they* don't do it, it means that
> > > there are ZERO use cases.
> >
> > This is an argumentum ad verecundiam.
>
>  Not at all. I never refer to systemd as an authority (lol).
>  It's just that systemd is maximalist: every use case they can think of,
> they include. (Since the unit file format is extensive, they have to.)
> They include way, way more than they should; they are by no means an
> example to follow. My point is that if a use case isn't planned for in
> systemd, there's a pretty good chance it's not a real one.

Users have been able to run oneshots at boot since before systemd
existed: @reboot in a user crontab. That alone should prove the use case
is real.

And systemd apparently *does* have the ability to run user stuff without
the user logging in:
https://wiki.archlinux.org/title/Systemd/User#Automatic_start-up_of_systemd_user_instances

I didn't find an @reboot equivalent for systemd timers in a quick
search, but since oneshots are used for managing state, my guess is
their answer would be "there should be a special systemd functionality
for managing that particular state, use that" and/or "fix the state in
the ExecStartPre for the service relying on the state". The latter would
even be reasonable: as a runit user, I've always been ensuring necessary
state in the ./run file, and never found the lack of explicit support
for oneshots limiting.

That is also why I can't form an opinion on the merits of Paul Sopka's
proposal. But effectively saying "(user)oneshots/s6-rc are useful while
the user is logged in, but not while the user is not logged in" seems
wrong. That's machinery that might be useful, or might not be, but it
shouldn't depend on or care about the user's login state.

cheers,
    Jan


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]