Dependencies for runit (again)

Dependencies for runit (again)

[Mike Bell]

From discussions on some list set up to discuss an alternative init
system for debian, I've garnered three main issues with runit's
current dependency system:

- It's not programmatically parsable, only a human can tell that a rule
  at the top of a shell script checking for a given socket is actually
  waiting for postgres. Prevents
- It encourages code duplication. The code to check if service X is up
  is not in service X's directory, but rather scattered over the run
  scripts of the services which depend upon it. [1]
- The "repeatedly exec until service is up" method seems to bother
  people.

I believe there may be advantages to considering a "proper" dependency
system for runit.

What I had in mind was firstly the addition of a new shell script in the
service's directory (anyone got a good name?), designed to test whether
the service is currently running. In its absence we would have to rely
on the "it hasn't restarted for a while" method.

Secondly, a new directory in a service directory, named dependencies,
which contains symlinks to other service directories. runsv then ensures
that each of these services is running (by either using the
aforementioned "am I up" test script itself, or by having the runsv of
the service itself use said script and then report the status).

Thirdly, though I'm not certain of the need for this, another directory
like the above describing services which should be taken down along with
the current service.

It's a fairly simple set of changes, but...
- it gets you dependencies which can be machine parsed and managed
  easily.

- it moves the detection of a daemon's status into that daemon itself
  where it belongs - rather than having it duplicated in each of the
  dependencies.

- it allows you to do virtual dependencies (contrived example, daemon X
  depends on either mysql or postgresql. Its dependencies simply link to
  sql-server rather than either one specifically. Certainly a lot easier
  than a mess of shell scripting with logical ors to detect each sql
  daemon it can use.)

- it fits better with package management. An upgrade to postgresql may
  change the method for detecting if it's up, but with current runit
  you'd have to also update every package which depends on postgresql.

- it reduces the spin-and-die people seem to dislike so much about
  runit. Particularly for a very complicated setup with a lot of daemons
  depending on a single one (directly or indirectly) this could be a
  significant reduction.

- probably other things I haven't thought of.

Comments?

Re: Dependencies for runit (again)

[Stefan Karrmann]

Dear all,

Mike Bell (Tue, Aug 09, 2005 at 10:03:50AM -0700):
> From discussions on some list set up to discuss an alternative init
> system for debian, I've garnered three main issues with runit's
> current dependency system:
> 
> - It's not programmatically parsable, only a human can tell that a rule
>   at the top of a shell script checking for a given socket is actually
>   waiting for postgres. Prevents
> - It encourages code duplication. The code to check if service X is up
>   is not in service X's directory, but rather scattered over the run
>   scripts of the services which depend upon it. [1]
> - The "repeatedly exec until service is up" method seems to bother
>   people.

IMHO, the latter is a main point of daemontools and runit.

> I believe there may be advantages to considering a "proper" dependency
> system for runit.
> 
> What I had in mind was firstly the addition of a new shell script in the
> service's directory (anyone got a good name?), designed to test whether

my 2 ¢: `accepting' - think of sockets.

You may use pipe-tools if you prefer events over polling.

> the service is currently running. In its absence we would have to rely
> on the "it hasn't restarted for a while" method.
> 
> Secondly, a new directory in a service directory, named dependencies,
> which contains symlinks to other service directories. runsv then ensures
> that each of these services is running (by either using the
> aforementioned "am I up" test script itself, or by having the runsv of
> the service itself use said script and then report the status).

Why should runsv do the work? Write a new command, e.g.
`start-dependencies', which starts all services in the sub-directory
dependencies. Then run it as first command from the run-command of
a service.

How are the signals from runit or supervise handled here?

> Thirdly, though I'm not certain of the need for this, another directory
> like the above describing services which should be taken down along with
> the current service.

Again, why should runsv do it? Write `stop-dependencies' and enter it into
the finish-command of the service.

> It's a fairly simple set of changes, but...
> - it gets you dependencies which can be machine parsed and managed
>   easily.
> 
> - it moves the detection of a daemon's status into that daemon itself
>   where it belongs - rather than having it duplicated in each of the
>   dependencies.
> 
> - it allows you to do virtual dependencies (contrived example, daemon X
>   depends on either mysql or postgresql. Its dependencies simply link to
>   sql-server rather than either one specifically. Certainly a lot easier
>   than a mess of shell scripting with logical ors to detect each sql
>   daemon it can use.)
> 
> - it fits better with package management. An upgrade to postgresql may
>   change the method for detecting if it's up, but with current runit
>   you'd have to also update every package which depends on postgresql.
> 
> - it reduces the spin-and-die people seem to dislike so much about
>   runit. Particularly for a very complicated setup with a lot of daemons
>   depending on a single one (directly or indirectly) this could be a
>   significant reduction.

Regards,
-- 
Stefan

Re: Dependencies for runit (again)

[Gerrit Pape]

On Tue, Aug 09, 2005 at 10:34:50PM +0200, Stefan Karrmann wrote:
> Mike Bell (Tue, Aug 09, 2005 at 10:03:50AM -0700):
> > From discussions on some list set up to discuss an alternative init
> > system for debian, I've garnered three main issues with runit's
> > current dependency system:
> > 
> > - It's not programmatically parsable, only a human can tell that a rule
> >   at the top of a shell script checking for a given socket is actually
> >   waiting for postgres. Prevents
> > - It encourages code duplication. The code to check if service X is up
> >   is not in service X's directory, but rather scattered over the run
> >   scripts of the services which depend upon it. [1]
> > - The "repeatedly exec until service is up" method seems to bother
> >   people.

> > I believe there may be advantages to considering a "proper" dependency
> > system for runit.
> > 
> > What I had in mind was firstly the addition of a new shell script in the
> > service's directory (anyone got a good name?), designed to test whether
> 
> my 2 ¢: `accepting' - think of sockets.

We have ./run and ./finish, I would call it ./check.

> > the service is currently running. In its absence we would have to rely
> > on the "it hasn't restarted for a while" method.
> > 
> > Secondly, a new directory in a service directory, named dependencies,
> > which contains symlinks to other service directories. runsv then ensures
> > that each of these services is running (by either using the
> > aforementioned "am I up" test script itself, or by having the runsv of
> > the service itself use said script and then report the status).
> 
> Why should runsv do the work? Write a new command, e.g.
> `start-dependencies', which starts all services in the sub-directory
> dependencies. Then run it as first command from the run-command of
> a service.

Yes, I think Mike's ideas are pretty thought through, and can well be
tested with runit, only implemented through the scripts, see also

 http://article.gmane.org/gmane.linux.debian.devel.initscripts-ng/27

How about a subdirectory ./depends/ in the service directory containing
links or files naming the dependencies, and this in the ./run script?:
 #!/bin/sh
 for i in `ls -1 ./depends/ 2>/dev/null || :`; do
   sv start $i || exit 1
   ! test -x /var/service/$i/check || $_ || exit 1
 done
 exec service-daemon

> > Thirdly, though I'm not certain of the need for this, another directory
> > like the above describing services which should be taken down along with
> > the current service.
> 
> Again, why should runsv do it? Write `stop-dependencies' and enter it into
> the finish-command of the service.

Or in the ./control/d script.  Restarting dependent services if a
service restarts normally shouldn't be necessary, but people might want
to take dependent services down automatically when taking a specific
service down, I'm not sure.

> > It's a fairly simple set of changes, but...
> > - it gets you dependencies which can be machine parsed and managed
> >   easily.

 # ls -1 /var/service/foo/depends/

 # ls /var/service/*/depends/foo

> > - it moves the detection of a daemon's status into that daemon itself
> >   where it belongs - rather than having it duplicated in each of the
> >   dependencies.
> > 
> > - it allows you to do virtual dependencies (contrived example, daemon X
> >   depends on either mysql or postgresql. Its dependencies simply link to
> >   sql-server rather than either one specifically. Certainly a lot easier
> >   than a mess of shell scripting with logical ors to detect each sql
> >   daemon it can use.)
> > 
> > - it fits better with package management. An upgrade to postgresql may
> >   change the method for detecting if it's up, but with current runit
> >   you'd have to also update every package which depends on postgresql.
> > 
> > - it reduces the spin-and-die people seem to dislike so much about
> >   runit. Particularly for a very complicated setup with a lot of daemons
> >   depending on a single one (directly or indirectly) this could be a
> >   significant reduction.

Yes, sounds good.

Regards, Gerrit.

Re: Dependencies for runit (again)

[Paul Jarc]

Gerrit Pape <pape@smarden.org> wrote:
>  for i in `ls -1 ./depends/ 2>/dev/null || :`; do
>    sv start $i || exit 1
>    ! test -x /var/service/$i/check || $_ || exit 1
>  done

It would be nice to have this bit as its own command, so a run script
would only have to call something like "check-depends ./depends"
without any further duplication of code between run scripts.

Also, one benefit of letting the dependency's own runsv handle
invocation of the check script is that it gives the check script the
same consistent process state as the corresponding run script.


paul

Re: Dependencies for runit (again)

[Gerrit Pape]

On Fri, Aug 12, 2005 at 01:32:53PM -0400, Paul Jarc wrote:
> Gerrit Pape <pape@smarden.org> wrote:
> >  for i in `ls -1 ./depends/ 2>/dev/null || :`; do
> >    sv start $i || exit 1
> >    ! test -x /var/service/$i/check || $_ || exit 1
> >  done
> 
> It would be nice to have this bit as its own command, so a run script
> would only have to call something like "check-depends ./depends"
> without any further duplication of code between run scripts.

Yes, but it's ok to try it out; 'sv status' and/or 'sv start' maybe
should make use of the ./check scripts, and there's also this idea

 http://article.gmane.org/gmane.comp.sysutils.supervision.general/724

> Also, one benefit of letting the dependency's own runsv handle
> invocation of the check script is that it gives the check script the
> same consistent process state as the corresponding run script.

Hmm, it would be nice to have a guaranteed process state for ./check,
true.

Regards, Gerrit.

Re: Dependencies for runit (again)

[Gerrit Pape]

On Fri, Aug 12, 2005 at 01:32:53PM -0400, Paul Jarc wrote:
> Gerrit Pape <pape@smarden.org> wrote:
> >  for i in `ls -1 ./depends/ 2>/dev/null || :`; do
> >    sv start $i || exit 1
> >    ! test -x /var/service/$i/check || $_ || exit 1
> >  done
> 
> It would be nice to have this bit as its own command, so a run script
> would only have to call something like "check-depends ./depends"
> without any further duplication of code between run scripts.

With having the sv program now automatically using the ./check script,
this can be replaced with, e.g.

 sv start `ls -1 ./depends/ 2>/dev/null || :`

or if the ./depends/ directory contains symlinks

 sv start ./depends/*

Regards, Gerrit.