runit and System shutdown

runit and System shutdown

[Sascha Hüdepohl]

Hi,

i am using runit 1.0.5 on my Debian/Sarge for all the Daemons except
postfix. runsvdir-start is run from inittab.

Now i wonder what happens to my Daemons on System shutdown. As far as i
understand there is no explicit "stop" for the Services. init will
change to Runlevel 0 and /etc/rc0.d/S20sendsigs will send SIG15 to all
running Processes. This will hit all runsv svlogd and even runsvdir at
the same time. To me this sounds like total loss of control. Its quite a
mess. Is there something i dont understand? Shouldnt there be an
init-script that does runsvctrl d /var/services/* as the first Action
in Runlevel 0?

TIA
sascha

Re: runit and System shutdown

[Henrik Heil]

Hi,

> i am using runit 1.0.5 on my Debian/Sarge for all the Daemons except
> postfix. runsvdir-start is run from inittab.

I have a similar setup on 2 servers and was wondering about the right 
way to do this.

> To me this sounds like total loss of control. Its quite a
> mess. Is there something i dont understand? Shouldnt there be an
> init-script that does runsvctrl d /var/services/* as the first Action
> in Runlevel 0?

In addition to the the shutdown of runsv, services and logservices in no 
particular order I recently noticed that with the default inittab-entry 
in runlevel 6:

* sendsigs TERM makes runsvdir exit immediatly, tells all processes to 
terminate an waits 5 seconds
* meanwhile SysV-init restarts runsvdir and runsvdir starts all services
* after 5 seconds all services are killed(-9) by sendsigs

I am using this inittab-entry for now:

RT1a:12345:respawn:/usr/sbin/runsvdir-start
RT1b:06:once:/usr/sbin/runsvdir-start

I am not shure about the shutdown-initscript. I was thinking in that 
direction but would like to use "svwaitdown -x -t 10 /var/services/*" 
because

* it waits for the services to actually stop.
* I believe it guarantees to stop the logservice after the service (is 
this correct?).

So an (untested) shutdown-script would be:

kill `pidof runsvdir`
svwaitdown -x -t 10 /var/services/*

(only works if runsvdir does not respawn -- see above)

Apart from the SysV-integration problem with the respawning runsvdir I 
think runit does the right thing when the commands receive a TERM signal:

If runsv receives a TERM it acts like runsvctl x.
If runsvdir receives a TERM signal, it exits with 0 immediately.

Even if you don't stop any services I believe it's unlikely that runsv 
tries to restart a service before it receives sendsigs TERM and itsself 
waits for the service to exit. So if you don't have somthing that takes 
more than 5 seconds to stop nothing too bad should happen.

But I am still not too familiar with process supervision an would really 
like to hear comments or a recommended way to stop runit under SysV-init 
from the experts.

Regards,
Henrik

-- 
Henrik Heil, zweipol Coy & Heil GbR
http://www.zweipol.net/

Re: runit and System shutdown

[Gerrit Pape]

On Mon, Jan 30, 2006 at 09:46:52PM +0100, Henrik Heil wrote:
> >i am using runit 1.0.5 on my Debian/Sarge for all the Daemons except
> >postfix. runsvdir-start is run from inittab.

For running postfix under service supervision, take a look at the
runit-services Debian package (sid/etch) I introduced recently.  It
includes a service directory for Debian's postfix.

> I have a similar setup on 2 servers and was wondering about the right 
> way to do this.
> 
> >To me this sounds like total loss of control. Its quite a
> >mess. Is there something i dont understand? Shouldnt there be an
> >init-script that does runsvctrl d /var/services/* as the first Action
> >in Runlevel 0?
> 
> In addition to the the shutdown of runsv, services and logservices in no 
> particular order I recently noticed that with the default inittab-entry 
> in runlevel 6:
> 
> * sendsigs TERM makes runsvdir exit immediatly, tells all processes to 
> terminate an waits 5 seconds
> * meanwhile SysV-init restarts runsvdir and runsvdir starts all services
> * after 5 seconds all services are killed(-9) by sendsigs

Yes, the shutdown with runsvdir-start run from inittab seems to be far
from perfect.

> I am using this inittab-entry for now:
> 
> RT1a:12345:respawn:/usr/sbin/runsvdir-start
> RT1b:06:once:/usr/sbin/runsvdir-start
> 
> I am not shure about the shutdown-initscript. I was thinking in that 
> direction but would like to use "svwaitdown -x -t 10 /var/services/*" 
> because
> 
> * it waits for the services to actually stop.
> * I believe it guarantees to stop the logservice after the service (is 
> this correct?).

Yes this is correct.  In contrary to daemontools, there's only one
supervisor process per service, handling the service and the log
service.  When told to exit, runsv only tells the service to terminate,
and waits for the logger to stop afterwards (automatically due to closed
standard input).  So no logs are lost.

> So an (untested) shutdown-script would be:
> 
> kill `pidof runsvdir`
> svwaitdown -x -t 10 /var/services/*

I haven't updated the stage 3 scripts yet in runit-1.3.3, but after the
sv program has been introduced, better use something like this

 echo 'Waiting for services to stop...'
 sv -w196 force-stop /var/service/*
 sv exit /var/service/*

> (only works if runsvdir does not respawn -- see above)
> 
> Apart from the SysV-integration problem with the respawning runsvdir I 
> think runit does the right thing when the commands receive a TERM signal:
> 
> If runsv receives a TERM it acts like runsvctl x.
> If runsvdir receives a TERM signal, it exits with 0 immediately.

This is the right thing when using runit as process 1, yes.  The stage 3
script then can take care of reliable and clean shutdown.

> Even if you don't stop any services I believe it's unlikely that runsv 
> tries to restart a service before it receives sendsigs TERM and itsself 
> waits for the service to exit. So if you don't have somthing that takes 
> more than 5 seconds to stop nothing too bad should happen.
> 
> But I am still not too familiar with process supervision an would really 
> like to hear comments or a recommended way to stop runit under SysV-init 
> from the experts.

I personally no longer run runit under sysvinit, but use runit as
process no 1 and possibly some sysv init script in stage 1.

Basically you can switch from init to runit as process 1 without
changing the system bootup, you only lose sysv runlevel handling, and
the respawn feature through inittab AFAICS.  If you by default boot into
runlevel 2, just add the rc scripts to stage 1 (/etc/runit/1), services
normally run through init scripts should come up as usual, e.g.

 #!/bin/sh
 # system one time tasks
 PATH=/sbin:/bin:/usr/sbin:/usr/bin
 /etc/init.d/rcS
 /etc/init.d/rc 2
 ...

HTH, Gerrit.