supervision - discussion about system services, daemon supervision, init, runlevel management, and tools such as s6 and runit
 help / color / mirror / Atom feed
* using runit as init
@ 2008-01-03 20:51 Bernhard Graf
  2008-01-04  0:22 ` Mike Buland
  0 siblings, 1 reply; 37+ messages in thread
From: Bernhard Graf @ 2008-01-03 20:51 UTC (permalink / raw)
  To: supervision

Hi list, happy new year!

I'm playing a little with runit as replacement for SysV init as roughly 
described in http://smarden.org/runit/replaceinit.html .

I adjusted the three runit stage scripts to play with an OpenSUSE 10.3 
Linux system. So far runit only supervises the (min)gettys, whereas all 
services that come with the distribution are still managed 
from /etc/init/rc that gets executed at the end of /etc/runit/1 .
The whole thing is wrapped into two RPMs (runit and 
runit-sysvinit-compat) that replace the sysvinit RPM installation.

Seems to work quite well, provided that I didn't do much more testing 
than boot, reboot, halt for now.

What worries me now is a smooth transition for running systems from init 
to runit.

In general I'd prefer to leave most service where they are, and just 
supervise certain ones, e.g. ssh, web server, all the djb s/w and 
custom daemons like FastCGI processes.

The problem is that runit makes (nearly) no effort to support a system 
with supervised and "traditional" non-supervised daemons.
Perhaps this is also the reason why runit didn't gain much attention in 
all the years, because concept and code are very well done AFAICT.

At the moment I'm struggling with these questions:

- A way to boot into a certain runlevel (say "init xxx" at boot prompt).
  runit and runit-init don't care about these, where as (Smoorenburg's)
  init passes the current runlevel in environment variable RUNLEVEL.
- Changing runlevels. There is no way to say "init 3" anymore.
  runsvchdir doesn't care about the SysV init scripts that are started
  in /etc/runit/1.
  runit-init only supports SysV init args "0" and "6" when not runing as
  process 1.
- utmp/wtmp support is missing completely, besides utmpset which should
  be named utmpclr or so.

Maybe somebody can give some advice or can share some example code or 
ideas.

In case of interest and if someone has some public space at hand I would 
upload the SRPMs.
-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-03 20:51 using runit as init Bernhard Graf
@ 2008-01-04  0:22 ` Mike Buland
  2008-01-05  0:06   ` Bernhard Graf
       [not found]   ` <12EC84FDD73F4BD8A78E7501EB19F1E2@home.internal>
  0 siblings, 2 replies; 37+ messages in thread
From: Mike Buland @ 2008-01-04  0:22 UTC (permalink / raw)
  To: supervision

In general, if you want to retain init-style runlevels, then you don't want to 
replace SysV init.  I have switched to runit entirely, it manages every 
service, and it does indeed support "runlevels."  See 
http://smarden.org/runit/runsvchdir.8.html for details.  Each runit runlevel 
is really just another service directory, runsvchdir changes which service 
directory you are using.  the runit "init" program is provided mainly for 
compatibility.

There is no reason for runlevels to be numbered, and the kernel offers no 
support for runlevels, nor do any bootloaders.  When you pass 2, 3, 4, S, 
etc. to the kernel, it is ignored as an unrecognized parameter, but made 
available to running processes a number of ways.  Runit makes the commandline 
as well as any variables that are set on the kernel commandline available to 
the script 1 (and maybe 2 and 3, but I use it in 1).  Therefore if you set a 
variable in the bootloader on the kernel commandline named "runlevel" then 
you can use that in script 1 to change your runlevel.

However, I would say based on the way you are using runit as process 1, you 
would probably be better off sticking with sysvinit and adding runsvdir to 
your inittab (a script very similar to the example script 2 provided with 
runit is just what you want).

As for the runlevel code, I got this from this very list a long time ago, give 
it a shot in script 1:

# Set runlevel to:
# - single       if kernel has param: S
# - RUNLEVELNAME if kernel has param: runlevel=RUNLEVELNAME
# - default      if kernel has no params or unable to set requested runlevel
grep -q '\(^\| \)S\( \|$\)' /proc/cmdline && runlevel='single'
runsvchdir ${runlevel:-default} || runsvchdir default

This wored great for me, then in grub I added "runlevel=X" 
or "runlevel=console"

And finally, you want the normal setup, but point /var/service 
to /etc/runit/runsvdir/current, if it's not there, it will be created when 
runsvchdir is called, or you can create it yourself, it's a symlink to the 
current service directory.  Each directory in  /etc/runit/runsvdir is a 
complete (and normal) service directory.

utmp/wtmp support is done, I believe, through an external program, check out 
the getty*/finish scripts that are provided with the default distribution.

--Mike

On Thursday 03 January 2008 01:51:21 pm Bernhard Graf wrote:
> Hi list, happy new year!
>
> I'm playing a little with runit as replacement for SysV init as roughly
> described in http://smarden.org/runit/replaceinit.html .
>
> I adjusted the three runit stage scripts to play with an OpenSUSE 10.3
> Linux system. So far runit only supervises the (min)gettys, whereas all
> services that come with the distribution are still managed
> from /etc/init/rc that gets executed at the end of /etc/runit/1 .
> The whole thing is wrapped into two RPMs (runit and
> runit-sysvinit-compat) that replace the sysvinit RPM installation.
>
> Seems to work quite well, provided that I didn't do much more testing
> than boot, reboot, halt for now.
>
> What worries me now is a smooth transition for running systems from init
> to runit.
>
> In general I'd prefer to leave most service where they are, and just
> supervise certain ones, e.g. ssh, web server, all the djb s/w and
> custom daemons like FastCGI processes.
>
> The problem is that runit makes (nearly) no effort to support a system
> with supervised and "traditional" non-supervised daemons.
> Perhaps this is also the reason why runit didn't gain much attention in
> all the years, because concept and code are very well done AFAICT.
>
> At the moment I'm struggling with these questions:
>
> - A way to boot into a certain runlevel (say "init xxx" at boot prompt).
>   runit and runit-init don't care about these, where as (Smoorenburg's)
>   init passes the current runlevel in environment variable RUNLEVEL.
> - Changing runlevels. There is no way to say "init 3" anymore.
>   runsvchdir doesn't care about the SysV init scripts that are started
>   in /etc/runit/1.
>   runit-init only supports SysV init args "0" and "6" when not runing as
>   process 1.
> - utmp/wtmp support is missing completely, besides utmpset which should
>   be named utmpclr or so.
>
> Maybe somebody can give some advice or can share some example code or
> ideas.
>
> In case of interest and if someone has some public space at hand I would
> upload the SRPMs.



^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-04  0:22 ` Mike Buland
@ 2008-01-05  0:06   ` Bernhard Graf
       [not found]   ` <12EC84FDD73F4BD8A78E7501EB19F1E2@home.internal>
  1 sibling, 0 replies; 37+ messages in thread
From: Bernhard Graf @ 2008-01-05  0:06 UTC (permalink / raw)
  To: supervision

Mike Buland wrote:

> In general, if you want to retain init-style runlevels, then you
> don't want to replace SysV init.  I have switched to runit entirely,
> it manages every service, and it does indeed support "runlevels." 
> See
> http://smarden.org/runit/runsvchdir.8.html for details.  Each runit
> runlevel is really just another service directory, runsvchdir changes
> which service directory you are using.  the runit "init" program is
> provided mainly for compatibility.
>
> There is no reason for runlevels to be numbered, and the kernel
> offers no support for runlevels, nor do any bootloaders.  When you
> pass 2, 3, 4, S, etc. to the kernel, it is ignored as an unrecognized
> parameter, but made available to running processes a number of ways. 
> Runit makes the commandline as well as any variables that are set on
> the kernel commandline available to the script 1 (and maybe 2 and 3,
> but I use it in 1).  Therefore if you set a variable in the
> bootloader on the kernel commandline named "runlevel" then you can
> use that in script 1 to change your runlevel.
>
> However, I would say based on the way you are using runit as process
> 1, you would probably be better off sticking with sysvinit and adding
> runsvdir to your inittab (a script very similar to the example script
> 2 provided with runit is just what you want).

As I wrote in my previous email I want an easy way to switch from SysV 
init to runit (or may be another one that supports process 
supervision). Since I have to care for a couple of production systems 
this switch must meet some conditions:
- full support for the old init system
- keep manual adjusting to a minimum

ATM I'm starting daemontools svscan from inttab on those hosts in 
addition to the normal SysV init stuff. This works but is not really 
pretty - I'd like to have a more consistent administration interface.

The goal is to install runit with RPM, reboot and then link the 
daemontools things into runit's service directory.

> As for the runlevel code, I got this from this very list a long time
> ago, give it a shot in script 1:
>
> # Set runlevel to:
> # - single       if kernel has param: S
> # - RUNLEVELNAME if kernel has param: runlevel=RUNLEVELNAME
> # - default      if kernel has no params or unable to set requested
> runlevel grep -q '\(^\| \)S\( \|$\)' /proc/cmdline &&
> runlevel='single' runsvchdir ${runlevel:-default} || runsvchdir
> default

Good tip! /proc/cmdline seems a smart way. Question 1 resolved. :-)

> utmp/wtmp support is done, I believe, through an external program,
> check out the getty*/finish scripts that are provided with the
> default distribution.

utmpset only clears lines from utmp, there is no way of setting anything 
as the name suggest.

-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* RE: using runit as init
       [not found]   ` <12EC84FDD73F4BD8A78E7501EB19F1E2@home.internal>
@ 2008-01-05  7:45     ` rehan khan
  2008-01-05 23:17       ` Bernhard Graf
  0 siblings, 1 reply; 37+ messages in thread
From: rehan khan @ 2008-01-05  7:45 UTC (permalink / raw)
  To: supervision

[-- Attachment #1: Type: text/plain, Size: 5073 bytes --]


You could also try the following:

http://fedorafastboot.wiki.sourceforge.net/The+Runlevel+Scheme

It is something I have been working on for a couple of months and is a little rough around the edges. Mainly developed for fedora but it is fairly generalised so I guess it can easily be ported. There are no 1, 2 and 3 scripts as described in the runit docs. Runit is controlled by inittab. Each runlevel change runs a runsvchdir command to a different directory. So for each service you are monitoring you will need a symlink for that service in the runlevel directories. You will need to put a 'runsvdir -P runlevel1' command somewhere early in the startup - I have it right after there is a r/w filesystem and selinux relabel or just as the last command in rc.sysinit.

so something like :

runsvchdir runlevel1
runsvdir -P /var/service

runlevelS can be symlinked to runlevel1 or have it's own directory.

 Init is still process 1 and init runlevels run as normal. No changes to the kernel parameters are neccessary (as it is still init). The checksrv.sh and checksrvgtk.sh (gui for x using zenity) scripts are quite useful in seeing what services are running/stopped or failing and they should work fine in Suse (there might be bashisms that need sorting to get them working ok). Annvix distro has a pretty good script for managing runit services (which works on fedora, but there are long delays (~2-3mins at least on fedora) while it figures out if a service is failing. The above scripts can get the status quicker although there is a small chance it might be inaccurate (1 in every 100 times it is run) for a failing service.

The init scripts are direct ports of the fedora scripts so they might only be helpful to you as additional examples of how to start a particular services.

I haven't done much work on the runlevel scheme as I mainly use the incremental scheme (which gets me to a desktop fast). If you try it I would appreciate any feedback on what works and more importantly what doesn't.

regs
Rehan

-----Original Message-----
From: Bernhard Graf [mailto:list-supervision@augensalat.de]
Sent: Sat 1/5/2008 00:15
To: supervision@list.skarnet.org
Subject: Re: using runit as init
 
Mike Buland wrote:

> In general, if you want to retain init-style runlevels, then you
> don't want to replace SysV init.  I have switched to runit entirely,
> it manages every service, and it does indeed support "runlevels." 
> See
> http://smarden.org/runit/runsvchdir.8.html for details.  Each runit
> runlevel is really just another service directory, runsvchdir changes
> which service directory you are using.  the runit "init" program is
> provided mainly for compatibility.
>
> There is no reason for runlevels to be numbered, and the kernel
> offers no support for runlevels, nor do any bootloaders.  When you
> pass 2, 3, 4, S, etc. to the kernel, it is ignored as an unrecognized
> parameter, but made available to running processes a number of ways. 
> Runit makes the commandline as well as any variables that are set on
> the kernel commandline available to the script 1 (and maybe 2 and 3,
> but I use it in 1).  Therefore if you set a variable in the
> bootloader on the kernel commandline named "runlevel" then you can
> use that in script 1 to change your runlevel.
>
> However, I would say based on the way you are using runit as process
> 1, you would probably be better off sticking with sysvinit and adding
> runsvdir to your inittab (a script very similar to the example script
> 2 provided with runit is just what you want).

As I wrote in my previous email I want an easy way to switch from SysV 
init to runit (or may be another one that supports process 
supervision). Since I have to care for a couple of production systems 
this switch must meet some conditions:
- full support for the old init system
- keep manual adjusting to a minimum

ATM I'm starting daemontools svscan from inttab on those hosts in 
addition to the normal SysV init stuff. This works but is not really 
pretty - I'd like to have a more consistent administration interface.

The goal is to install runit with RPM, reboot and then link the 
daemontools things into runit's service directory.

> As for the runlevel code, I got this from this very list a long time
> ago, give it a shot in script 1:
>
> # Set runlevel to:
> # - single       if kernel has param: S
> # - RUNLEVELNAME if kernel has param: runlevel=RUNLEVELNAME
> # - default      if kernel has no params or unable to set requested
> runlevel grep -q '\(^\| \)S\( \|$\)' /proc/cmdline &&
> runlevel='single' runsvchdir ${runlevel:-default} || runsvchdir
> default

Good tip! /proc/cmdline seems a smart way. Question 1 resolved. :-)

> utmp/wtmp support is done, I believe, through an external program,
> check out the getty*/finish scripts that are provided with the
> default distribution.

utmpset only clears lines from utmp, there is no way of setting anything 
as the name suggest.

-- 
Bernhard Graf





[-- Attachment #2: Type: text/html, Size: 6225 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-05  7:45     ` rehan khan
@ 2008-01-05 23:17       ` Bernhard Graf
  2008-01-08  7:11         ` Vincent Danen
  0 siblings, 1 reply; 37+ messages in thread
From: Bernhard Graf @ 2008-01-05 23:17 UTC (permalink / raw)
  To: supervision

rehan khan wrote:

> You could also try the following:
>
> http://fedorafastboot.wiki.sourceforge.net/The+Runlevel+Scheme

Thanks for the pointer.
I'm checking all related sites and fedorafastboot was one of the first 
that I looked at. Has some nice infos and inspirations.

-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-05 23:17       ` Bernhard Graf
@ 2008-01-08  7:11         ` Vincent Danen
  2008-01-08 22:28           ` Bernhard Graf
  0 siblings, 1 reply; 37+ messages in thread
From: Vincent Danen @ 2008-01-08  7:11 UTC (permalink / raw)
  To: Bernhard Graf; +Cc: supervision

[-- Attachment #1: Type: text/plain, Size: 1223 bytes --]

* Bernhard Graf <list-supervision@augensalat.de> [2008-01-06 00:17:51 +0100]:

>rehan khan wrote:
>
>> You could also try the following:
>>
>> http://fedorafastboot.wiki.sourceforge.net/The+Runlevel+Scheme
>
>Thanks for the pointer.
>I'm checking all related sites and fedorafastboot was one of the first 
>that I looked at. Has some nice infos and inspirations.

You may also be interested in what Annvix has done.  Annvix has used
runit for init for quite a few years now, and very successfully.  It
still handles traditional "initscripts", but doesn't handle number-based
runlevels or runlevel switching (largely due to the fact that Annvix has
no GUI and thus has no real need for that).

But the /sbin/rc script is completely custom, as are the 1, 2, and 3
scripts (using execline).  There's no real documentation for anything,
but (I hope) the code is clean enough to be understandable:

http://svn.annvix.org/cgi-bin/viewvc.cgi/tools/runit/trunk/

That's got the relevant runit scripts we use to handle init.  Note that
we don't completely do away with sysvinit... there are some useful tools
that come with it; we just don't use sysvinit's init.

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-08  7:11         ` Vincent Danen
@ 2008-01-08 22:28           ` Bernhard Graf
  2008-01-09  2:05             ` Vincent Danen
  0 siblings, 1 reply; 37+ messages in thread
From: Bernhard Graf @ 2008-01-08 22:28 UTC (permalink / raw)
  To: supervision

Vincent Danen wrote:

> You may also be interested in what Annvix has done.  Annvix has used
> runit for init for quite a few years now, and very successfully.  It
> still handles traditional "initscripts", but doesn't handle
> number-based runlevels or runlevel switching (largely due to the fact
> that Annvix has no GUI and thus has no real need for that).

Of course I also discovered Annvix already. Nice work (and a lot of, I 
suppose). Brought me to execline among other things, which I knew for 
quite some time, but never actually used so far.

> That's got the relevant runit scripts we use to handle init.  Note
> that we don't completely do away with sysvinit... there are some
> useful tools that come with it; we just don't use sysvinit's init.

Actually I'm missing two essential things in runit, that would allow to 
replace "standard" (Linux/SysV) init while keeping the start/Stop 
scripts in the beginning:

- possibility to set no-respawn mode for a service, instead run command
  (e.g. "/etc/init.d/service start") when service appears in runsvdir's
  directory and run other command (e.g. "/etc/init.d/service stop) when
  it disappears
- service dependencies

Another interesting project is ninit [1] and its ancestor minit [2]

[1] http://riemann.fmi.uni-sofia.bg/ninit/
[2] http://www.fefe.de/minit/

Those have dependencies and can start services w/o respawn, but they 
don't have a svscan-like function, they can start dependend services at 
start-up - later services must be brought up and down by [mn]svc 
command. Potentially this would fit better into an existing SysV-based
init scheme, because "/etc/init.d/service start" would just execute 
[mn]svc -u service and "/etc/init.d/service stop" [mn]svc -d service.

Still runit is much more elegant IMHO by starting and removing services 
as they appear and disappear - this would make runlevels really easy.
-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-08 22:28           ` Bernhard Graf
@ 2008-01-09  2:05             ` Vincent Danen
  2008-01-09 23:06               ` Bernhard Graf
  0 siblings, 1 reply; 37+ messages in thread
From: Vincent Danen @ 2008-01-09  2:05 UTC (permalink / raw)
  To: Bernhard Graf; +Cc: supervision

[-- Attachment #1: Type: text/plain, Size: 3407 bytes --]

* Bernhard Graf <list-supervision@augensalat.de> [2008-01-08 23:28:06 +0100]:

>Vincent Danen wrote:
>
>> You may also be interested in what Annvix has done.  Annvix has used
>> runit for init for quite a few years now, and very successfully.  It
>> still handles traditional "initscripts", but doesn't handle
>> number-based runlevels or runlevel switching (largely due to the fact
>> that Annvix has no GUI and thus has no real need for that).
>
>Of course I also discovered Annvix already. Nice work (and a lot of, I 
>suppose). Brought me to execline among other things, which I knew for 
>quite some time, but never actually used so far.

Yeah, Annvix is a lot of work.  =)  And thank you.  Execline is
definitely nice..  using it with dietlibc for some stuff (like mingetty
and runit itself) really keeps the overhead down.

>> That's got the relevant runit scripts we use to handle init.  Note
>> that we don't completely do away with sysvinit... there are some
>> useful tools that come with it; we just don't use sysvinit's init.
>
>Actually I'm missing two essential things in runit, that would allow to 
>replace "standard" (Linux/SysV) init while keeping the start/Stop 
>scripts in the beginning:
>
>- possibility to set no-respawn mode for a service, instead run command
>  (e.g. "/etc/init.d/service start") when service appears in runsvdir's
>  directory and run other command (e.g. "/etc/init.d/service stop) when
>  it disappears

Wasn't this addressed already?  Or maybe I'm missing something?  Can't
you use the down script?  I.e. you could use run to start it and once it
exits, doesn't runit look for the down script to execute prior to
restarting?  If so, you could use a 'sv down' or something similar (maybe
even something as simple as touching 'down') to prevent it from starting
again, right?

Of course, if you do something like that, you might need to write a
wrapper that would remove the down file before calling 'svc up'.

>- service dependencies

We've tackled this somewhat, but Annvix has a frontend called srv which
handles the dependencies.  It's quite crude and definitely could use
some work, but we've found there are very few things outside of nfs that
have strict "service X needs to be up and running before service Y"
dependencies.

>Another interesting project is ninit [1] and its ancestor minit [2]
>
>[1] http://riemann.fmi.uni-sofia.bg/ninit/
>[2] http://www.fefe.de/minit/
>
>Those have dependencies and can start services w/o respawn, but they 
>don't have a svscan-like function, they can start dependend services at 
>start-up - later services must be brought up and down by [mn]svc 
>command. Potentially this would fit better into an existing SysV-based
>init scheme, because "/etc/init.d/service start" would just execute 
>[mn]svc -u service and "/etc/init.d/service stop" [mn]svc -d service.

Never looked at these.  svscan is definitely useful.  Do they supervise
services as well?

>Still runit is much more elegant IMHO by starting and removing services 
>as they appear and disappear - this would make runlevels really easy.

For me, I'm satisfied with runit -- it does the job and it does it
extremely well.  We've been using it as the defacto init system on
Annvix for quite a few years and I can't see that changing anytime soon.

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09  2:05             ` Vincent Danen
@ 2008-01-09 23:06               ` Bernhard Graf
  2008-01-09 23:34                 ` Mike Buland
  2008-01-09 23:35                 ` KORN Andras
  0 siblings, 2 replies; 37+ messages in thread
From: Bernhard Graf @ 2008-01-09 23:06 UTC (permalink / raw)
  To: supervision

Vincent Danen wrote:

> Yeah, Annvix is a lot of work.  =)  And thank you.  Execline is
> definitely nice..  using it with dietlibc for some stuff (like
> mingetty and runit itself) really keeps the overhead down.

I've had no luck with execline and dietlibc: Got "segmentation fault" 
errors that went away after building execline with glibc.

Of course I should test this a bit further and file an error report...

> >> That's got the relevant runit scripts we use to handle init.  Note
> >> that we don't completely do away with sysvinit... there are some
> >> useful tools that come with it; we just don't use sysvinit's init.
> >
> >Actually I'm missing two essential things in runit, that would allow
> > to replace "standard" (Linux/SysV) init while keeping the
> > start/Stop scripts in the beginning:
> >
> >- possibility to set no-respawn mode for a service, instead run
> > command (e.g. "/etc/init.d/service start") when service appears in
> > runsvdir's directory and run other command (e.g.
> > "/etc/init.d/service stop) when it disappears
>
> Wasn't this addressed already?  Or maybe I'm missing something? 
> Can't you use the down script?  I.e. you could use run to start it
> and once it exits, doesn't runit look for the down script to execute
> prior to restarting?  If so, you could use a 'sv down' or something
> similar (maybe even something as simple as touching 'down') to
> prevent it from starting again, right?
>
> Of course, if you do something like that, you might need to write a
> wrapper that would remove the down file before calling 'svc up'.

I would like to just link init scripts into runsvdir's directory, so 
they are started automatically. But as we know init scripts terminate 
and runsv would restart them immediately. I'd like to have a 
non-respawn mode for runsv.
I think this could be realised by a wrapper that is called by runsv, 
runs the SysV start script and then waits for SIGTERM.

But maybe I'm missing something and there is already a way to achieve 
this.

> >- service dependencies
>
> We've tackled this somewhat, but Annvix has a frontend called srv
> which handles the dependencies.  It's quite crude and definitely
> could use some work, but we've found there are very few things
> outside of nfs that have strict "service X needs to be up and running
> before service Y" dependencies.

About service dependencies: There is a link on Gerrit's site to an IMHO 
elegant way: [3], realised in [4]. This concept would require changing 
how runsvdir starts services.

Having all this it would be simple to convert the links in directories
/etc/init.d/rc[0123456S].d into correspondig links 
in /etc/runit/runsvdir/[0123456S] and replace init-daemon-install-tools 
like chkconfig or insserv by runit-compliant versions.
So no need to change all daemons with those init.d start-stop scripts. 
Even those that are installed later would continue to work.

> >Another interesting project is ninit [1] and its ancestor minit [2]
> >
> >[1] http://riemann.fmi.uni-sofia.bg/ninit/
> >[2] http://www.fefe.de/minit/

> Never looked at these.  svscan is definitely useful.  Do they
> supervise services as well?

Of course they do. Those don't respawn terminated processes by default, 
but can be told to do so. Supervising is done by [mn]init directly, so 
there are not n supervisor processes for n supervised processes, but 
only one process that usually runs at pid 1 and supervises all n 
processes.

Another interesting feature: You don't have to put an executable program 
into run, instead run can be a softlink to a program and params for 
this program are simply stored in a file named params.

Instead of starting a getty in a script:
#!/bin/sh
exec /usr/bin/setsid /sbin/mingetty --noclear tty1

just symlink run to /usr/bin/setsid  and put into params:
/sbin/mingetty
--noclear
tty1

Thus no shell (or any other interpreter like execline) has to be loaded.

[mn]init doesn't poll the service directory as svscan/runsvdir do.
Can be seen as advantage, when you don't want permanent disk access.


[3] http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/index.html
[4] http://www.kernel.org/pub/linux/utils/util-linux/
-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09 23:06               ` Bernhard Graf
@ 2008-01-09 23:34                 ` Mike Buland
  2008-01-09 23:51                   ` Charlie Brady
                                     ` (2 more replies)
  2008-01-09 23:35                 ` KORN Andras
  1 sibling, 3 replies; 37+ messages in thread
From: Mike Buland @ 2008-01-09 23:34 UTC (permalink / raw)
  To: supervision

Regaurding service dependancies, Richard Gooch's method is actually 
unnecesarry in a sytem such as runit, and service dependancy is already 
handled in an almost identicle way, but with less system overhead.  

At the begining of a run script just start the services you depend on, and 
exit if the sv command returns non-zero.  I.e. from hal's run script:
sv start /etc/service/dbus || exit 1

If this feels less elegent to you than "need X" you could always wrap it with 
another script. that would just make your scripts look prettier.  Also, this 
has a few interesting advantages.  Firstly, sv start X will wait be default 
for 7 seconds for the service to come up, even if it's already in the process 
of starting, so there are never any conflicts.  If the service is already up, 
it returns immediately indicating success.  If the service has not come up 
yet, or something has gone wrong, it will return non-zero and the hal script 
(in this example) will also exit non-zero, indicating an error and runit will 
retry a second or two later.

This is very similar to the need method, except that in my opinion it has a 
few advantages, especially relating to the asynchronous nature of the way 
runit works.  In the above example, there is no extra accounting being done, 
no extra programs need to be written, it is efficient, safe, and will keep 
trying in case there's a temporary error on either service.

As far as running SysV init style scripts, usually handled on modern linux 
systems (redhat, slackware, debian, etc) by the master rc script, are you 
suggesting that rc, to control runlevel changes be placed in it's own 
service, or that each SysV "service" gets it's own runit service?

In the former case, I can't understand why you would want the rc script to be 
handled by a service, you're right, you're gaining very little, and making 
extra work.  The two ideologies are diametrically opposed, you can't really 
reconcile them, nor should you be able to.  However, using runit as init 
gives you three scripts that run that let you do anything you want.  When I 
was transitioning from SysV to Runit inits I simply called rc in script 1 and 
3 and left 2 more or less alone from the reference.

If you're referring to the later case, you'll need to do a lot more work to 
synchronize the rc scripts and the runit services, I can't imagine how just 
making symlinks between the init.d runlevel directories and runsvdir service 
directories will actually work.

Finally, if you really do want to start an rc script and make it appear as 
though it were a normal runit system service, then I would reccomend starting 
it in a run script, then executing a command that blocks forever.  The 
service will use effectively no CPU time, but will stay running, forever.  
Then you can call the corrisponding rc 6 or whatever you want in the finish 
script of the service.

It's really the closest I can come to a cleaner hack, but really, I would go 
with the former option or just sack the SysV style and switch over entirely 
to runit, it's so much nicer and anyone can make the run scripts, most of 
which are only a couple lines...

--Mike

P.S.  If you really want to go with the "down" idea, and not have a dummy 
service running that you can stop later, and that runit can stop 
automatically for you at system shutdown, etc.  Then consider just adding a 
couple of lines such as:

touch /etc/rc-compat/down
sv start /etc/rc-compat

into your script 1, don't put it in 2, if 2 dies it's restarted, and rc 
scripts shouldn't be called more than once in a row if it's avoidable.  The 
above case will ensure there is a down file, and start the service, when the 
service finishes, it will not start.  Be aware that this is basically the 
uber-complex version of just calling the rc script by hand, and really gives 
you nothing except concurrent execution of the rc scripts and the other runit 
processes, which could also be accomplished in bash with a &.

On Wednesday 09 January 2008 04:06:54 pm Bernhard Graf wrote:
> Vincent Danen wrote:
> > Yeah, Annvix is a lot of work.  =)  And thank you.  Execline is
> > definitely nice..  using it with dietlibc for some stuff (like
> > mingetty and runit itself) really keeps the overhead down.
>
> I've had no luck with execline and dietlibc: Got "segmentation fault"
> errors that went away after building execline with glibc.
>
> Of course I should test this a bit further and file an error report...
>
> > >> That's got the relevant runit scripts we use to handle init.  Note
> > >> that we don't completely do away with sysvinit... there are some
> > >> useful tools that come with it; we just don't use sysvinit's init.
> > >
> > >Actually I'm missing two essential things in runit, that would allow
> > > to replace "standard" (Linux/SysV) init while keeping the
> > > start/Stop scripts in the beginning:
> > >
> > >- possibility to set no-respawn mode for a service, instead run
> > > command (e.g. "/etc/init.d/service start") when service appears in
> > > runsvdir's directory and run other command (e.g.
> > > "/etc/init.d/service stop) when it disappears
> >
> > Wasn't this addressed already?  Or maybe I'm missing something?
> > Can't you use the down script?  I.e. you could use run to start it
> > and once it exits, doesn't runit look for the down script to execute
> > prior to restarting?  If so, you could use a 'sv down' or something
> > similar (maybe even something as simple as touching 'down') to
> > prevent it from starting again, right?
> >
> > Of course, if you do something like that, you might need to write a
> > wrapper that would remove the down file before calling 'svc up'.
>
> I would like to just link init scripts into runsvdir's directory, so
> they are started automatically. But as we know init scripts terminate
> and runsv would restart them immediately. I'd like to have a
> non-respawn mode for runsv.
> I think this could be realised by a wrapper that is called by runsv,
> runs the SysV start script and then waits for SIGTERM.
>
> But maybe I'm missing something and there is already a way to achieve
> this.
>
> > >- service dependencies
> >
> > We've tackled this somewhat, but Annvix has a frontend called srv
> > which handles the dependencies.  It's quite crude and definitely
> > could use some work, but we've found there are very few things
> > outside of nfs that have strict "service X needs to be up and running
> > before service Y" dependencies.
>
> About service dependencies: There is a link on Gerrit's site to an IMHO
> elegant way: [3], realised in [4]. This concept would require changing
> how runsvdir starts services.
>
> Having all this it would be simple to convert the links in directories
> /etc/init.d/rc[0123456S].d into correspondig links
> in /etc/runit/runsvdir/[0123456S] and replace init-daemon-install-tools
> like chkconfig or insserv by runit-compliant versions.
> So no need to change all daemons with those init.d start-stop scripts.
> Even those that are installed later would continue to work.
>
> > >Another interesting project is ninit [1] and its ancestor minit [2]
> > >
> > >[1] http://riemann.fmi.uni-sofia.bg/ninit/
> > >[2] http://www.fefe.de/minit/
> >
> > Never looked at these.  svscan is definitely useful.  Do they
> > supervise services as well?
>
> Of course they do. Those don't respawn terminated processes by default,
> but can be told to do so. Supervising is done by [mn]init directly, so
> there are not n supervisor processes for n supervised processes, but
> only one process that usually runs at pid 1 and supervises all n
> processes.
>
> Another interesting feature: You don't have to put an executable program
> into run, instead run can be a softlink to a program and params for
> this program are simply stored in a file named params.
>
> Instead of starting a getty in a script:
> #!/bin/sh
> exec /usr/bin/setsid /sbin/mingetty --noclear tty1
>
> just symlink run to /usr/bin/setsid  and put into params:
> /sbin/mingetty
> --noclear
> tty1
>
> Thus no shell (or any other interpreter like execline) has to be loaded.
>
> [mn]init doesn't poll the service directory as svscan/runsvdir do.
> Can be seen as advantage, when you don't want permanent disk access.
>
>
> [3] http://www.atnf.csiro.au/people/rgooch/linux/boot-scripts/index.html
> [4] http://www.kernel.org/pub/linux/utils/util-linux/



^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09 23:06               ` Bernhard Graf
  2008-01-09 23:34                 ` Mike Buland
@ 2008-01-09 23:35                 ` KORN Andras
  2008-01-10  8:39                   ` Bernhard Graf
  1 sibling, 1 reply; 37+ messages in thread
From: KORN Andras @ 2008-01-09 23:35 UTC (permalink / raw)
  To: supervision

On Thu, Jan 10, 2008 at 12:06:54AM +0100, Bernhard Graf wrote:

> I would like to just link init scripts into runsvdir's directory, so 
> they are started automatically. But as we know init scripts terminate 
> and runsv would restart them immediately. I'd like to have a 
> non-respawn mode for runsv.

So have your script remove the symlink to the directory that contains it
from /service before it exits (or in ./finish).

> Having all this it would be simple to convert the links in directories
> /etc/init.d/rc[0123456S].d into correspondig links 
> in /etc/runit/runsvdir/[0123456S] and replace init-daemon-install-tools 
> like chkconfig or insserv by runit-compliant versions.
> So no need to change all daemons with those init.d start-stop scripts. 
> Even those that are installed later would continue to work.

I think this is a misguided attempt to provide "uniformity". Traditional
initscript are quite horrible; I invite you to read, for example, Ubuntu's
apache2 initscript and be appalled at all the unnecessary complexity and
race conditions galore.

One of the chief motivations for switching away from an init-like system,
where services daemonise, to a runit-like system where they are supervised
by their parent process is, or at least should be, the desire to get away
from those race conditions and the fragility of it all. To get away from pid
lotto.

What reason would justify importing that mess into this clean new service
supervision and system startup scheme?

FWIW, services with initscripts you haven't converted to runit services
naturally continue to work. You can just do "/etc/init.d/foo start" even if
your process 1 is runit. Runit can even start these services automatically.
You can even arrange for them to be started and stopped on runlevel changes
(runsvchdir's) with the help of some kludgery (e.g. like the one I suggested
above).

The real solution, however, is of course to convert the messy and complex
initscripts into neat and clean run scripts. You only need to do this once
per daemon, not once per daemon per computer. Chances are someone already
has a run script for your daemon, so you needn't really do anything at all.

Why would you want to keep initscripts around to the extent of creating
one-shot runit services? Maybe there is a different solution.

Andras

-- 
                 Andras Korn <korn at chardonnay.math.bme.hu>
                 <http://chardonnay.math.bme.hu/~korn/>	QOTD:
           If at first you don't succeed, you must be a programmer.


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09 23:34                 ` Mike Buland
@ 2008-01-09 23:51                   ` Charlie Brady
  2008-01-10  9:22                     ` Bernhard Graf
  2008-01-10  9:20                   ` Bernhard Graf
       [not found]                   ` <C1A5323F485E4A75B75ADB58B664E35E@home.internal>
  2 siblings, 1 reply; 37+ messages in thread
From: Charlie Brady @ 2008-01-09 23:51 UTC (permalink / raw)
  To: supervision


On Wed, 9 Jan 2008, Mike Buland wrote:

> Finally, if you really do want to start an rc script and make it appear as
> though it were a normal runit system service, then I would reccomend starting
> it in a run script, then executing a command that blocks forever.  The
> service will use effectively no CPU time, but will stay running, forever.

For instance, compile:

main()
{
    pause();
}

into /usr/local/bin/pause, and then have a run script:

#! /bin/sh
do_something_you_care_about
exec /usr/local/bin/pause



^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09 23:35                 ` KORN Andras
@ 2008-01-10  8:39                   ` Bernhard Graf
  0 siblings, 0 replies; 37+ messages in thread
From: Bernhard Graf @ 2008-01-10  8:39 UTC (permalink / raw)
  To: supervision

KORN Andras wrote:

> > I would like to just link init scripts into runsvdir's directory,
> > so they are started automatically. But as we know init scripts
> > terminate and runsv would restart them immediately. I'd like to
> > have a non-respawn mode for runsv.
>
> So have your script remove the symlink to the directory that contains
> it from /service before it exits (or in ./finish).

In the sense of self-modifying code? No, thanks!

> > Having all this it would be simple to convert the links in
> > directories /etc/init.d/rc[0123456S].d into correspondig links
> > in /etc/runit/runsvdir/[0123456S] and replace
> > init-daemon-install-tools like chkconfig or insserv by
> > runit-compliant versions.
> > So no need to change all daemons with those init.d start-stop
> > scripts. Even those that are installed later would continue to
> > work.
>
> I think this is a misguided attempt to provide "uniformity".
> Traditional initscript are quite horrible; I invite you to read, for
> example, Ubuntu's apache2 initscript and be appalled at all the
> unnecessary complexity and race conditions galore.

I'm not advocating init scripts. Compared to daemontools scripts they 
really suck. As I already wrote several times I'm looking for a way to 
replace traditional init by something than *can* supervise but also 
handles the old init scripts correctly.

> One of the chief motivations for switching away from an init-like
> system, where services daemonise, to a runit-like system where they
> are supervised by their parent process is, or at least should be, the
> desire to get away from those race conditions and the fragility of it
> all. To get away from pid lotto.
>
> What reason would justify importing that mess into this clean new
> service supervision and system startup scheme?

I really like the way runit works. I'm a long time user of DJBware and 
know the concept of daemontools quite well.
Maybe I'd be happy with /package too.

But I'm a lazy boy, using a widely accepted Linux distribution saves me 
a lot of time. Therefore I want a drop-in replacement for SysV init.
Most people are like me. And that's the reason why most people stick 
with archaic software like SysV init. Give them an alternative that 
simply works in their current environment and is better / cleaner / 
smaller, and more developers / package maintainers would start putting 
daemontools-like run scripts into their packages, saving you the work 
of adjusting your setup on every host over and over again.

-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09 23:34                 ` Mike Buland
  2008-01-09 23:51                   ` Charlie Brady
@ 2008-01-10  9:20                   ` Bernhard Graf
  2008-01-10 20:06                     ` Mike Buland
       [not found]                     ` <85040AD9CA634253A8FDB9F7DA6BD200@home.internal>
       [not found]                   ` <C1A5323F485E4A75B75ADB58B664E35E@home.internal>
  2 siblings, 2 replies; 37+ messages in thread
From: Bernhard Graf @ 2008-01-10  9:20 UTC (permalink / raw)
  To: supervision

On Thursday 10 January 2008 00:34, Mike Buland wrote:
> Regaurding service dependancies, Richard Gooch's method is actually
> unnecesarry in a sytem such as runit, and service dependancy is
> already handled in an almost identicle way, but with less system
> overhead.
>
> At the begining of a run script just start the services you depend
> on, and exit if the sv command returns non-zero.  I.e. from hal's run
> script: sv start /etc/service/dbus || exit 1

I tend to object:

You mean starting a bunch of programs over and over again, because many
of them quit immediatly is less system overhead than starting programs
that pause until those programs they depend on are started?

Also imagine one service A that depends on service B, C and D. You have
to put the checks for availability of B, C, D into A's run script.
In a complex system more services depend on B, C and D and probably
others. You have to put the same checks (and others) in all those
services. Also those checks tend to be more complicated than just saying
"need B C D".

> As far as running SysV init style scripts, usually handled on modern
> linux systems (redhat, slackware, debian, etc) by the master rc
> script, are you suggesting that rc, to control runlevel changes be
> placed in it's own service, or that each SysV "service" gets it's own
> runit service?

The latter.

> If you're referring to the later case, you'll need to do a lot more
> work to synchronize the rc scripts and the runit services, I can't
> imagine how just making symlinks between the init.d runlevel
> directories and runsvdir service directories will actually work.

In current distributions (hopefully) an init.d script is configured
using a program that sets the appropriate symlinks. The information for
run levels and dependencies are taken from the init scripts [5], [6].

If you'd replace init by runit, you had to replace those installtion
programs as well.

This installer would install the example from [5] as:

# ls -l /etc/runit/sv/lsb-ourdb
-rw-------   1 root root      0 Jan 10 09:00 initd
-rw-------   1 root root     30 Jan 10 09:00 need
lrwxrwxrwx   1 root root     19 Jan 10 09:00 run -> /etc/init.d/lsb-ourdb
# cat /etc/runit/sv/lsb-ourdb/need
$local_fs
$network
$remote_fs
# ls -l /etc/runit/runsvdir/2/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00 /etc/runit/runsvdir/2/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
# ls -l /etc/runit/runsvdir/3/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00 /etc/runit/runsvdir/3/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
# ls -l /etc/runit/runsvdir/4/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00 /etc/runit/runsvdir/4/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
# ls -l /etc/runit/runsvdir/5/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00 /etc/runit/runsvdir/5/lsb-ourdb -> /etc/runit/sv/lsb-ourdb

File initd is a flag for runit to run the script with argument "start"
first, then wait for SIGINT and run the script with argument "stop".

File need contains services that runit has to be started first.
"$service" for "meta services" as set with "provide" in Gooch's concept.

> Finally, if you really do want to start an rc script and make it
> appear as though it were a normal runit system service, then I would
> reccomend starting it in a run script, then executing a command that
> blocks forever.  The service will use effectively no CPU time, but
> will stay running, forever. Then you can call the corrisponding rc 6
> or whatever you want in the finish script of the service.

I will try this, and see how far I get...


[5] http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/initscrcomconv.html
[6] http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-generic/initsrcinstrm.html

-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-09 23:51                   ` Charlie Brady
@ 2008-01-10  9:22                     ` Bernhard Graf
  0 siblings, 0 replies; 37+ messages in thread
From: Bernhard Graf @ 2008-01-10  9:22 UTC (permalink / raw)
  To: supervision

Charlie Brady wrote:

> On Wed, 9 Jan 2008, Mike Buland wrote:
> > Finally, if you really do want to start an rc script and make it
> > appear as though it were a normal runit system service, then I
> > would reccomend starting it in a run script, then executing a
> > command that blocks forever.  The service will use effectively no
> > CPU time, but will stay running, forever.
>
> For instance, compile:
>
> main()
> {
>     pause();
> }
>
> into /usr/local/bin/pause, and then have a run script:
>
> #! /bin/sh
> do_something_you_care_about
> exec /usr/local/bin/pause

Nice. Will try it.
-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-10  9:20                   ` Bernhard Graf
@ 2008-01-10 20:06                     ` Mike Buland
  2008-01-11  7:58                       ` Bernhard Graf
       [not found]                     ` <85040AD9CA634253A8FDB9F7DA6BD200@home.internal>
  1 sibling, 1 reply; 37+ messages in thread
From: Mike Buland @ 2008-01-10 20:06 UTC (permalink / raw)
  To: supervision



On Thursday 10 January 2008 02:20:51 am Bernhard Graf wrote:
> On Thursday 10 January 2008 00:34, Mike Buland wrote:
> > Regaurding service dependancies, Richard Gooch's method is actually
> > unnecesarry in a sytem such as runit, and service dependancy is
> > already handled in an almost identicle way, but with less system
> > overhead.
> >
> > At the begining of a run script just start the services you depend
> > on, and exit if the sv command returns non-zero.  I.e. from hal's run
> > script: sv start /etc/service/dbus || exit 1
>
> I tend to object:
>
> You mean starting a bunch of programs over and over again, because many
> of them quit immediatly is less system overhead than starting programs
> that pause until those programs they depend on are started?
>
> Also imagine one service A that depends on service B, C and D. You have
> to put the checks for availability of B, C, D into A's run script.
> In a complex system more services depend on B, C and D and probably
> others. You have to put the same checks (and others) in all those
> services. Also those checks tend to be more complicated than just saying
> "need B C D".

I'm honestly not sure how your need solution covers these mysterious more 
complicated, but you're just using need in your example below, so...

Anyway, need will not block forever, it needs to do a number of different 
things periodically to ensure that the service has not gone into an 
unexpected state, etc, etc.  Starting a script once every seven seconds, as I 
mentioned, or at a larger interval, as you suggested, I do not feel is 
significant system overhead.  If you're using bash, bash will be using about 
2 megs of memory (on my machine), most of which is actually shared libraries, 
and not overhead for bash (libc, ncurses, libreadline, libhistory, etc), and 
no cpu time for that seven seconds (or more).  This is so little cpu usage 
that even on the oldest machines I've run runit on it doesn't even register 
as a single ms of cpu usage, if your desktop/server is worse...then stick 
with init, I guess...

Yes, for me, that is completely acceptable.  I don't agree that the overhead 
is necessarily any larger when restarting bash (cached exe image, fast 
startup times, and most of the "wasted memory" isn't actually wasted) 
compared to a need script running forever.  And frankly, I like the 
opportunity to do other things periodically when the script may fail a 
dependency check, such as taking it down if it has failed for too long, or 
clearing out a cache file that may have gotten in the way, an errant pid 
file, etc, etc, etc.  These are things that you cannot do with a need script.

Bottom line, the runit method is more flexible, and even if it uses more 
resources, it's fine by me (and it's still much less than traditional rc).

>
> > As far as running SysV init style scripts, usually handled on modern
> > linux systems (redhat, slackware, debian, etc) by the master rc
> > script, are you suggesting that rc, to control runlevel changes be
> > placed in it's own service, or that each SysV "service" gets it's own
> > runit service?
>
> The latter.
>
> > If you're referring to the later case, you'll need to do a lot more
> > work to synchronize the rc scripts and the runit services, I can't
> > imagine how just making symlinks between the init.d runlevel
> > directories and runsvdir service directories will actually work.
>
> In current distributions (hopefully) an init.d script is configured
> using a program that sets the appropriate symlinks. The information for
> run levels and dependencies are taken from the init scripts [5], [6].
>
> If you'd replace init by runit, you had to replace those installtion
> programs as well.
>
> This installer would install the example from [5] as:
>
> # ls -l /etc/runit/sv/lsb-ourdb
> -rw-------   1 root root      0 Jan 10 09:00 initd
> -rw-------   1 root root     30 Jan 10 09:00 need
> lrwxrwxrwx   1 root root     19 Jan 10 09:00 run -> /etc/init.d/lsb-ourdb
> # cat /etc/runit/sv/lsb-ourdb/need
> $local_fs
> $network
> $remote_fs
> # ls -l /etc/runit/runsvdir/2/lsb-ourdb
> lrwxrwxrwx   1 root root     19 Jan 10 09:00
> /etc/runit/runsvdir/2/lsb-ourdb -> /etc/runit/sv/lsb-ourdb # ls -l
> /etc/runit/runsvdir/3/lsb-ourdb
> lrwxrwxrwx   1 root root     19 Jan 10 09:00
> /etc/runit/runsvdir/3/lsb-ourdb -> /etc/runit/sv/lsb-ourdb # ls -l
> /etc/runit/runsvdir/4/lsb-ourdb
> lrwxrwxrwx   1 root root     19 Jan 10 09:00
> /etc/runit/runsvdir/4/lsb-ourdb -> /etc/runit/sv/lsb-ourdb # ls -l
> /etc/runit/runsvdir/5/lsb-ourdb
> lrwxrwxrwx   1 root root     19 Jan 10 09:00
> /etc/runit/runsvdir/5/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
>
> File initd is a flag for runit to run the script with argument "start"
> first, then wait for SIGINT and run the script with argument "stop".
>
> File need contains services that runit has to be started first.
> "$service" for "meta services" as set with "provide" in Gooch's concept.
>
> > Finally, if you really do want to start an rc script and make it
> > appear as though it were a normal runit system service, then I would
> > reccomend starting it in a run script, then executing a command that
> > blocks forever.  The service will use effectively no CPU time, but
> > will stay running, forever. Then you can call the corrisponding rc 6
> > or whatever you want in the finish script of the service.
>
> I will try this, and see how far I get...
>
>
> [5]
> http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-ge
>neric/initscrcomconv.html [6]
> http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core-ge
>neric/initsrcinstrm.html

I'm not actually going to pursue this.  My opinion, and I know I'm not alone, 
is that you should really let runit run the rc script and let the normal init 
scripts do the only thing they're mediocre at, or switch over completely to 
runit and drop the lsb/rc/sysv style.  Honestly, for someone who is 
self-professed "lazy" you're going through a lot of unnecessary work to hack 
a broken system into a nice one and lose all of the things except the cli 
that runit provides.  You won't actually gain nice script supervision this 
way, you won't be able to tell which services are really up or not except by 
testing the normal init status functions of the old init scripts, you will be 
able to "start" and "stop" the scripts, but it doesn't buy you anything 
useful.

Honestly, I was against sysvinit/rc style boot scripts for a number of 
reasons.  They're a pain to write and maintain, they offer no guarantees, 
there's an amazing library of bash scripts with every distro that *try* to 
sotra find programs that have been disowned completely, and there are no 
guarantees in that.

I wouldn't want a system that hacked init scripts into runit services to 
become popular, because it doesn't add anything, and it makes runit look bad.  
Runit is a wonderful new (in that not many people have seen the light yet) 
system that addresses many, many problems with other init systems and service 
systems.  I have seen several software packages that were absolutely 
brilliant die before their time because of improper use making them "look 
bad," and I don't want that to happen to runit.

If what you come up with really helps and is a definite improvement in doing 
things the runit way, then I applaud you, and if what you work on in the 
meantime works for you, then great, I'm very happy for you, but I'm not going 
to argue the point, I just hope that it doesn't wind up like so many 
marriages of technology and bringing out the worst in both parent systems.

--Mike

P.S.  I'm not trying to start a flame war or anything, I just really like 
runit, and don't want to see it fall into disuse due to misconceptions about 
how it works.  The beuty of it, the simplicty, and the assurances it provides 
are all thrown out the window by having the runit scritps start and disown 
processes of their own, then try to hunt them down later.  Please at least 
consider it.  I won't pontificate at you again ;)


^ permalink raw reply	[flat|nested] 37+ messages in thread

* RE: using runit as init
       [not found]                   ` <C1A5323F485E4A75B75ADB58B664E35E@home.internal>
@ 2008-01-10 21:56                     ` rehan khan
  0 siblings, 0 replies; 37+ messages in thread
From: rehan khan @ 2008-01-10 21:56 UTC (permalink / raw)
  To: Bernhard Graf, supervision

Starting services over and over seems odd to me too. Unfortunately I
have come to believe that this is unavoidable in any supervision system
without getting every project/programmer of every app that needs an init
script to put in some code to support a 'wait/pause' feature. Of course
this will never happen. Getting runit to do it will bloat runit beyond
any usefulness.

I don't believe runit is a good replacement for init because it does not
solve all the challenges an init system may need to overcome. However
the things that it does do it does really well. So it is a good add-on
to SysV init. IMHO it is a great adjunct to any existing init and using
both together provides one with an environment that is 100% backwards
compatible, whilst providing service supervision and parallel startup. I
can't see any compelling reasons to replace init with runit except for
special cases like memory constrained systems using a dietlibc compiled
runit.

Many techniques have been discussed recently on the list to run services
once or to wedge system init scripts into a runit environment but they
are workarounds at best or kludges at worst. The best idea is to not use
runit for these tasks and to continue to use init. This will save you
lots of time trying to resolve issues that don't really need to be
resolved (they are taken care of quite well by init). The only downside
AFAIKS is that you don't get any benefit from parallelisation. On a
server system this is not too great an issue but on desktop systems it
is more of an issue.

D-bus and haldaemon are getting a strong following especially in the
Gnome world and these services tend to develop dependencies built upon
them. e.g. Bluetooth, avahi, yum-updatesd, bind and others depend on
d-bus and sometimes also haldaemon through d-bus on Fedora. D-bus dying
breaks all these other services and re-starting d-bus does not fix them
(they don't re-attach to d-bus automatically). They all have to be
stopped and restarted. This is where runit falls down if you have a
default installation or use execline based scripts. A runit environment
needs to be wrapped in some other environment that takes proper care of
dependancies (not just start-up ones but stop/restart dependancies as
well) and other caretaking tasks.

[ btw, d-bus is a very stable app and has not crashed in my experience
so no disrespect to the d-bus developers :) ]

I have come up with some of this wrapper code in the fedorafastboot
project. So when a service starts up, it 'registers' itself with any
dependancies it starts. If those dependancies stop for any reason their
finish script stops the dependant services as well (there is no
mechanism in runit to do this). What has still not been done is a proper
restart. When d-bus stops and the dependant services also stop only
d-bus gets restarted not the dependant services (which ideally should
also be restarted). It's all very messy and overly complicated but
eventually I will have all this covered somehow, hopefully in a
user-transparent way where the user only has to specify the dependancies
in the run script and the wrapper code does all the rest of the work.

Still, I have it working in a useful way right now where if d-bus stops
or crashes it will pull down any services that depend on it as well. A
ctrl-alt-backspace restarts X and as X is also a supervised with a list
of dependancies it restarts all the dependancies and restarts X (plus
all the panel applets which depend on those services). Still a bit kooky
but it does not require a reboot, just an X restart.

Init still runs the iptables scripts, kudzu, wine and a couple of other
(non-daemon) scripts which won't benefit from being managed by runit.

Perhaps what would be most beneficial for runit long term would be to
add runit's service supervision functionality natively into InitNG
and/or Upstart.

R


-----Original Message-----
From: Bernhard Graf [mailto:list-supervision@augensalat.de] 
Sent: 10 January 2008 09:31
To: supervision@list.skarnet.org
Subject: Re: using runit as init

On Thursday 10 January 2008 00:34, Mike Buland wrote:
> Regaurding service dependancies, Richard Gooch's method is actually
> unnecesarry in a sytem such as runit, and service dependancy is
> already handled in an almost identicle way, but with less system
> overhead.
>
> At the begining of a run script just start the services you depend
> on, and exit if the sv command returns non-zero.  I.e. from hal's run
> script: sv start /etc/service/dbus || exit 1

I tend to object:

You mean starting a bunch of programs over and over again, because many
of them quit immediatly is less system overhead than starting programs
that pause until those programs they depend on are started?

Also imagine one service A that depends on service B, C and D. You have
to put the checks for availability of B, C, D into A's run script.
In a complex system more services depend on B, C and D and probably
others. You have to put the same checks (and others) in all those
services. Also those checks tend to be more complicated than just saying
"need B C D".

> As far as running SysV init style scripts, usually handled on modern
> linux systems (redhat, slackware, debian, etc) by the master rc
> script, are you suggesting that rc, to control runlevel changes be
> placed in it's own service, or that each SysV "service" gets it's own
> runit service?

The latter.

> If you're referring to the later case, you'll need to do a lot more
> work to synchronize the rc scripts and the runit services, I can't
> imagine how just making symlinks between the init.d runlevel
> directories and runsvdir service directories will actually work.

In current distributions (hopefully) an init.d script is configured
using a program that sets the appropriate symlinks. The information for
run levels and dependencies are taken from the init scripts [5], [6].

If you'd replace init by runit, you had to replace those installtion
programs as well.

This installer would install the example from [5] as:

# ls -l /etc/runit/sv/lsb-ourdb
-rw-------   1 root root      0 Jan 10 09:00 initd
-rw-------   1 root root     30 Jan 10 09:00 need
lrwxrwxrwx   1 root root     19 Jan 10 09:00 run ->
/etc/init.d/lsb-ourdb
# cat /etc/runit/sv/lsb-ourdb/need
$local_fs
$network
$remote_fs
# ls -l /etc/runit/runsvdir/2/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00
/etc/runit/runsvdir/2/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
# ls -l /etc/runit/runsvdir/3/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00
/etc/runit/runsvdir/3/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
# ls -l /etc/runit/runsvdir/4/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00
/etc/runit/runsvdir/4/lsb-ourdb -> /etc/runit/sv/lsb-ourdb
# ls -l /etc/runit/runsvdir/5/lsb-ourdb
lrwxrwxrwx   1 root root     19 Jan 10 09:00
/etc/runit/runsvdir/5/lsb-ourdb -> /etc/runit/sv/lsb-ourdb

File initd is a flag for runit to run the script with argument "start"
first, then wait for SIGINT and run the script with argument "stop".

File need contains services that runit has to be started first.
"$service" for "meta services" as set with "provide" in Gooch's concept.

> Finally, if you really do want to start an rc script and make it
> appear as though it were a normal runit system service, then I would
> reccomend starting it in a run script, then executing a command that
> blocks forever.  The service will use effectively no CPU time, but
> will stay running, forever. Then you can call the corrisponding rc 6
> or whatever you want in the finish script of the service.

I will try this, and see how far I get...


[5]
http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core
-generic/initscrcomconv.html
[6]
http://refspecs.linux-foundation.org/LSB_3.1.0/LSB-Core-generic/LSB-Core
-generic/initsrcinstrm.html

-- 
Bernhard Graf




^ permalink raw reply	[flat|nested] 37+ messages in thread

* RE: using runit as init
       [not found]                     ` <85040AD9CA634253A8FDB9F7DA6BD200@home.internal>
@ 2008-01-10 22:08                       ` rehan khan
  2008-01-11  1:28                         ` Charlie Brady
  0 siblings, 1 reply; 37+ messages in thread
From: rehan khan @ 2008-01-10 22:08 UTC (permalink / raw)
  To: Mike Buland, supervision


Although the dummy process resolves this particular problem it's not
very clean. I would prefer not to have processes knocking about which do
nothing.

Putting an 'sv d .' in the finish script seems to be the best solution
to run once scripts run by runsvdir. I still think that a mechanism
built into the runsvdir to handle run once scripts would be better.

R


^ permalink raw reply	[flat|nested] 37+ messages in thread

* RE: using runit as init
  2008-01-10 22:08                       ` rehan khan
@ 2008-01-11  1:28                         ` Charlie Brady
  0 siblings, 0 replies; 37+ messages in thread
From: Charlie Brady @ 2008-01-11  1:28 UTC (permalink / raw)
  To: rehan khan; +Cc: Mike Buland, supervision


On Thu, 10 Jan 2008, rehan khan wrote:

>
> Although the dummy process resolves this particular problem it's not
> very clean.

"this particular problem"? You haven't quoted anything, so we don't know 
which particular problem is.

> I would prefer not to have processes knocking about which do
> nothing.

I'm guessing you are referring to my suggestion of a program which calls 
paus().

> Putting an 'sv d .' in the finish script seems to be the best solution
> to run once scripts run by runsvdir.

Yes, that sounds reasonable. That wouldn't be usable with daemontools, 
however. It also means that 'sv t .' would be a no-op, so wouldn't be 
usable with an init script wrapper which translated "service thing 
restart" to 'sv t /service/thing".

> I still think that a mechanism built into the runsvdir to handle run 
> once scripts would be better.

Perhaps.


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-10 20:06                     ` Mike Buland
@ 2008-01-11  7:58                       ` Bernhard Graf
  2008-01-11 14:30                         ` Mike Buland
  0 siblings, 1 reply; 37+ messages in thread
From: Bernhard Graf @ 2008-01-11  7:58 UTC (permalink / raw)
  To: supervision

Mike Buland wrote:

It seems you simply refuse the actual point of the discussion I started:

My goal is not to make runit like SysV init.

My goal is to replace init by something that supervises daemon processes 
in the sense of daemontools.

The discourse is about the way to that goal.

And of course about shortcomings in runit / other init system / the 
daemontools concept as process 1 in general. This should be allowed 
without making someone feel defensive.
-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-11  7:58                       ` Bernhard Graf
@ 2008-01-11 14:30                         ` Mike Buland
  2008-01-12 10:18                           ` Bernhard Graf
  0 siblings, 1 reply; 37+ messages in thread
From: Mike Buland @ 2008-01-11 14:30 UTC (permalink / raw)
  To: supervision

yes, you're completely right.  I'm sorry.  I do overstate and come off as a 
jerk sometimes, most times, maybe...

I feel that your proposal is effectively hacking a good system for supervising 
some/most/all processes and losing all the cool extras that runit offers.  
Really, if what you want is supervising daemon processes, runit does it well, 
but the rc scripts are not compatible with that concept.  They all start 
programs that daemonize themselves, or *shudder* fork bash and use bash to 
daemonize them.

I really like runit, but I believe that there is no good option for allowing 
runit to supervise these daemons without writing runit specific (ok, 
daemontools could run most of them) scripts.  That, or maybe using the 
program group hack from daemontools (I beleive chpst can do the same thing, 
it's burried in the docs though).

The problem is the forking/daemonizing, it means that when those processes do 
that, you're no longer supervising them.  I'm willing to try and help with a 
good way of doing that, but I fear there is none, sometimes these systems 
just aren't compatible with one another.

Fortunately, I don't have one run script that's more than 4 or 5 lines, and 
most of that is whitespace :)

Sorry
--Mike

On Friday 11 January 2008 12:58:32 am Bernhard Graf wrote:
> Mike Buland wrote:
>
> It seems you simply refuse the actual point of the discussion I started:
>
> My goal is not to make runit like SysV init.
>
> My goal is to replace init by something that supervises daemon processes
> in the sense of daemontools.
>
> The discourse is about the way to that goal.
>
> And of course about shortcomings in runit / other init system / the
> daemontools concept as process 1 in general. This should be allowed
> without making someone feel defensive.



^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-11 14:30                         ` Mike Buland
@ 2008-01-12 10:18                           ` Bernhard Graf
  2008-01-12 17:13                             ` supervising (Re: using runit as init) Charlie Brady
                                               ` (3 more replies)
  0 siblings, 4 replies; 37+ messages in thread
From: Bernhard Graf @ 2008-01-12 10:18 UTC (permalink / raw)
  To: supervision

Mike Buland wrote:

> yes, you're completely right.  I'm sorry.  I do overstate and come
> off as a jerk sometimes, most times, maybe...

I think you are overstating ... ;-)

But nevermind. The whole thread gave me some valuable input.

> Fortunately, I don't have one run script that's more than 4 or 5
> lines, and most of that is whitespace :)

Lucky you.
You don't run mysql, do you?
-- 
Bernhard Graf


^ permalink raw reply	[flat|nested] 37+ messages in thread

* supervising (Re: using runit as init)
  2008-01-12 10:18                           ` Bernhard Graf
@ 2008-01-12 17:13                             ` Charlie Brady
  2008-01-12 17:32                               ` supervising mysql (Re: supervising (Re: using runit as init)) Charlie Brady
  2008-01-13  4:40                             ` using runit as init Vincent Danen
                                               ` (2 subsequent siblings)
  3 siblings, 1 reply; 37+ messages in thread
From: Charlie Brady @ 2008-01-12 17:13 UTC (permalink / raw)
  To: Bernhard Graf; +Cc: supervision


On Sat, 12 Jan 2008, Bernhard Graf wrote:

> You don't run mysql, do you?

What's so hard about mysql (other than the parent process not passing on 
signals to children)?

-bash-3.00$ cat run
#! /bin/sh

exec 2>&1

exec /usr/libexec/mysqld \
  --defaults-file=/etc/my.cnf \
   --basedir=/usr \
    --datadir=/var/lib/mysql \
     --user=mysql \
      --pid-file=/var/run/mysqld/mysqld.pid
-bash-3.00$
-bash-3.00$ for i in control/* ; do sudo echo $i: ; sudo cat $i  ; done
control/d:
#! /bin/sh

exec kill -TERM $(cat /var/run/mysqld/mysqld.pid)
control/i:
#! /bin/sh

exec kill -INT $(cat /var/run/mysqld/mysqld.pid)
control/q:
#! /bin/sh

exec kill -QUIT $(cat /var/run/mysqld/mysqld.pid)
control/t:
#! /bin/sh

exec kill -TERM $(cat /var/run/mysqld/mysqld.pid)



^ permalink raw reply	[flat|nested] 37+ messages in thread

* supervising mysql (Re: supervising (Re: using runit as init))
  2008-01-12 17:13                             ` supervising (Re: using runit as init) Charlie Brady
@ 2008-01-12 17:32                               ` Charlie Brady
  0 siblings, 0 replies; 37+ messages in thread
From: Charlie Brady @ 2008-01-12 17:32 UTC (permalink / raw)
  To: supervision; +Cc: Bernhard Graf


[Sorry - missed a word in the Subject.]




^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-12 10:18                           ` Bernhard Graf
  2008-01-12 17:13                             ` supervising (Re: using runit as init) Charlie Brady
@ 2008-01-13  4:40                             ` Vincent Danen
  2008-01-13 15:36                               ` Charlie Brady
       [not found]                             ` <CDFFB8AF013F4762AE02CF6A1BDCB27A@home.internal>
  2008-01-13 18:52                             ` Mike Buland
  3 siblings, 1 reply; 37+ messages in thread
From: Vincent Danen @ 2008-01-13  4:40 UTC (permalink / raw)
  To: Bernhard Graf; +Cc: supervision

[-- Attachment #1: Type: text/plain, Size: 572 bytes --]

* Bernhard Graf <list-supervision@augensalat.de> [2008-01-12 11:18:51 +0100]:

>> Fortunately, I don't have one run script that's more than 4 or 5
>> lines, and most of that is whitespace :)
>
>Lucky you.
>You don't run mysql, do you?

I do, but it could be done in less than 5 lines.  My current runscript
for mysql is 29 lines, but a lot of that is error checking and settting
variables, none of which would be required for a single person who knows
what he's doing.

mysqld is no more complex than anything else.

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* RE: using runit as init
       [not found]                             ` <CDFFB8AF013F4762AE02CF6A1BDCB27A@home.internal>
@ 2008-01-13 10:35                               ` rehan khan
  2008-01-14  3:50                                 ` Vincent Danen
  0 siblings, 1 reply; 37+ messages in thread
From: rehan khan @ 2008-01-13 10:35 UTC (permalink / raw)
  To: Vincent Danen, Bernhard Graf; +Cc: supervision

[-- Attachment #1: Type: text/plain, Size: 2954 bytes --]

Vincent Danen <vdanen <at> annvix.org> writes:

> 
> * Bernhard Graf <list-supervision <at> augensalat.de> [2008-01-12 11:18:51 +0100]:
> 
> >> Fortunately, I don't have one run script that's more than 4 or 5
> >> lines, and most of that is whitespace :)
> >
> >Lucky you.
> >You don't run mysql, do you?
> 
> I do, but it could be done in less than 5 lines.  My current runscript
> for mysql is 29 lines, but a lot of that is error checking and settting
> variables, none of which would be required for a single person who knows
> what he's doing.
> 
> mysqld is no more complex than anything else.
> 

I think this is one of the problems with why runit has had slower takeup and initng and upstart are stealing it's thunder, in the major distro arena - no offence to the fine annvix distro. Much of the documentation available on the net provide these highly abreviated init scripts which might only apply to linux (runit is multi OS). This is not actually what is required in a general purpose OS. The scripts need to be complete in the sense that they can cope with most things thrown at them including incomplete configurations, initial startups etc. In fact when one gets to mad java apps like Tomcat and Hsqldb the scripts become nightmares as they have to cope with so many OS's and java versions and such like.

mysqld is an interesting example. On a freshly installed OS nothing is initialised for mysql. How would one cope with this situation? The fedora scripts ask the user for input and setup initial databases.

Runit does not have any in-built mechanism for this this scenario and it needs to be built into the supporting scripts. Runit also does not have any mechanism to signal the system that user input is required and that all other services should not write to the console while this is happening. Runit also does not have a mechanism to signal that a particular services needs to have exclusive access to the system (useful if one wants to migrate the rc.sysinit script to a bunch of runit services). The scripts built around runit need to take care of these situations. At the end of startup there needs to be some kind of indication that services have failed/succeeded for some reason otherwise they may go un-noticed.

In some ways I think runsvdir could use some additional functionality and I am thinking of re-implementing it in python. Runsvdir does in effect supervise all the runsv processes (when it is used) and would be a good place to co-ordinate this kind of activity (process signalling, holding a dependancy database). I probably need to do this anyway to get Linux Standards Base compliance.

It would be nice to see a collection of multi-OS (linux, bsd etc) init scripts somewhere out there. Perhaps if someone could help me out with the cvs/svn setup on the fedorafastboot project we can get a general script repository setup?

R

http://sourceforge.net/projects/fedorafastboot


[-- Attachment #2: Type: text/html, Size: 3606 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13  4:40                             ` using runit as init Vincent Danen
@ 2008-01-13 15:36                               ` Charlie Brady
  2008-01-13 18:28                                 ` Mike Buland
  2008-01-14  3:53                                 ` Vincent Danen
  0 siblings, 2 replies; 37+ messages in thread
From: Charlie Brady @ 2008-01-13 15:36 UTC (permalink / raw)
  To: Vincent Danen; +Cc: supervision


On Sat, 12 Jan 2008, Vincent Danen wrote:

> mysqld is no more complex than anything else.

Have they fixed it then, so that the master process kills children before 
dying?


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13 15:36                               ` Charlie Brady
@ 2008-01-13 18:28                                 ` Mike Buland
  2008-01-13 18:39                                   ` Charlie Brady
  2008-01-14  3:53                                 ` Vincent Danen
  1 sibling, 1 reply; 37+ messages in thread
From: Mike Buland @ 2008-01-13 18:28 UTC (permalink / raw)
  To: supervision

Yes, read the (very lame) docs on mysqlmanager.  It also now responds to other 
signals (such as term) so it plays nicely with runit :)

mysqlmanager was introduced sometime in mysql 5, not sure when.

--Mike

P.S.  My mysql script doesn't do all of that, I leave it to mysqlmanager, and 
I don't worry about the database being there, the installer takes care of all 
that on my system.  I could probably do more with error checking, but I'm not 
sure what I would cover.  My script is two lines including the bash ID at the 
top.

On Sunday 13 January 2008 08:36:31 am Charlie Brady wrote:
> On Sat, 12 Jan 2008, Vincent Danen wrote:
> > mysqld is no more complex than anything else.
>
> Have they fixed it then, so that the master process kills children before
> dying?



^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13 18:28                                 ` Mike Buland
@ 2008-01-13 18:39                                   ` Charlie Brady
  2008-01-13 18:49                                     ` Mike Buland
       [not found]                                     ` <6A64B0D384404190ACB76E0A376CD148@home.internal>
  0 siblings, 2 replies; 37+ messages in thread
From: Charlie Brady @ 2008-01-13 18:39 UTC (permalink / raw)
  To: Mike Buland; +Cc: supervision




On Sun, 13 Jan 2008, Mike Buland wrote:

> On Sunday 13 January 2008 08:36:31 am Charlie Brady wrote:
>> On Sat, 12 Jan 2008, Vincent Danen wrote:
>>> mysqld is no more complex than anything else.
>>
>> Have they fixed it then, so that the master process kills children before
>> dying?
> Yes, read the (very lame) docs on mysqlmanager.  It also now responds to other
> signals (such as term) so it plays nicely with runit :)

This might actually mean "no". They haven't fixed the signal 
handling in mysqld, but have instead introduced a new complicated program 
to try to work around the issues.

> mysqlmanager was introduced sometime in mysql 5, not sure when.

Introduced in 5.0.3. Not used by default since 5.0.4.


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13 18:39                                   ` Charlie Brady
@ 2008-01-13 18:49                                     ` Mike Buland
  2008-01-14  3:55                                       ` Vincent Danen
       [not found]                                     ` <6A64B0D384404190ACB76E0A376CD148@home.internal>
  1 sibling, 1 reply; 37+ messages in thread
From: Mike Buland @ 2008-01-13 18:49 UTC (permalink / raw)
  To: supervision


Hey, a complicated program they made is better than one I have to manage.

If you do read the docs, they don't handle signals in mysqld, like I say, 
mysqlmanager does.  They don't defend it, but they do explain that the only 
way to safely shutdown mysql is to send it a command via the named socket.  
Since this can't be done easily from the commandline without a password, 
unless you configrue mysql to accept the command from root without a 
password, then this seems like a pretty decent solution, since this program 
basically supervises it's child mysql processes and sends them a safe 
shutdown signal when the mysqlmanager program receives a term signal.

All the scripts I've read from debian and redhat just send a kill...although 
this seems alright most of the time (and they are official scripts), that 
doesn't gurantee that data is flushed to disk, if mysql is in the middle of a 
large operation, that data is gone.  This solution does solve that problem, 
and keep runit from waiting for mysql to TERM.

--Mike

On Sunday 13 January 2008 11:39:56 am Charlie Brady wrote:
> On Sun, 13 Jan 2008, Mike Buland wrote:
> > On Sunday 13 January 2008 08:36:31 am Charlie Brady wrote:
> >> On Sat, 12 Jan 2008, Vincent Danen wrote:
> >>> mysqld is no more complex than anything else.
> >>
> >> Have they fixed it then, so that the master process kills children
> >> before dying?
> >
> > Yes, read the (very lame) docs on mysqlmanager.  It also now responds to
> > other signals (such as term) so it plays nicely with runit :)
>
> This might actually mean "no". They haven't fixed the signal
> handling in mysqld, but have instead introduced a new complicated program
> to try to work around the issues.
>
> > mysqlmanager was introduced sometime in mysql 5, not sure when.
>
> Introduced in 5.0.3. Not used by default since 5.0.4.



^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-12 10:18                           ` Bernhard Graf
                                               ` (2 preceding siblings ...)
       [not found]                             ` <CDFFB8AF013F4762AE02CF6A1BDCB27A@home.internal>
@ 2008-01-13 18:52                             ` Mike Buland
  3 siblings, 0 replies; 37+ messages in thread
From: Mike Buland @ 2008-01-13 18:52 UTC (permalink / raw)
  To: supervision


If you don't mind, since I am actually interested, would you explain to me why 
you think I'm wrong.  Really about your solution for porting scripts not 
leveraging any of the advantages of runit.  If we can actually make runit 
better, I'm for it.  I'm just not yet convinced that any suggestions so far 
would help (insofar as increasing flexibility, adding features, and not 
overcomplicating).

--Mike

On Saturday 12 January 2008 03:18:51 am Bernhard Graf wrote:
> Mike Buland wrote:
> > yes, you're completely right.  I'm sorry.  I do overstate and come
> > off as a jerk sometimes, most times, maybe...
>
> I think you are overstating ... ;-)
>
> But nevermind. The whole thread gave me some valuable input.
>
> > Fortunately, I don't have one run script that's more than 4 or 5
> > lines, and most of that is whitespace :)
>
> Lucky you.
> You don't run mysql, do you?



^ permalink raw reply	[flat|nested] 37+ messages in thread

* RE: using runit as init
       [not found]                                     ` <6A64B0D384404190ACB76E0A376CD148@home.internal>
@ 2008-01-13 21:06                                       ` rehan khan
  0 siblings, 0 replies; 37+ messages in thread
From: rehan khan @ 2008-01-13 21:06 UTC (permalink / raw)
  To: Mike Buland, supervision

[-- Attachment #1: Type: text/plain, Size: 2343 bytes --]

AFAICS the redhat scripts send a term, wait 10 secs then send a kill to the mysql processes ending with the parent process.

From what you guys are saying this is not the right way to stop mysql? If it isn't then someone should probably mention it to Redhat :)


Rehan


-----Original Message-----
From: Mike Buland [mailto:mike@geekgene.com]
Sent: 13 January 2008 19:01
To: supervision@list.skarnet.org
Subject: Re: using runit as init


Hey, a complicated program they made is better than one I have to manage.

If you do read the docs, they don't handle signals in mysqld, like I say, 
mysqlmanager does.  They don't defend it, but they do explain that the only 
way to safely shutdown mysql is to send it a command via the named socket.  
Since this can't be done easily from the commandline without a password, 
unless you configrue mysql to accept the command from root without a 
password, then this seems like a pretty decent solution, since this program 
basically supervises it's child mysql processes and sends them a safe 
shutdown signal when the mysqlmanager program receives a term signal.

All the scripts I've read from debian and redhat just send a kill...although 
this seems alright most of the time (and they are official scripts), that 
doesn't gurantee that data is flushed to disk, if mysql is in the middle of a 
large operation, that data is gone.  This solution does solve that problem, 
and keep runit from waiting for mysql to TERM.

--Mike

On Sunday 13 January 2008 11:39:56 am Charlie Brady wrote:
> On Sun, 13 Jan 2008, Mike Buland wrote:
> > On Sunday 13 January 2008 08:36:31 am Charlie Brady wrote:
> >> On Sat, 12 Jan 2008, Vincent Danen wrote:
> >>> mysqld is no more complex than anything else.
> >>
> >> Have they fixed it then, so that the master process kills children
> >> before dying?
> >
> > Yes, read the (very lame) docs on mysqlmanager.  It also now responds to
> > other signals (such as term) so it plays nicely with runit :)
>
> This might actually mean "no". They haven't fixed the signal
> handling in mysqld, but have instead introduced a new complicated program
> to try to work around the issues.
>
> > mysqlmanager was introduced sometime in mysql 5, not sure when.
>
> Introduced in 5.0.3. Not used by default since 5.0.4.





[-- Attachment #2: Type: text/html, Size: 3097 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13 10:35                               ` rehan khan
@ 2008-01-14  3:50                                 ` Vincent Danen
  0 siblings, 0 replies; 37+ messages in thread
From: Vincent Danen @ 2008-01-14  3:50 UTC (permalink / raw)
  To: rehan khan; +Cc: Bernhard Graf, supervision

[-- Attachment #1: Type: text/plain, Size: 4155 bytes --]

* rehan khan <rehan.khan@dsl.pipex.com> [2008-01-13 10:35:40 -0000]:

>> >> Fortunately, I don't have one run script that's more than 4 or 5
>> >> lines, and most of that is whitespace :)
>> >
>> >Lucky you.
>> >You don't run mysql, do you?
>> 
>> I do, but it could be done in less than 5 lines.  My current runscript
>> for mysql is 29 lines, but a lot of that is error checking and settting
>> variables, none of which would be required for a single person who knows
>> what he's doing.
>> 
>> mysqld is no more complex than anything else.
>> 
>
>I think this is one of the problems with why runit has had slower takeup and initng and upstart are stealing it's thunder, in the major distro arena - no offence to the fine annvix distro. Much of the documentation available on the net provide these highly abreviated init scripts which might only apply to linux (runit is multi OS). This is not actually what is required in a general purpose OS. The scripts need to be complete in the sense that they can cope with most things thrown at them including incomplete configurations, initial startups etc. In fact when one gets to mad java apps like Tomcat and Hsqldb the scripts become nightmares as they have to cope with so many OS's and java versions and such like.
>
>mysqld is an interesting example. On a freshly installed OS nothing is initialised for mysql. How would one cope with this situation? The fedora scripts ask the user for input and setup initial databases.

Depends on the OS.  For annvix, the default database is setup when the
rpm is installed.  I believe most distros do something similar...
granted, I don't use much beyond annvix and mandriva, but I can't
imagine anyone thinking that an initscript is a good place to be asking
for user configuration.  Initscripts should be non-interactive... it
certainly isn't the right place to be asking users for configuration info.

>Runit does not have any in-built mechanism for this this scenario and it needs to be built into the supporting scripts. Runit also does not have any mechanism to signal the system that user input is required and that all other services should not write to the console while this is happening. Runit also does not have a mechanism to signal that a particular services needs to have exclusive access to the system (useful if one wants to migrate the rc.sysinit script to a bunch of runit services). The scripts built around runit need to take care of these situations. At the end of startup there needs to be some kind of indication that services have failed/succeeded for some reason otherwise they may go un-noticed.

This could be useful, granted, but doesn't necessarily need to be done
by runit.  A simple script called after everything gets started could
check to see what is down and should be up, then report on those.  with
annvix, we use a frontend to sv called srv, and srv --list will tell you
what is up, down, should be up, when it changed state, etc.

But I don't think runit needs to change to allow people to execute
configuration during service startup becuase I think these services
should be configured before they're told to start.

Distros that use initscripts to setup services themselves are, imho,
bastardizing what initscripts were designed for in a very wrong way.

>In some ways I think runsvdir could use some additional functionality and I am thinking of re-implementing it in python. Runsvdir does in effect supervise all the runsv processes (when it is used) and would be a good place to co-ordinate this kind of activity (process signalling, holding a dependancy database). I probably need to do this anyway to get Linux Standards Base compliance.
>
>It would be nice to see a collection of multi-OS (linux, bsd etc) init scripts somewhere out there. Perhaps if someone could help me out with the cvs/svn setup on the fedorafastboot project we can get a general script repository setup?

The annvix svn is open and we have runscripts for a number of services
that can easily be cherrypicked by anyone interested.
http://svn.annvix.org/.

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13 15:36                               ` Charlie Brady
  2008-01-13 18:28                                 ` Mike Buland
@ 2008-01-14  3:53                                 ` Vincent Danen
  1 sibling, 0 replies; 37+ messages in thread
From: Vincent Danen @ 2008-01-14  3:53 UTC (permalink / raw)
  To: Charlie Brady; +Cc: supervision

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

* Charlie Brady <charlieb-supervision@budge.apana.org.au> [2008-01-13 10:36:31 -0500]:

>> mysqld is no more complex than anything else.
>
> Have they fixed it then, so that the master process kills children before 
> dying?

No, but my mysqld.finish script calls "mysqladmin shutdown" and has for
a few years now with no adverse effects at all.

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-13 18:49                                     ` Mike Buland
@ 2008-01-14  3:55                                       ` Vincent Danen
  2008-01-14 15:11                                         ` Charlie Brady
  0 siblings, 1 reply; 37+ messages in thread
From: Vincent Danen @ 2008-01-14  3:55 UTC (permalink / raw)
  To: Mike Buland; +Cc: supervision

[-- Attachment #1: Type: text/plain, Size: 1334 bytes --]

* Mike Buland <mike@geekgene.com> [2008-01-13 11:49:58 -0700]:

>Hey, a complicated program they made is better than one I have to manage.
>
>If you do read the docs, they don't handle signals in mysqld, like I say, 
>mysqlmanager does.  They don't defend it, but they do explain that the only 
>way to safely shutdown mysql is to send it a command via the named socket.  
>Since this can't be done easily from the commandline without a password, 
>unless you configrue mysql to accept the command from root without a 
>password, then this seems like a pretty decent solution, since this program 
>basically supervises it's child mysql processes and sends them a safe 
>shutdown signal when the mysqlmanager program receives a term signal.
>
>All the scripts I've read from debian and redhat just send a kill...although 
>this seems alright most of the time (and they are official scripts), that 
>doesn't gurantee that data is flushed to disk, if mysql is in the middle of a 
>large operation, that data is gone.  This solution does solve that problem, 
>and keep runit from waiting for mysql to TERM.

Hmmm... I didn't know anything about this mysqlmanager.  Care to share
your runit script?  =)  I may have to look into using that instead of
calling mysqld directly.

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-14  3:55                                       ` Vincent Danen
@ 2008-01-14 15:11                                         ` Charlie Brady
  2008-01-14 15:21                                           ` Vincent Danen
  0 siblings, 1 reply; 37+ messages in thread
From: Charlie Brady @ 2008-01-14 15:11 UTC (permalink / raw)
  To: Vincent Danen; +Cc: Mike Buland, supervision


On Sun, 13 Jan 2008, Vincent Danen wrote:

> Hmmm... I didn't know anything about this mysqlmanager.  Care to share
> your runit script?  =)  I may have to look into using that instead of
> calling mysqld directly.

Why? You haven't had any trouble reports about mysql startup, have you?


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: using runit as init
  2008-01-14 15:11                                         ` Charlie Brady
@ 2008-01-14 15:21                                           ` Vincent Danen
  0 siblings, 0 replies; 37+ messages in thread
From: Vincent Danen @ 2008-01-14 15:21 UTC (permalink / raw)
  To: Charlie Brady; +Cc: Mike Buland, supervision

[-- Attachment #1: Type: text/plain, Size: 1232 bytes --]

* Charlie Brady <charlieb-supervision@budge.apana.org.au> [2008-01-14 10:11:41 -0500]:

> On Sun, 13 Jan 2008, Vincent Danen wrote:
>
>> Hmmm... I didn't know anything about this mysqlmanager.  Care to share
>> your runit script?  =)  I may have to look into using that instead of
>> calling mysqld directly.
>
> Why? You haven't had any trouble reports about mysql startup, have you?

No, actually mysql runs quite reliably the way I call it, and with using
the mysqladmin shutdown command to turn it off.  But if there's a better
(more proper) way to do it, then I'm interested in looking at it.

mysqlmanager does look interesting.. looks like you can manager more
than one database instance starting from the same supervisor instance,
so I could be running a local-only db, a remote db, and another remote
db, just by fiddling with /etc/my.cnf and restarting the manager.

Just because it's worked for me doesn't mean it's not possible for it to
bork something... maybe I've been lucky, maybe mysql is robust enough to
handle it... maybe what I'm doing is perfectly acceptable, but there's
nothing wrong with looking at the "recommended" way of doing things.  =)

-- 
Vincent Danen @ http://linsec.ca/

[-- Attachment #2: Type: application/pgp-signature, Size: 194 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

end of thread, other threads:[~2008-01-14 15:21 UTC | newest]

Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-01-03 20:51 using runit as init Bernhard Graf
2008-01-04  0:22 ` Mike Buland
2008-01-05  0:06   ` Bernhard Graf
     [not found]   ` <12EC84FDD73F4BD8A78E7501EB19F1E2@home.internal>
2008-01-05  7:45     ` rehan khan
2008-01-05 23:17       ` Bernhard Graf
2008-01-08  7:11         ` Vincent Danen
2008-01-08 22:28           ` Bernhard Graf
2008-01-09  2:05             ` Vincent Danen
2008-01-09 23:06               ` Bernhard Graf
2008-01-09 23:34                 ` Mike Buland
2008-01-09 23:51                   ` Charlie Brady
2008-01-10  9:22                     ` Bernhard Graf
2008-01-10  9:20                   ` Bernhard Graf
2008-01-10 20:06                     ` Mike Buland
2008-01-11  7:58                       ` Bernhard Graf
2008-01-11 14:30                         ` Mike Buland
2008-01-12 10:18                           ` Bernhard Graf
2008-01-12 17:13                             ` supervising (Re: using runit as init) Charlie Brady
2008-01-12 17:32                               ` supervising mysql (Re: supervising (Re: using runit as init)) Charlie Brady
2008-01-13  4:40                             ` using runit as init Vincent Danen
2008-01-13 15:36                               ` Charlie Brady
2008-01-13 18:28                                 ` Mike Buland
2008-01-13 18:39                                   ` Charlie Brady
2008-01-13 18:49                                     ` Mike Buland
2008-01-14  3:55                                       ` Vincent Danen
2008-01-14 15:11                                         ` Charlie Brady
2008-01-14 15:21                                           ` Vincent Danen
     [not found]                                     ` <6A64B0D384404190ACB76E0A376CD148@home.internal>
2008-01-13 21:06                                       ` rehan khan
2008-01-14  3:53                                 ` Vincent Danen
     [not found]                             ` <CDFFB8AF013F4762AE02CF6A1BDCB27A@home.internal>
2008-01-13 10:35                               ` rehan khan
2008-01-14  3:50                                 ` Vincent Danen
2008-01-13 18:52                             ` Mike Buland
     [not found]                     ` <85040AD9CA634253A8FDB9F7DA6BD200@home.internal>
2008-01-10 22:08                       ` rehan khan
2008-01-11  1:28                         ` Charlie Brady
     [not found]                   ` <C1A5323F485E4A75B75ADB58B664E35E@home.internal>
2008-01-10 21:56                     ` rehan khan
2008-01-09 23:35                 ` KORN Andras
2008-01-10  8:39                   ` Bernhard Graf

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).