reboot->wtmp

reboot->wtmp

[Alex Efros]

Hi!

I'm now try to install and configure runit-init on Gentoo Linux.
Looks like nearly everything is working fine, but there still a couple
of problems:

- I've not found a way to add 'reboot' or 'runlevel' records to wtmp file.
  I don't really need 'runlevel' records, but it will be useful to see
  'reboot' records when running `last -20`. I think I can write perl script
  which will add binary record to wtmp file (this script will be executed at
  end of /etc/runit/3), but I suppose there already exists ease way to do it?

- I very like djb's phylosofy and soft, and also like runit's features...
  but I also like to have selected lines from *ALL* logs stored in
  separate file (like /var/log/everything/current) and also shown at tty10
  for monitoring purposes... At this time I see only two ways to do it and
  they both too complex for this simple task.
  First way:
    1) all services use it's own svlogd, as usually
    2) socklog send syslog/klog messages to it's svlogd too
    3) all svlogd for all these services including socklog configured to:
	exec 2>/var/log/socklog-everything/.notify
       and then 'e*' in their config files
    4) there a additional svlogd which read /var/log/socklog-everything/.notify
       and save these 'catch-all' logs to /var/log/everything/
  Second way:
    1) all services use it's own svlogd, as usually
    2) these svlogd configured to send everything to syslog using UDP/127.0.0.1
    3) socklog now act as 'catch-all-logs' service and use it's own
       svlogd to filter message to different log dirs
  At first I'm not sure is it possible (and safe) to just run:
    svlogd options ... </var/log/socklog-everything/.notify
  or I should use etcat utility? How about 'etcat isn't crash proof'?
  At second I don't like to use UDP because it's not crash proof too.

  Also... it's not really clear for me why each service should have it's
  own log daemon - is there is no SAFE way to send messages from all
  services to one svlogd daemon (prefixing them with service name) which
  then will store all messages to different directories? This is very like
  how syslog work, but syslog is unreliable, slow and require support in
  software... so is it possible to have syslog-like daemon, which will be
  reliable, fast and able to process usuall "STDOUT" output from services
  (probably prefix'ed with software name/pid using simple utility in pipe)?


P.S. I've compared daemontools vs runit (just line-by-line compare
documentation for similar utilites). If anybody want it, I can post this
comparation here (I think it will be nice to have webpage with
this information at runit's website - just to make more clear for people
what they need: daemontools or runit). But this will take some time because
I've wrote it in native language (russian) and I should translate it to
english before post.

-- 
			WBR, Alex.

/var/log/everything/

[Alex Efros]

Hi!

On Wed, Feb 18, 2004 at 08:22:19AM +0200, Alex Efros wrote:
>   but I also like to have selected lines from *ALL* logs stored in
>   separate file (like /var/log/everything/current) and also shown at tty10

I've just found one more way which is simple enough:
    - service's svlogd executed in ./log/run with 2>&1 (so it's stderr
      will go to runsvdir's stdout instead of runsvdir's title)
    - all svlogd configured to 'e*'
    - runsvdir's stdout piped (in /etc/runit/2) to another svlogd
      which will store logs to /var/log/everything/

I think this should work safe, fast and flexible. Am I right?


P.S. I've found runsvdir's stdout by default sent to first virtual console -
why not to current (/dev/console)?

-- 
			WBR, Alex.

Re: reboot->wtmp

[Clemens Fischer]

* Alex Efros:

> I very like djb's phylosofy and soft, and also like runit's
> features...  but I also like to have selected lines from *ALL* logs
> stored in separate file (like /var/log/everything/current) and also
> shown at tty10 for monitoring purposes...

there are a couple of other logging tools you could combine.  the one
that comes to my mind is the syslogtools package by tim goodwin.

> P.S. I've compared daemontools vs runit (just line-by-line compare
> documentation for similar utilites). If anybody want it, I can post
> this comparation here (I think it will be nice to have webpage with
> this information at runit's website - just to make more clear for
> people what they need: daemontools or runit). But this will take
> some time because I've wrote it in native language (russian) and I
> should translate it to english before post.

this might be valuable to a number of people.  can you post the
translation some place?

  clemens

runit VS daemontools

[Alex Efros]

Hi!

On Wed, Feb 18, 2004 at 06:30:09PM +0100, Clemens Fischer wrote:
> this might be valuable to a number of people.  can you post the
> translation some place?

Comparation of runit's documentation VS daemontools's documentation. ;-)
(I've also added some noted from myself, not from documentation.)

This text may contain mistypo, bugs and even bad ideas. :)

Legend:
 * just note
 - runit don't have this daemontools feature
 + runit add this feature

* runit can be used as replacement for sysvinit and daemontools
  but also can be used together with them
+ man and --help documentation
# maybe daemontools also has similar stdout/stderr behaviour, but I never
# seen it in daemontools before
* runsvdir's stdout by default going to first (?!) virtual console
* stdout of ./{run,finish} (if there is no ./log/run) and stdout of
  ./log/run is going to runsvdir's stdout
* stderr of ./run and ./finish and ./log/run going to runsvdir's title
# runsvdir   VS   svscanboot + svscan + readproctitle
+ runsvdir has functionality of svscanboot + svscan + readproctitle
  which is very useful and safe against another one bash process in
  memory (for pipe between svscan and readproctitle)
* runsvdir execute runsv (svscan execute supervise)
+ service will be stopped automatically if it disappear from /var/service/
  directory (so there is no need for additional svc -dx command after
  removing service)
# runsv   VS   supervise
+ support ./finish script which (if exists) will be executed after ./run
  exits (./finish stdout also send to ./log/run if it exists)
+ current service status available in text format in files
  ./supervise/{stat,pid} (supervise-compatible binary status in
  ./supervise/status is also supported)
+ when terminating service if it has ./log/run then runsv will first
  terminate ./run then close ./log/run's stdin then wait until ./log/run
  exits - this make logs more reliable
+ both ./run and ./log/run served by single runsv process
# runsvctrl   VS   svc
+ able to send SIGUSR1 and SIGUSR2 to service
- don't able to send more than one command at once like `svc - dx`
* commands are readable plain text ("down" instead of "-d")
# runsvstat   VS   svok + svstat
* has functionality of svok (exit code) and svstat (print server status)
* can optionally show ./log/ statuses in addition to main service status
# chpst   VS   pgrphack + setuidgid + envuidgid + envdir + setlock + softlimit
* has functionality of pgrphack + setuidgid +envuidgid + envdir + setlock +
  softlimit
+ allows set group different from user's main group
+ able to close STDIN/STDOUT/STDERR
- no way to define what to do (exit(0) or print error and exit(1)) in case if
  it fail to open/close/lock lock-file
- no way to set soft limit to hard limit like softlimit do it (bug in docs?)
- no way to set separate memory limits (data/stack/locked/total) - they all
  set to same value
- no way to set soft limits for RSS and CPU
# svlogd   VS   multilog
* executing external program as "processor" explained in documenation in
  much more details, but it's unclear is it differ from multilog
+ log configurations moved from multilog's paramemters to /logdir/config
  (file may not exists), it also support comments
+ may send data to syslog using UDP
+ a little extended patterns for selecting strings to log
+ able to add timestamp with microseconds(micro?) in sortable text format
  (it's unclean from docs, but probably in GMT)
+ able to replace unprintable plus user-defined chars by one user-defined
+ it's possible to control maximum string length and size of read buffer
+ SIGHUP: reopen all logs and reread /logdir/config files
+ SIGALRM: force (non-empty) log rotation now
# utils only in daemontools:
- fghack
- tai64n
- tai64nlocal
# added service dependencies
+ svwaitup ( http://smarden.org/pape/djb/daemontools/noinit.html )
    * should be executed from ./run before service daemon itself
    + able to wait until required services will be "up" for at least
      (configurable) timeout seconds
    - as author noted, this way isn't safe because race still possible;
      IMHO Right Way to realize service dependencies is use some sort of
      IPC between ./run and runsvdir so all ./run scripts will be able to
      register is't provides or dependences in one daemon which guarantied
      to be running (after runsvdir die system will be rebooted), and runsvdir
      will be able to control service executing order and check is they alive;
      btw, this race may be (?) worked around by stopping all services which
      require this one from it's ./finish script.
# added safe shutdown for services at reboot/halt
+ svwaitdown
    + able to send down, kill, exit commands to services and wait
      until they exits or timeout happens
# added runlevels support
+ runsvchdir
# added sysvinit replacement
+ runit-init: replaced /sbin/init; execute runit at system boot time
+ runit
    + simple /etc/inittab replacement by using 4 hardcoded script names
      for initializing (/etc/runit/1), execute services including getties
      (/etc/runit/2), prepare for reboot/halt (/etc/runit/3) and process
      ctrl+alt+del (/etc/runit/ctrlaltdel)
    + restart /etc/runit/2 if it died (hmm.. looks like on my server it
      going to reboot instead... is it my mistake or bug in runit?)
    + SIGINT: emulate ctrl+alt+del
    + SIGCONT: doing reboot/halt if executable file /etc/runit/stopit exists
      (hmm, looks like it impossible to reboot while executing /etc/runit/1 -
      for example if it executed /sbin/sulogin I'm unable to reboot/halt from
      it...)
+ utmpset
    * complement runit-init functionality (sysvinit add records to utmp/wtmp,
      but runit-init doesn't) by adding 'logout' information to utmp/wtmp
      (should be executed in ./finish script for getty-* services)
# new/changed files and directories:
/etc/runit/1
/etc/runit/2
/etc/runit/3
/etc/runit/reboot
    define what should be done on shutdown: halt or reboot
/etc/runit/ctrlaltdel
    script for processing ctrl+alt+del (able to block rebooting)
/etc/runit/stopit
    allow or deny system reboot (for use in ctrlaltdel script, for example)
/var/service/
    now - symlink, was directory /service/
/etc/runit/runsvdir/current/
    symlink to current runlevel (really, to directory with services, "default"
    for example)
/etc/runit/runsvdir/default/
    directory with services for runlevel "default"
./finish
    executed after ./run exits
./supervise/{stat,pid}
    contain service status in text format
/logdir/config
    contain svlogd paramemtes for this /logdir/

-- 
			WBR, Alex.

Re: reboot->wtmp

[Gerrit Pape]

On Wed, Feb 18, 2004 at 08:22:19AM +0200, Alex Efros wrote:
> I'm now try to install and configure runit-init on Gentoo Linux.
> Looks like nearly everything is working fine, but there still a couple
> of problems:
> 
> - I've not found a way to add 'reboot' or 'runlevel' records to wtmp file.
>   I don't really need 'runlevel' records, but it will be useful to see
>   'reboot' records when running `last -20`. I think I can write perl script
>   which will add binary record to wtmp file (this script will be executed at
>   end of /etc/runit/3), but I suppose there already exists ease way to do it?

utmp/wtmp handling is hardly portable.  There's utmp and utmpx, and the
utmp structures differ.  runit only includes the utmpset program for
utmp/wtmp handling, so you would need your own, probably platform
specific, program.

Instead of implementing more utmp support, I personally would like to use
a different, more modern, implementation of login accounting.
 
> - I very like djb's phylosofy and soft, and also like runit's features...
>   but I also like to have selected lines from *ALL* logs stored in
>   separate file (like /var/log/everything/current) and also shown at tty10
>   for monitoring purposes... At this time I see only two ways to do it and
>   they both too complex for this simple task.

I'm not sure why you really want this, but anyway, why not simply use
syslog?  Run a socklog-unix service, and use a program like 'logger' or
'splogger' for your other services to log through /dev/log, if you want.

>   Also... it's not really clear for me why each service should have it's
>   own log daemon

I think it's a good thing to have the logs separated, and so completely
independent from each other.  You can process them under different user
ids, have them written to different partitions, calculate and define
specific storage size limits; if one service acts up and rapidly fills
its log, it won't affect other services, ...

You can merge log files from different services, and, if you prepend
timestamps, sort them to view logs from multiple services.

Regards, Gerrit.

Re: runit VS daemontools

[Gerrit Pape]

On Wed, Feb 18, 2004 at 10:48:02PM +0200, Alex Efros wrote:
> On Wed, Feb 18, 2004 at 06:30:09PM +0100, Clemens Fischer wrote:
> > this might be valuable to a number of people.  can you post the
> > translation some place?
> 
> Comparation of runit's documentation VS daemontools's documentation. ;-)
> (I've also added some noted from myself, not from documentation.)

This is very nice, and nearly complete.  Thanks a lot.

> # runsvctrl   VS   svc
> + able to send SIGUSR1 and SIGUSR2 to service
> - don't able to send more than one command at once like `svc - dx`

You can simply use
 # echo -n 'dx' >/var/service/<service>/supervise/control

> # chpst   VS   pgrphack + setuidgid + envuidgid + envdir + setlock + softlimit
> * has functionality of pgrphack + setuidgid +envuidgid + envdir + setlock +
>   softlimit
> + allows set group different from user's main group
> + able to close STDIN/STDOUT/STDERR
> - no way to define what to do (exit(0) or print error and exit(1)) in case if
>   it fail to open/close/lock lock-file
> - no way to set soft limit to hard limit like softlimit do it (bug in docs?)
> - no way to set separate memory limits (data/stack/locked/total) - they all
>   set to same value
> - no way to set soft limits for RSS and CPU

You can run the chpst program as 'setlock' or 'softlimit' (or
'dsetlock', "dsoftlimit'), then it accepts the same options as the
corresponding daemontools programs.

Regards, Gerrit.

Re: runit VS daemontools

[Alex Efros]

Hi!

On Thu, Feb 19, 2004 at 04:21:32PM +0000, Gerrit Pape wrote:
> This is very nice, and nearly complete.  Thanks a lot.

It will be very good if you add to your website something like it. :-)

> You can simply use
>  # echo -n 'dx' >/var/service/<service>/supervise/control

Yeah, I know. :-) As far as I understand I even can use 'svc'..?

> You can run the chpst program as 'setlock' or 'softlimit' (or
> 'dsetlock', "dsoftlimit'), then it accepts the same options as the
> corresponding daemontools programs.

Cool!

Can you please explain one more thing? I don't understand goal of these lines
in your example for /etc/runit/1 file for debian:

    touch /etc/runit/stopit
    chmod 0 /etc/runit/stopit

Looks like everything working without them too... :)

-- 
			WBR, Alex.

socklog-kern

[Alex Efros]

Hi!

Looks like socklog-kern should be run as root instead of nobody.
If ./run file is:

    exec </proc/kmsg
    exec 2>&1
    exec chpst -unobody socklog ucspi

then log file contain only these lines:

2004-02-20_18:59:56.08857 socklog: fatal: read(): permission denied

P.S. Kernel 2.6.1.

-- 
			WBR, Alex.

Re: socklog-kern

[Laurent Bercot]

> Looks like socklog-kern should be run as root instead of nobody.

 Under a Linux-2.6 kernel, yes, it must.
 Under a Linux-2.4 kernel, it should not.
 The semantics of /proc/kmsg has been changed between 2.4 and 2.6.

-- 
 Ska

Re: runit VS daemontools

[Gerrit Pape]

On Fri, Feb 20, 2004 at 05:24:30AM +0200, Alex Efros wrote:
> Can you please explain one more thing? I don't understand goal of
> these lines in your example for /etc/runit/1 file for debian:
> 
>     touch /etc/runit/stopit
>     chmod 0 /etc/runit/stopit

If runit receives a CONT signal, and the file /etc/runit/stopit exists
and has the execute by owner permission set, runit is told to shutdown
the system.

As an one time initialization task stage 1 ensures that the stopit file
exists and has the execute by owner permission unset.  runit supports
magic files being symbolic links.

Regards, Gerrit.

Re: runit VS daemontools

[Gerrit Pape]

On Wed, Feb 18, 2004 at 10:48:02PM +0200, Alex Efros wrote:
> Comparation of runit's documentation VS daemontools's documentation. ;-)
> # svlogd   VS   multilog

> + able to add timestamp with microseconds(micro?) in sortable text format
>   (it's unclean from docs, but probably in GMT)

I always intended to have svlogd print human readable timestamps in UTC,
but did it wrong, and didn't notice because most of my systems' clocks
are set to UTC.  It's changed now in release 1.0.1.

Regards, Gerrit.