supervision - discussion about system services, daemon supervision, init, runlevel management, and tools such as s6 and runit
 help / color / mirror / Atom feed
From: Jeff <sysinit@yandex.com>
To: supervision <supervision@list.skarnet.org>
Subject: supervising the supervisor
Date: Fri, 03 May 2019 06:13:00 +0200	[thread overview]
Message-ID: <6084291556856780@sas2-a1efad875d04.qloud-c.yandex.net> (raw)

from the last replies we have the following possibilities regarding
process #1's supervision capabilities:

- no supervision/respawning (maybe also not handling system shutdown
  at all, too):
  simplifies the process #1 implementation (especially in the latter case),
  supervision can be delegated to a subprocess which also simplifies that
  supervisor's implenation since there is no need for it to handle process #1
  specific duties
  (given there are more than just reaping zombies as the default subreaper
  and successfully starting at least one necessary child process (to which the
  remaining duties are delegated)).
  disadvantage: "incorrect" behaviour when all other processes die, leads to
  a bricked system, deep shit ahead.

- respawning (at least one) given services/daemons, possibly even with log
  output redirection to logger processes (s6-svscan et al)

- a compromise between the above 2 solutions:
  process #1 supervises (i. e. respawns, possibly only under certain conditions)
  at most 2 subprocesses (a real "supervisor") and maybe redirects
  its output via pipe(2) to a separate supervised dedicated logger subprocess.
  
  in that case those child processes should only be respawned under certain
  conditions (respawn throttling maybe, i. e. stop respawning if one of the 2
  repeatedly fails in a certain amount of time). if those conditions are not met
  it should start a single user rescue shell (possible via sulogin) and/or reboot.

  only in case the logger child process repeatedly fails: do not redirect the
  supervisor's output, use our own (possibly opened by the kernel) output fds
  for the supervisor child process (probably the console device) instead
  of the pipe fds.

  it could also be a good idea to close all of process #1 stdio fds and only
  open the console device for output when the need arises.
  this has the advantage that we do not have this device open all the time
  (in case /dev needs to get re/unmounted).

again (as we are at it ;-):

in the last case:
when said "supervisor" is s6-svscan (or perpd for that) it would be helpful
for the process #1 implementor (me) if it could manage its own output logger
via a command line option (akin to dt encore's "svscan") since it saves
him from opening the pipe, comparing terminated child PIDs with an
additional (the logger's) PID, and managing additional emergency situations
caused by the logger's failure himself
(especially since s6-svscan does a lot of additional stuff like catching
signals and running the corresponding scripts anyway).
:PP



                 reply	other threads:[~2019-05-03  4:13 UTC|newest]

Thread overview: [no followups] expand[flat|nested]  mbox.gz  Atom feed

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6084291556856780@sas2-a1efad875d04.qloud-c.yandex.net \
    --to=sysinit@yandex.com \
    --cc=supervision@list.skarnet.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).