supervision - discussion about system services, daemon supervision, init, runlevel management, and tools such as s6 and runit
 help / color / mirror / Atom feed
From: Dewayne Geraghty <>
To: "" <>
Subject: Re: s6-rc : Anomalies or normal behaviour
Date: Tue, 6 Oct 2020 14:57:28 +1100	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <em7ac79949-ed7c-4913-8318-9dd0afc38f59@elzian>

On 4/10/2020 1:14 pm, Laurent Bercot wrote:
>> 1. I expected to see the date in seconds since time epoch, but result is
>> variable name
>> # execlineb -Pc 'backtick D { date "+%s" } echo $D'
>> $D
>  Normal behaviour, since there's no shell to interpret $D as the
> contents of variable D. Try using "importas D D" before the echo:
> it will read the value of D and substitute $D with this value, so
> echo will print the value. Yeah, execline is annoying like that, it's
> just a habit to take.
>  Also, you generally want "backtick -n", to chomp the newline at
> the end of your input.
>> ---
>> 2. When I use emptyenv within an execlineb script, I have a "defunct"
>> zombie process
>> 89685  3  S<       0:00.01   |-- s6-supervise base:time-srv
>>  3020  -  S<s      0:00.03   | `-- /usr/local/sbin/ntpd -c /etc/ntp.conf
>> -N -g -u ntpd --nofork
>>  3601  -  Z<       0:00.00   |   `-- <defunct>
>> The time server script is
>> #!/usr/local/bin/execlineb -P
>> emptyenv
>> multidefine -d " " "base time ntpd /usr/local/sbin/ntpd" { JAIL SERVICE
>> background { echo Starting service $SERVICE using $PROGRAM on $JAIL
>> under user $USER }
>> fdmove 2 1
>> redirfd -w 1 /m/base:time/fifo
>> $PROGRAM -c /etc/ntp.conf -N -g -u $USER --nofork
>> removing emptyenv, prevents the zombie from being created.  Is this
>> normal?
>  The zombie is the echo program in your background block, since it's a
> direct child of your run script and there's nothing that reaps it
> after it's forked (fdmove, redirfd, ntpd - those programs don't expect
> to inherit a child). So the zombie is expected. To prevent that, use
> "background -d", which will doublefork your echo program, so it will
> be reparented to pid 1 which will reap it properly.
EDIT My error, the problem was background, and -d fixes this.
>  The anomaly is that you *don't* have that zombie without emptyenv;
> my first guess is that there's something in your environment that changes
> the behaviour of ntpd and makes it reap the zombie somehow.
>> ---
>> 3. Is it normal/standard/good practice to include a dependency in a
>> bundle.  For example, I have a "time" bundle whose contents are
>> time-srv.  time-srv starts the ntpd service, and has as a dependency
>> time-log.
>> Using "s6-rc -u change time", everything behaves as documented, ie
>> starts "time" which starts time-log, then time-srv.  However
>> # s6-rc -v 9 -d change base:time
>> s6-rc: info: bringing selected services down
>> s6-rc: info: processing service base:time-srv: stopping
>> s6-rc: info: service base:time-srv stopped successfully
>> # Starting logging service time for base with user s6log folder
>> /var/log/time
>> and the time-log continues running.
>  If you only have time-srv in your 'time' bundle, then time-srv and
> time are equivalent. Telling s6-rc to bring down time will do the
> exact same thing as telling it to bring down time-srv. time-log is
> not impacted. So the behaviour is expected.
>  If you want "s6-rc -d change time" to also bring down time-log, then
> yes, you should add time-log to the time bundle. Then 'time' will
> address both time-srv and time-log.
>> y 6 seconds  # This is time-srv
>> up (pid 85131) 6 seconds                      # This is time-log,so it
>> has been restarted
>  If you're using a manually created named pipe to transmit data
> from time-srv to time-log, that pipe will close when time-srv exits,
> and your logger will get EOF and probably exit, which is why it
> stopped; but time-log's supervisor has received no instruction that
> it should stop, so it will restart it. This is also expected.
>  The simplest way of achieving the behaviour you want is s6-rc's
> integrated pipeline feature. Get rid of your named pipe and of your
> stdout (for time-srv) and stdin (for time-log) redirections; get rid
> of your time bundle definition. Then declare time-log as a consumer
> for time-srv and time-srv as a producer for time-log. In the
> time-log source definition directory, write 'time' into the
> pipeline-name file. Then recompile your database.
>  This will automatically create a pipe between time-srv and time-log;
> the pipe will be held open so it won't close even if one of the
> processes exits; and it will automatically create a 'time' bundle
> that contains both time-srv and time-log.
>  You're on the right track. :)
> -- 
>  Laurent

Thank-you very much.  Using your advise (re 1 & 2) I've redeployed our
testing platform and everything works as expected :)

re 3. Implementing the producer-for/consumer-for pair, we've gone from
(The application server in jail b3 to log server in jail b2 Ref1).

# cat b3:named-setup2/up
#!/usr/local/bin/execlineb -P
define D /m/b3/fifo/named
foreground { if -n { test -p $D } foreground { /usr/bin/mkfifo $D } }
foreground { /usr/sbin/chown s6log:named $D }
foreground { /bin/chmod 720 $D }

# cat b3:named2/run
#!/usr/local/bin/execlineb -P
fdmove 2 1
redirfd -w 1 /m/b3/fifo/named
/usr/sbin/jexec b3 /usr/local/sbin/named -f -n 1 -U 1 -u bind -c

# cat b3:named-log2/run
#!/usr/local/bin/execlineb -P
redirfd -r 0 /m/b3/fifo/named
/usr/sbin/jexec -U s6log b2 /usr/local/bin/s6-log -b n14 r7000 s100000
S3000000 !"/usr/bin/xz -7q" /var/log/named
#Read as: run in jail b2 as user s6log the s6-log program

# cat b3:named3/run
#!/usr/local/bin/execlineb -P
/usr/sbin/jexec b3 /usr/local/sbin/named -f -n 1 -U 1 -u bind -c

# cat b3:named-log3/run
#!/usr/local/bin/execlineb -P
/usr/sbin/jexec -U s6log b2 /usr/local/bin/s6-log -b n14 r7000 s100000
S3000000 !"/usr/bin/xz -7q" /var/log/named

A significant reduction in complexity.  However, and the reason for my
delay in replying.  Magic happened!  I was now transmitting data which
crossed jail barriers (from b3 "named" to b2 "named logging").  I needed
to consult with one of the FreeBSD developers to ensure that a security
hole wasn't occurring. :)

It appears (and I'm assuming) that s6 uses pseudo terminal sub-system to
communicate. In this specific case below, per pts/3
# procstat -f 96796 95651 92390 | grep -E "text|pts"
96796 named             text v r r-------   -       - -
96796 named                2 v c rw------  71   10677 -   /dev/pts/3

95651 s6-log            text v r r-------   -       - -
95651 s6-log               1 v c rw------  71   10677 -   /dev/pts/3
95651 s6-log               2 v c rw------  71   10677 -   /dev/pts/3

92390 s6-fdholderd      text v r r-------   -       - -
92390 s6-fdholderd         2 v c rw------  71   10677 -   /dev/pts/3

I don't know if this is a good thing, so further investigation required.
 But for now s6 continues to make magic happen.

Kind regards, Dewayne.


  reply	other threads:[~2020-10-06  3:58 UTC|newest]

Thread overview: 6+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-10-03 22:30 Dewayne Geraghty
2020-10-04  1:58 ` Dewayne Geraghty
2020-10-04  2:20   ` Laurent Bercot
2020-10-04  2:14 ` Laurent Bercot
2020-10-06  3:57   ` Dewayne Geraghty [this message]
2020-10-06 10:29     ` Laurent Bercot

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).