From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.4 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,MAILING_LIST_MULTI,NICE_REPLY_A autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 25503 invoked from network); 6 Oct 2020 03:58:38 -0000 Received: from alyss.skarnet.org (95.142.172.232) by inbox.vuxu.org with ESMTPUTF8; 6 Oct 2020 03:58:38 -0000 Received: (qmail 20577 invoked by uid 89); 6 Oct 2020 03:59:00 -0000 Mailing-List: contact supervision-help@list.skarnet.org; run by ezmlm Sender: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Received: (qmail 20570 invoked from network); 6 Oct 2020 03:59:00 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=heuristicsystems.com.au; s=hsa; t=1601956654; x=1602561455; bh=xy7BwH0hopGQWakFcwtg7rzSXNPM8fxILZWnVlqifnc=; h=From:Subject:To:Message-ID:Date; b=YUHijs0Zg3FDnZ0H1eGa/TE+MBQNhYxqf/dkrykf5rH+76Tm+1zbuS1YQSoNaCgd5 lcC0ySE1/vf3EbT+o0WoUtl6obJU1q1qJeJfqIzxHRAoExuPbLADTnhgPcvt2vnTft Vjrb8skqBHrwnj5ZqcdbpN4c4X061DI+5wj4m+SEZDLibnZdmD0Bk X-Authentication-Warning: b3.hs: Host noddy.hs [10.0.5.3] claimed to be [10.0.5.3] From: Dewayne Geraghty Subject: Re: s6-rc : Anomalies or normal behaviour To: "supervision@list.skarnet.org" References: <780655eb-a904-8b29-b559-80a7a0abc9f1@heuristicsystems.com.au> Message-ID: Date: Tue, 6 Oct 2020 14:57:28 +1100 User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:78.0) Gecko/20100101 Thunderbird/78.2.2 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-GB Content-Transfer-Encoding: 8bit On 4/10/2020 1:14 pm, Laurent Bercot wrote: >> 1. I expected to see the date in seconds since time epoch, but result is >> variable name >> # execlineb -Pc 'backtick D { date "+%s" } echo $D' >> $D > >  Normal behaviour, since there's no shell to interpret $D as the > contents of variable D. Try using "importas D D" before the echo: > it will read the value of D and substitute $D with this value, so > echo will print the value. Yeah, execline is annoying like that, it's > just a habit to take. >  Also, you generally want "backtick -n", to chomp the newline at > the end of your input. > > >> --- >> 2. When I use emptyenv within an execlineb script, I have a "defunct" >> zombie process >> 89685  3  S<       0:00.01   |-- s6-supervise base:time-srv >>  3020  -  S> -N -g -u ntpd --nofork >>  3601  -  Z<       0:00.00   |   `-- >> >> The time server script is >> #!/usr/local/bin/execlineb -P >> emptyenv >> multidefine -d " " "base time ntpd /usr/local/sbin/ntpd" { JAIL SERVICE >> USER PROGRAM } >> background { echo Starting service $SERVICE using $PROGRAM on $JAIL >> under user $USER } >> fdmove 2 1 >> redirfd -w 1 /m/base:time/fifo >> $PROGRAM -c /etc/ntp.conf -N -g -u $USER --nofork >> >> removing emptyenv, prevents the zombie from being created.  Is this >> normal? > >  The zombie is the echo program in your background block, since it's a > direct child of your run script and there's nothing that reaps it > after it's forked (fdmove, redirfd, ntpd - those programs don't expect > to inherit a child). So the zombie is expected. To prevent that, use > "background -d", which will doublefork your echo program, so it will > be reparented to pid 1 which will reap it properly. > EDIT My error, the problem was background, and -d fixes this. >  The anomaly is that you *don't* have that zombie without emptyenv; > my first guess is that there's something in your environment that changes > the behaviour of ntpd and makes it reap the zombie somehow. > > >> --- >> 3. Is it normal/standard/good practice to include a dependency in a >> bundle.  For example, I have a "time" bundle whose contents are >> time-srv.  time-srv starts the ntpd service, and has as a dependency >> time-log. >> >> Using "s6-rc -u change time", everything behaves as documented, ie >> starts "time" which starts time-log, then time-srv.  However >> >> # s6-rc -v 9 -d change base:time >> s6-rc: info: bringing selected services down >> s6-rc: info: processing service base:time-srv: stopping >> s6-rc: info: service base:time-srv stopped successfully >> # Starting logging service time for base with user s6log folder >> /var/log/time >> >> and the time-log continues running. > >  If you only have time-srv in your 'time' bundle, then time-srv and > time are equivalent. Telling s6-rc to bring down time will do the > exact same thing as telling it to bring down time-srv. time-log is > not impacted. So the behaviour is expected. > >  If you want "s6-rc -d change time" to also bring down time-log, then > yes, you should add time-log to the time bundle. Then 'time' will > address both time-srv and time-log. > > >> y 6 seconds  # This is time-srv >> up (pid 85131) 6 seconds                      # This is time-log,so it >> has been restarted > >  If you're using a manually created named pipe to transmit data > from time-srv to time-log, that pipe will close when time-srv exits, > and your logger will get EOF and probably exit, which is why it > stopped; but time-log's supervisor has received no instruction that > it should stop, so it will restart it. This is also expected. > >  The simplest way of achieving the behaviour you want is s6-rc's > integrated pipeline feature. Get rid of your named pipe and of your > stdout (for time-srv) and stdin (for time-log) redirections; get rid > of your time bundle definition. Then declare time-log as a consumer > for time-srv and time-srv as a producer for time-log. In the > time-log source definition directory, write 'time' into the > pipeline-name file. Then recompile your database. > >  This will automatically create a pipe between time-srv and time-log; > the pipe will be held open so it won't close even if one of the > processes exits; and it will automatically create a 'time' bundle > that contains both time-srv and time-log. > >  You're on the right track. :) > > -- >  Laurent > > Laurent, Thank-you very much. Using your advise (re 1 & 2) I've redeployed our testing platform and everything works as expected :) re 3. Implementing the producer-for/consumer-for pair, we've gone from (The application server in jail b3 to log server in jail b2 Ref1). # cat b3:named-setup2/up #!/usr/local/bin/execlineb -P define D /m/b3/fifo/named foreground { if -n { test -p $D } foreground { /usr/bin/mkfifo $D } } foreground { /usr/sbin/chown s6log:named $D } foreground { /bin/chmod 720 $D } # cat b3:named2/run #!/usr/local/bin/execlineb -P fdmove 2 1 redirfd -w 1 /m/b3/fifo/named /usr/sbin/jexec b3 /usr/local/sbin/named -f -n 1 -U 1 -u bind -c /usr/local/etc/namedb/named.conf # cat b3:named-log2/run #!/usr/local/bin/execlineb -P emptyenv redirfd -r 0 /m/b3/fifo/named /usr/sbin/jexec -U s6log b2 /usr/local/bin/s6-log -b n14 r7000 s100000 S3000000 !"/usr/bin/xz -7q" /var/log/named #Read as: run in jail b2 as user s6log the s6-log program TO # cat b3:named3/run #!/usr/local/bin/execlineb -P /usr/sbin/jexec b3 /usr/local/sbin/named -f -n 1 -U 1 -u bind -c /usr/local/etc/namedb/named.conf # cat b3:named-log3/run #!/usr/local/bin/execlineb -P emptyenv /usr/sbin/jexec -U s6log b2 /usr/local/bin/s6-log -b n14 r7000 s100000 S3000000 !"/usr/bin/xz -7q" /var/log/named A significant reduction in complexity. However, and the reason for my delay in replying. Magic happened! I was now transmitting data which crossed jail barriers (from b3 "named" to b2 "named logging"). I needed to consult with one of the FreeBSD developers to ensure that a security hole wasn't occurring. :) It appears (and I'm assuming) that s6 uses pseudo terminal sub-system to communicate. In this specific case below, per pts/3 # procstat -f 96796 95651 92390 | grep -E "text|pts" 96796 named text v r r------- - - - /jails/b3/usr/local/sbin/named 96796 named 2 v c rw------ 71 10677 - /dev/pts/3 95651 s6-log text v r r------- - - - /jails/b2/usr/local/bin/s6-log 95651 s6-log 1 v c rw------ 71 10677 - /dev/pts/3 95651 s6-log 2 v c rw------ 71 10677 - /dev/pts/3 92390 s6-fdholderd text v r r------- - - - /usr/local/bin/s6-fdholderd 92390 s6-fdholderd 2 v c rw------ 71 10677 - /dev/pts/3 I don't know if this is a good thing, so further investigation required. But for now s6 continues to make magic happen. Kind regards, Dewayne. References: 1. https://www.freebsd.org/cgi/man.cgi?query=jail&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE&arch=default&format=html 2. https://www.freebsd.org/cgi/man.cgi?query=nullfs&apropos=0&sektion=0&manpath=FreeBSD+12.1-RELEASE&arch=default&format=html