* Re: First time caller to the show - am I understanding the fifo trick correctly? [not found] <202108250806.17P86E1p031259@invictus.wa.us.umbrellix.net> @ 2021-08-25 11:57 ` Laurent Bercot 2021-08-25 12:19 ` Ellenor Bjornsdottir 0 siblings, 1 reply; 3+ messages in thread From: Laurent Bercot @ 2021-08-25 11:57 UTC (permalink / raw) To: Ellenor Malik, supervision >Forgiving privilege separation failures and minor grammatical mistakes, does it look as if I understand the fifo trick's application in practice? Hi Ellenor, Yes, I think you have the right idea. The goal here is to redirect s6-svscan's own stdout and stderr to the stdin of the catch-all logger process, so that the supervision tree's messages, and the messages from every service that lacks a dedicated logger, go to the catch-all logger instead of /dev/console. (Because /dev/console is a terrible default place to send logs and should only be used for very critical messages such as kernel panics, or, in our userland case, for catch-all logger failures.) The problem is that we want the catch-all logger to run as a service under the supervision tree, so the s6-log process does not exist yet when we exec into s6-svscan: it will be spawned later as a grandchild of s6-svscan (with an s6-supervise intermediary). So we cannot use an anonymous pipe for this. We use a fifo instead: we can redirect init's stdout and stderr to a fifo, and later on, when the catch-all logger starts, we can instruct it (in its run script) to read from the fifo. But the Unix fifo semantics say that we *cannot* open a fifo for writing while there is no reader: open() would either block (default flags) or return -1 ENXIO (with O_NONBLOCK). So the "fifo trick" is: 1. open the fifo for reading 2. open it for writing, which now works 3. close the reading end At this point, any write() to the fifo will fail with -1 EPIPE. That is not a problem per se, except it will also generate a SIGPIPE, so in order to avoid crashing and burning, it is important to ignore SIGPIPE at the very least - or, better, to make sure that no process writes to the fifo until the catch-all logger is up. This is the case for s6-svscan and s6-supervise, so our system structure is safe; but we need to make sure that no other process starts before the catch-all logger is up, else they will just eat a SIGPIPE and die. In the s6-l-i model, s6-svscan is executed as soon as possible, on a very minimal supervision tree that only contains the catch-all logger and a few other essential "early services" (such as the shutdown daemon and an early getty). All the rest of the initialization is done in "stage 2 init", which is a script run as a child of s6-l-i's. So the end of the "fifo trick" uses the Unix fifo semantics as a synchronization mechanism: 4. fork 5. In the child, close our fd to the fifo 6. In the child, open the fifo for writing once again, *without* O_NONBLOCK. This last open() will block until the fifo has a reader. That ensures the child will only resume once the parent has completed its work and executed into s6-svscan, and the supervision tree has started and the catch-all logger is running. Then the child can exec into stage 2 init and perform the rest of the work with the guarantee that the supervision tree is operational and all the stdout and stderr messages go to the catch-all logger by default. To see exactly how to implement stage 1 init and the fifo trick as an execline script, you can checkout (or download) any version of s6-l-i *prior to* 1.0.0.0; try version 0.4.0.1, downloadable from skarnet.org if you type the URL by hand, and accessible via the v0.4.0.1 tag in git. It is very different from what it is now, as in there is no sysv compatibility at all, but stage 1 should be understandable. A few months ago, I tried adding a few conditional compilation options to s6-l-i to make it work under FreeBSD, but unfortunately the organization of the FreeBSD init is so different from Linux's, especially shutdown-wise, that my attempt only succeeded in turning the package into an unholy plate of spaghetti. At some point in the future, however, a similar-but-separate s6-freebsd-init package may make sense. -- Laurent ^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: First time caller to the show - am I understanding the fifo trick correctly? 2021-08-25 11:57 ` First time caller to the show - am I understanding the fifo trick correctly? Laurent Bercot @ 2021-08-25 12:19 ` Ellenor Bjornsdottir 0 siblings, 0 replies; 3+ messages in thread From: Ellenor Bjornsdottir @ 2021-08-25 12:19 UTC (permalink / raw) To: supervision [-- Attachment #1: Type: text/plain, Size: 4521 bytes --] Personally, I'm ~banging my head against the wall by~ writing a fork of FreeBSD init which, instead of moving to its read_ttys stage, achieves what s6-l-i does with spawning runcom2 and (s6-)svscan (but does so in a sufficiently generic manner that you don't have to do the blocking fifo trick during runcom2 if you don't want to). On 25 August 2021 11:57:11 UTC, Laurent Bercot <ska-supervision@skarnet.org> wrote: >>Forgiving privilege separation failures and minor grammatical mistakes, does it look as if I understand the fifo trick's application in practice? > > Hi Ellenor, > Yes, I think you have the right idea. > > The goal here is to redirect s6-svscan's own stdout and stderr to >the stdin of the catch-all logger process, so that the supervision >tree's messages, and the messages from every service that lacks a >dedicated logger, go to the catch-all logger instead of /dev/console. >(Because /dev/console is a terrible default place to send logs and >should only be used for very critical messages such as kernel panics, >or, in our userland case, for catch-all logger failures.) > > The problem is that we want the catch-all logger to run as a service >under the supervision tree, so the s6-log process does not exist yet >when we exec into s6-svscan: it will be spawned later as a grandchild >of s6-svscan (with an s6-supervise intermediary). So we cannot use an >anonymous pipe for this. > > We use a fifo instead: we can redirect init's stdout and stderr to >a fifo, and later on, when the catch-all logger starts, we can >instruct it (in its run script) to read from the fifo. > > But the Unix fifo semantics say that we *cannot* open a fifo for >writing while there is no reader: open() would either block (default >flags) or return -1 ENXIO (with O_NONBLOCK). So the "fifo trick" is: >1. open the fifo for reading >2. open it for writing, which now works >3. close the reading end > >At this point, any write() to the fifo will fail with -1 EPIPE. That is >not a problem per se, except it will also generate a SIGPIPE, so in >order to avoid crashing and burning, it is important to ignore SIGPIPE >at the very least - or, better, to make sure that no process writes to >the fifo until the catch-all logger is up. This is the case for >s6-svscan >and s6-supervise, so our system structure is safe; but we need to make >sure that no other process starts before the catch-all logger is up, >else they will just eat a SIGPIPE and die. > > In the s6-l-i model, s6-svscan is executed as soon as possible, on a >very minimal supervision tree that only contains the catch-all logger >and a few other essential "early services" (such as the shutdown daemon >and an early getty). All the rest of the initialization is done in >"stage 2 init", which is a script run as a child of s6-l-i's. >So the end of the "fifo trick" uses the Unix fifo semantics as a >synchronization mechanism: >4. fork >5. In the child, close our fd to the fifo >6. In the child, open the fifo for writing once again, > *without* O_NONBLOCK. > > This last open() will block until the fifo has a reader. That >ensures the child will only resume once the parent has completed >its work and executed into s6-svscan, and the supervision tree has >started and the catch-all logger is running. Then the child can exec >into stage 2 init and perform the rest of the work with the guarantee >that the supervision tree is operational and all the stdout and stderr >messages go to the catch-all logger by default. > > To see exactly how to implement stage 1 init and the fifo trick as >an execline script, you can checkout (or download) any version of >s6-l-i *prior to* 1.0.0.0; try version 0.4.0.1, downloadable from >skarnet.org if you type the URL by hand, and accessible via the >v0.4.0.1 tag in git. It is very different from what it is now, as in >there is no sysv compatibility at all, but stage 1 should be >understandable. > > A few months ago, I tried adding a few conditional compilation options >to s6-l-i to make it work under FreeBSD, but unfortunately the >organization of the FreeBSD init is so different from Linux's, >especially shutdown-wise, that my attempt only succeeded in turning >the package into an unholy plate of spaghetti. At some point in the >future, however, a similar-but-separate s6-freebsd-init package may >make sense. > >-- > Laurent > -- Sent from my Android device with K-9 Mail. Please excuse my brevity. ^ permalink raw reply [flat|nested] 3+ messages in thread
* First time caller to the show - am I understanding the fifo trick correctly? @ 2021-08-25 8:06 Ellenor Malik 0 siblings, 0 replies; 3+ messages in thread From: Ellenor Malik @ 2021-08-25 8:06 UTC (permalink / raw) To: supervision Hi supervision@! I realise this is probably more of a skaware@ question as it makes extensive reference to execline and s6-l-i, but I've been perusing the source code of the latter, hoping that I could learn what "the fifo trick" actually was (I have to admit, I've also been perusing skarnet.org/software) and I think I now get it. In case the fifo doesn't exist, the stage 1 initializer (since I won't be using s6-l-i as I'm not on Linux) has to create the fifo that $LOGGERSERVICE reads (see note [loggerservice-runscript] later in this email), then, let's call that fifo $LOGGERSERVICE/fifo, that initializer has to exec into (execline) redirfd -w -n -b 1 $LOGGERSERVICE/fifo s6-svscan. This is so technically nonsensical that I should probably be barred from ever using a computer again, but please stick with me here. That svscan invocation (or any other service supervisor that doesn't mind SIGPIPE) would then initiate supervision of [loggerservice-runscript] and the knot would be untangled. [loggerservice-runscript] #!/package/admin/execline/command/execlineb /package/admin/execline/command/redirfd -r 0 $LOGGERSERVICE/fifo s6-log n10 s10485760 T !"gzip -nq9" $LOGDIRECTORY/s6-svscan # obviously one does not simply use $variables in a runscript - these'd be replaced by the absolute paths at compile time. Forgiving privilege separation failures and minor grammatical mistakes, does it look as if I understand the fifo trick's application in practice? Thanks a lot, -- Ellenor Bjornsdottir (she) sysadmin umbrellix.net aspiring OS integrator ^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2021-08-25 12:20 UTC | newest] Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- [not found] <202108250806.17P86E1p031259@invictus.wa.us.umbrellix.net> 2021-08-25 11:57 ` First time caller to the show - am I understanding the fifo trick correctly? Laurent Bercot 2021-08-25 12:19 ` Ellenor Bjornsdottir 2021-08-25 8:06 Ellenor Malik
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).