From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,MAILING_LIST_MULTI,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 6387 invoked from network); 22 Jun 2023 17:16:13 -0000 Received: from alyss.skarnet.org (95.142.172.232) by inbox.vuxu.org with ESMTPUTF8; 22 Jun 2023 17:16:13 -0000 Received: (qmail 29786 invoked by uid 89); 22 Jun 2023 17:16:38 -0000 Mailing-List: contact supervision-help@list.skarnet.org; run by ezmlm Sender: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Received: (qmail 29779 invoked from network); 22 Jun 2023 17:16:37 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687454169; x=1690046169; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:to:date:from:from:to:cc:subject :date:message-id:reply-to; bh=7x8cEY18UbYZiNeEHgFj0fH0xJ+GnEIzA1ntJvZVnj4=; b=bZJLyoNH3zLKJ/v5KJSQoQWcr4NauGrezst+k5YC5oTnwn3Xc26yojoNP17o/P4qBb rh2Ln4q/6RKDYdrOw34pivma5Ti8KA1oWFHJAp0iG5JPPqYTVKgB5u/0a78Fa6VeUYA1 Ai5SRJxPySXteGZCcN4dCg7ukj1KS5L0DIsLb3rJPjql6vIX3g9OOe8Lo0ePV1ugXhNW I3Yo075YWHIe0BwiQH+lNe7iD0qbQJDHZd2Y6pJF38bALHnfAu5wUV4y9VGfsF3JL8go 6UZ77X4it7Y7nQIS23t3XizB19aRX+tC1sOuAJ8LBGHrw7vG8nDAXB8uM4zvECqXXY9I kgOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687454169; x=1690046169; h=in-reply-to:content-disposition:mime-version:references :mail-followup-to:message-id:subject:to:date:from:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=7x8cEY18UbYZiNeEHgFj0fH0xJ+GnEIzA1ntJvZVnj4=; b=kc2+KBfEleeK20zmf5JKTGnxm8I1rlbQEt+eB0tRcSh/rBIlLNsLsFTRm0vDaryh85 xqNh7NF57QXxi9OO9xZiwhV7/UyhgQQO25tUgWvE/OBEEu5vVTpCP9ZxI5sgwXuVr/iP bePFggYPfZX3x5ymPbx1QUS5l7tdSZahACfVALzsBdNI3P8SKhrr7PZINA7gm+IU2bPz 0RqtRhKjCpXcJ3vzaBO/JVvk5ubOjNnugZz/UXpyGNf7jWN/aaWg3SXyr4wHI6DalFg+ Xlk6uBIhreJeHK+SPfFyJJpUwpWiTsY7ofsdXLx+iqpshjDvBPOheL8udTkufp/usv9Y hWIw== X-Gm-Message-State: AC+VfDyMuhi9mVGKAYZEGY2sRi9Y+FISlCZXB17XlNidxRl6fqA4tu9R 0T5CkM8HPGqiQWRLMgJWkMUDmOngIlM= X-Google-Smtp-Source: ACHHUZ6agO2cs28Q6h3N7ClX8/Os8CMVguM4kwSyx5UTTmSMCjdhIjuq5WwluAxbN5/EsmasUoBrfg== X-Received: by 2002:a4a:e1c1:0:b0:560:c73b:cd8f with SMTP id n1-20020a4ae1c1000000b00560c73bcd8fmr2672397oot.6.1687454168908; Thu, 22 Jun 2023 10:16:08 -0700 (PDT) From: "Casper Ti. Vector" X-Google-Original-From: "Casper Ti. Vector" Date: Fri, 23 Jun 2023 01:16:01 +0800 To: supervision@list.skarnet.org Subject: Re: logging services with shell interaction Message-ID: Mail-Followup-To: supervision@list.skarnet.org References: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline In-Reply-To: On Thu, Oct 21, 2021 at 02:01:29AM +0800, Casper Ti. Vector wrote: > As has been said by Laurent, in the presence of a supervision system > with reliable logging and proper rotation, what `procServ' mainly does > can be done better by something like `socat' which wraps something like > `recordio', which in turn wraps the actual service process (EPICS IOC). > The devil is in the details: most importantly, when the service is to > be stopped, the ideal situation is that the actual service process gets > killed, leading to the graceful exit of `recordio' and then `socat'. It is found that socat does not do I/O fan-in/fan-out with multiple clients; it also assumes the `exec:'-ed subprocess is constantly present (i.e. it does not handle IOC restarting). So I have written a dedicated program, ipctee (see below for link to source code), that does this. I have also written a program, iotrap, that after receiving a terminating signal, first closes the stdin of its children in the hope that the latter exits cleanly, and after a tunable delay forwards the signal. This way IOCs are allowed to really run their clean-up code, instead of just being killed instantly by the signal. > So the two wrapping programs need to propagate the killing signal, and > then exit after waiting for the subprocess; since `procServ' defaults > to kill the subprocess using SIGKILL, `recordio' also needs to translate > the signal if this is to be emulated. `socat' does this correctly when > the `sighup'/`sigint'/`sigquit' options are given for `exec' addresses, > but its manual page does not state about SIGTERM. `recordio' does not > seem to propagate (let alone translate) the signal; additionally, its > output format (which is after all mainly used for debugging) feels too > low-level to me, and perhaps needs to be adjusted. Closer inspection of recordio revealed that it was designed in a smarter way: after forking, the parent exec()s into the intended program, and the children is what actually does the work of I/O forwarding. This way recordio (the children) does not need to forward signals. Based on it, I have written a program, recordln, that performs more line-oriented recording: line fragments (without the line terminator) that go through the same fd consecutively are joined before being copied to stderr. > At the facility where I am from, we use CentOS 7 and unsupervised > procServ (triple shame for a systemd opponent, s6 enthusiast and > minimalist :(), because we have not yet been bitten by log rotation > problems. It also takes quite an amount of code to implement the > dynamic management of user supervision trees for IOCs, in addition > to the adjustments needed for `recordio'. To make the situation even > worse, we are also using procServControl; anyway, I still hope we can > get rid of procServ entirely someday. Source code for the programs above are available (licence: CC0) at These programs can be tested with (in three different terminals): $ ipctee /tmp/in.sock /tmp/out.sock $ socat unix-connect:/tmp/in.sock exec:'recordln iotrap /bin/sh',sigint,sigquit $ socat unix-connect:/tmp/out.sock - Please feel free to tell me in case you find any defect in the code. The dynamic management of IOC servicedirs is being developed, and will be tested internally here before a paper gets submitted somewhere. -- My current OpenPGP key: RSA4096/0x227E8CAAB7AA186C (expires: 2024.09.30) 7077 7781 B859 5166 AE07 0286 227E 8CAA B7AA 186C