From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.comp.sysutils.supervision.general/2632 Path: news.gmane.org!.POSTED.blaine.gmane.org!not-for-mail From: Dewayne Geraghty Newsgroups: gmane.comp.sysutils.supervision.general Subject: Re: A better method than daisy-chaining logging files? Date: Mon, 17 Jun 2019 16:25:15 +1000 Message-ID: <8447f17e-0960-196d-bdf5-64a3d203cff0@heuristicsystems.com.au> References: <16c909e2-9b84-63c8-7c60-380befe28f01@heuristicsystems.com.au> Mime-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Injection-Info: blaine.gmane.org; posting-host="blaine.gmane.org:195.159.176.226"; logging-data="173662"; mail-complaints-to="usenet@blaine.gmane.org" User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:60.0) Gecko/20100101 Thunderbird/60.7.1 Cc: "supervision@list.skarnet.org" To: Brett Neumeier , Laurent Bercot Original-X-From: supervision-return-2222-gcsg-supervision=m.gmane.org@list.skarnet.org Mon Jun 17 08:26:23 2019 Return-path: Envelope-to: gcsg-supervision@m.gmane.org Original-Received: from alyss.skarnet.org ([95.142.172.232]) by blaine.gmane.org with smtp (Exim 4.89) (envelope-from ) id 1hcl5y-000j46-MA for gcsg-supervision@m.gmane.org; Mon, 17 Jun 2019 08:26:23 +0200 Original-Received: (qmail 17452 invoked by uid 89); 17 Jun 2019 06:26:41 -0000 Mailing-List: contact supervision-help@list.skarnet.org; run by ezmlm Original-Sender: Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-Id: Original-Received: (qmail 17443 invoked from network); 17 Jun 2019 06:26:40 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=heuristicsystems.com.au; s=hsa; t=1560752716; x=1561357517; bh=tYtllGs23z1zrCFICgfcQt3Qa43Amynbg14m3msR2vU=; h=Subject:To:Cc:From:Message-ID:Date; b=MPrXe1twtdsakpW0B9+LolGAGR4MYjCCN9Cb5LBBPWwQv4OcQEn6JIB+LiaTl5vWS +AnTQhvN4UPMttlg4FmIa4TLEXRWlndZ51QUCWYz8PAaCUuLSUuaDk+dJbzhqLW7mA 34B8ZEdA7pmkIxZeJk8JnTewmXohVJiB5HaGkhxS4mYHVlhTfUqzA X-Authentication-Warning: b3.hs: Host noddy.hs [10.0.5.3] claimed to be [10.0.5.3] Openpgp: preference=signencrypt In-Reply-To: Content-Language: en-AU Xref: news.gmane.org gmane.comp.sysutils.supervision.general:2632 Archived-At: On 31/05/2019 10:52 pm, Brett Neumeier wrote: > On Fri, May 31, 2019 at 4:21 AM Laurent Bercot > wrote: > >>> I just attempted to link an apache24 instance to its log files via a >>> bundle, which isn't acceptable to s6-rc-compile. >> My advice is to use s6-rc's producer/consumer mechanism for one >> of the log streams, and use a named pipe for the other one, without >> cramming it into the s6-rc mechanism. That would typically mean: >> >> - configure apache24 to output its access log to stdout >> - declare apache24 as a producer for apache24-access-log and >> apache24-access-log as a consumer for apache24 >> - apache24-access-log is a simple s6-log invocation, reading >> from its stdin >> - mkfifo /var/run/apache24/error-fifo (with appropriate rights) >> - declare that apache24 outputs its error log to >> /var/run/apache24/error-fifo >> - apache24-error-log has its run script doing something like: >> redirfd -r 0 /var/run/apache24/error-fifo s6-log your-logging-script >> - manually list apache24-error-log in apache24's dependencies, so >> apache24 doesn't start before apache24-error-log. (The pipeline >> mechanism automatically adds apache24-access-log to apache24's deps.) >> - manually define any bundles you want. >> > > For what it's worth, I use approximately this setup on my s6- and > s6-rc-managed nginx server. The only difference is that I have nginx using > /dev/stdout as its _error_ stream; and then I have a service that creates a > separate fifo for each site defined in the nginx configuration. Nginx > writes each access log to the appropriate fifo, and there's a separate > s6-log process consuming from each of the fifos. I have had no problems > whatever with that setup, it works like a charm and was really pretty > straightforward to set up. > > In fact, I find that there are a lot of services I want to run that can > either log to syslog or write to a specific filesystem location, and the > same "service writes to a fifo, s6-log reads from the fifo" mechanism works > fine for all of them. Since I use that pattern so frequently, I create a > `/run/log-fifos` directory to contain all the fifos. I think that makes the > entire mechanism pretty obvious and transparent, which is my general goal > with system administration. > > Cheers, > > Brett > Thank-you both for your sound advise. I did in fact implement Laurent's suggestions, unfortunately I was a flu early-adopter here in Australia. Brett, I think I'm more on the same page now and upon reflection, my question was pretty much a newbie as I'd failed to fully grasp that that s6-rc is not independent of s6 and that s6 dependencies are my friend which they now are. My setup is a little more complicated. I have FreeBSD jails running the service, and create a shared mount point where the service runs within the jail, communicates to a nullfs device where the fifo queue resides. It all works nicely until there is a rotation, which I induce with # s6-svc -a /run/scan/apache24-error-log The result is a directory containing -rw-r--r-- 1 mylogger www 0B Jun 17 15:34 state -rw-r--r-- 1 mylogger www 0B Jun 17 15:34 lock -rwxr--r-- 1 mylogger www 329B Jun 17 15:34 previous -rw-r--r-- 1 mylogger www 0B Jun 17 15:34 current and an error message s6-log: warning: unable to finish previous .s to logdir /var/log/httpd/error: Operation not permitted I've su'ed into the /var/log/httpd/error as "logger" and I'm able to create and compress files within the directory; so there are no permission issues. And both execlineb and s6-log are installed with 766 privs. Does s6-log require root:wheel privs to perform functions within the log directory? FYI: and largely for those trying to use *BSD and slightly challenging setup: The final logger is #!/usr/local/bin/execlineb -P s6-setuidgid mylogger redirfd -r 0 /m/jail3/fifo/apache24-error s6-log -b n14 r7000 s100000 S3000000 /var/log/httpd/error # /m is specially mounted and accessible. :) And the apache24 httpd.conf contains ErrorLog "/fifo/apache24-error" For *BSD folks mkdir -p /m/jails3/fifo /jails/jail3/fifo mount -t nullfs /m/jail3/fifo /jails/jail3/fifo Kind regards, Dewayne