Managing difficult log files

Managing difficult log files

[Dewayne Geraghty]

I would appreciate advise as to whether these are acceptable questions
for this list?

I would like to use the s6-log features for applications that have
difficult log management.  In this instance, apache. From
procstat –f `pgrep httpd`
we're advised that each http has write-append to the ErrorLog, and if we
use s6-log then we have locking problems.

So an intermediate reader appears to be a sound workaround.  Does anyone
have a better solution than

ErrorLog “|'execlineb –Pc fdmove 1 0 s6-fdholder-retrieve
/s/live/servicedirs/s6rc-fdholder/s ‘pipe:s6rc-w-final-logger’ fdswap 0
1 sh -c "read X; echo ${X}"

Please note the above is for testing, as this will be moved into a
script where logs are scanned for anomalies.

Previously I tried:
ErrorLog "|/usr/local/bin/s6-log -b n32 s10000 S7000000 T !'/usr/bin/xz
-7q' /var/log/httpd-error"

I'm looking at using a similar approach to samba and others as I dig
into this.

Kind regards, Dewayne
PS Please be tolerant as manipulating descriptors is very new to me, but
a lot of people struggle with managing apache logs and this would be a
nice-to-have use of s6-log.  (Even if the users demand daily logs via
cron & 59 23 * * *  s6-svscan -a apache24-log ;) ).  Now to samba...

Re: Managing difficult log files

[Laurent Bercot]

>So an intermediate reader appears to be a sound workaround.  Does anyone
>have a better solution than
>
>ErrorLog “|'execlineb –Pc fdmove 1 0 s6-fdholder-retrieve
>/s/live/servicedirs/s6rc-fdholder/s ‘pipe:s6rc-w-final-logger’ fdswap 0
>1 sh -c "read X; echo ${X}"

1. "cat" should work instead of your shell excerpt, shouldn't it?
Isn't it supposed to flush on every line?

2. Instead of spawning a separate process for each ErrorLog, it
should be possible to create a fifo, and declare that fifo as your
ErrorLog file. Every httpd would then write into the fifo, and you
would have a single s6-log reading from it.

--
Laurent