runit 1.05: svlogd rotation

runit 1.05: svlogd rotation

[Alex Efros]

[-- Attachment #1: Type: text/plain, Size: 15299 bytes --]

Hi!

Probably I don't understand svlogd documentation correctly or there some
(documentation?) bugs lurking around... I noticed a huge number of rotated
log files @*.s exists, much more than allowed by logdir/config file in 'n'
directive.

In some config files I've set 'n0' to have no old rotated log files
(I need only up to last 1MB of data in "current" file). But looks like 'n0'
mean: don't remove log files at all. :) Ok, I will use 'n1' instead of 'n0'
in the future.

But there also a lot of logdirs with:
- no 'n' directive at all (this should be equal to 'n10')
- 'n2' directive
- 'n3' directive
- 'n5' directive
And in most of these directories I see a LOT of @*.s files instead of 2-3-5.
In some directories without 'n' directive also a lot of @*.s files exists.

I try to repeat this behaviour with tests like this:
    mkdir rotationtest
    echo -e 's999999\nn3\ne*\n' > rotationtest/config
    perl -le 'print "1" x 1000 for 1 .. 20000' | svlogd rotationtest/ 2>/dev/null
(s99999 and e* there because I use it in many system logdir/config files and I
wanna to have exactly same config file in this test as my system config files)
but in these tests rotation work as I expected, and no extra @*.s was created.
(BTW, looks like 'n1' and 'n2' has same effect - only 1 @*.s file created.)

Also there exists a lot of @*.u files. I know @*.u isn't usual rotated
log file, but 'incomplete' log file. I understand what it isn't correct
to rotate such incomplete logs just like usual logs because in this case
we can finish with only 10 incomplete log files with 100-200 bytes in
each left in log directory instead of last 10MB of log data.
What is recommended way to remove these @*.u files? Set special cron job?
I think it's better to automatically remove all @*.u files which OLDER than
all *@.s files in this logdir while rotating logs.

Below is example of my logdirs with extra @*.s files and their config files:

home /var/log # cat cron/config
s999999
n3
e*
home /var/log # ls -al cron
total 22688
drwxr-xr-x   2 root root   4272 Jan 21 19:50 .
drwxr-xr-x  28 root root    944 Jan 21 19:04 ..
-rw-r--r--   1 root root  14073 Oct 23 20:56 @40000000417a9b7e1e55c134.u
-rw-r--r--   1 root root   4356 Oct 23 21:19 @40000000417aa0d315ddec5c.u
-rw-r--r--   1 root root    556 Oct 23 21:22 @40000000417aa19b16c21a6c.u
-rw-r--r--   1 root root   2229 Oct 23 21:34 @40000000417aa4562aafe8ec.u
-rw-r--r--   1 root root  11844 Oct 23 22:38 @40000000417ab37221130c74.u
-rw-r--r--   1 root root   1349 Oct 23 22:45 @40000000417ab51f2e26914c.u
-rw-r--r--   1 root root    232 Oct 23 22:47 @40000000417ab57a2a4119bc.u
-rw-r--r--   1 root root    394 Oct 23 22:50 @40000000417ab61e22b9a7f4.u
-rw-r--r--   1 root root    556 Oct 23 22:53 @40000000417ab6ed19d234e4.u
-rw-r--r--   1 root root   1943 Oct 23 23:03 @40000000417ab94b35131ebc.u
-rw-r--r--   1 root root    880 Oct 23 23:09 @40000000417aba902c857024.u
-rw-r--r--   1 root root   1419 Oct 23 23:15 @40000000417abc013383b8a4.u
-rw-r--r--   1 root root    232 Oct 23 23:16 @40000000417abc6024b170b4.u
-rw-r--r--   1 root root    788 Oct 23 23:22 @40000000417abdf508384174.u
-rw-r--r--   1 root root  31270 Oct 24 02:19 @40000000417ae7410b493e7c.u
-rw-r--r--   1 root root  10378 Oct 24 03:15 @40000000417af4670179097c.u
-rw-r--r--   1 root root    394 Oct 24 03:18 @40000000417af4f235b5699c.u
-rw-r--r--   1 root root 220248 Oct 25 00:04 @40000000417c19033677ac14.u
-rw-r--r--   1 root root    928 Oct 25 00:10 @40000000417c1a7f27b6fff4.u
-rw-r--r--   1 root root   9462 Oct 25 01:03 @40000000417c26e824010c74.u
-rw-r--r--   1 root root  30052 Oct 25 03:46 @40000000417c4d0225eec314.u
-rw-r--r--   1 root root  84506 Oct 25 11:23 @40000000417cb8ed2878c56c.u
-rw-r--r--   1 root root   9506 Oct 25 12:17 @40000000417cc50e0ed82db4.u
-rw-r--r--   1 root root   2483 Oct 25 12:32 @40000000417cc8790978d7c4.u
-rw-r--r--   1 root root    880 Oct 25 12:38 @40000000417cc9ca0bc25294.u
-rw-r--r--   1 root root     70 Oct 25 12:39 @40000000417cca0107665344.u
-rw-r--r--   1 root root  12243 Oct 25 13:45 @40000000417cd9b208b2863c.u
-rw-r--r--   1 root root   4697 Oct 25 14:13 @40000000417cdffe07b292cc.u
-rw-r--r--   1 root root    863 Oct 25 14:16 @40000000417ce0f72f1ccc74.u
-rw-r--r--   1 root root   1528 Oct 25 14:27 @40000000417ce36d0ff23cbc.u
-rw-r--r--   1 root root     70 Oct 25 14:28 @40000000417ce3ae08d97314.u
-rw-r--r--   1 root root    539 Oct 25 14:30 @40000000417ce4170931b0ec.u
-rw-r--r--   1 root root   3131 Oct 25 14:48 @40000000417ce84c1287203c.u
-rw-r--r--   1 root root     70 Oct 25 14:49 @40000000417ce8b3091e676c.u
-rw-r--r--   1 root root     70 Oct 25 14:51 @40000000417ce8e607d45754.u
-rw-r--r--   1 root root    232 Oct 25 14:54 @40000000417ce9e10ec8531c.u
-rw-r--r--   1 root root   3077 Oct 25 15:12 @40000000417cee2a084a6214.u
-rw-r--r--   1 root root     70 Oct 25 15:14 @40000000417cee660657bdbc.u
-rw-r--r--   1 root root    232 Oct 25 15:16 @40000000417ceef90ae59e94.u
-rw-r--r--   1 root root   3293 Oct 25 15:35 @40000000417cf3520b952bfc.u
-rw-r--r--   1 root root    556 Oct 25 15:39 @40000000417cf45411c06924.u
-rw-r--r--   1 root root 197582 Oct 26 10:46 @40000000417e63682c4ba344.u
-rw-r--r--   1 root root 183754 Oct 27 10:46 @40000000417f52b8333e3b94.u
-rw-r--r--   1 root root    556 Oct 27 10:51 @40000000417f59090a5f99fc.u
-rw-r--r--   1 root root   1511 Oct 27 11:21 @40000000417fada62927166c.u
-rw-r--r--   1 root root   1042 Oct 27 17:21 @40000000417fb2c203490aa4.u
-rw-r--r--   1 root root 239491 Oct 28 17:05 @400000004180fd1929a47044.u
-rw-r--r--   1 root root     70 Oct 28 17:07 @400000004180fe082c2c7e9c.u
-rw-r--r--   1 root root 994148 Nov  1 11:39 @40000000418604f11666d33c.u
-rw-r--r--   1 root root     70 Nov  1 11:42 @40000000418605a12e5e5e74.u
-rw-r--r--   1 root root    539 Nov  1 11:45 @4000000041860700273ef194.u
-rw-r--r--   1 root root    232 Nov  1 11:51 @40000000418607530b1d8afc.u
-rw-r--r--   1 root root     70 Nov  1 11:52 @40000000418607e90bf034ac.u
-rw-r--r--   1 root root   1457 Nov  1 12:00 @4000000041860a670a6fd63c.u
-rw-r--r--   1 root root    232 Nov  1 12:06 @4000000041860bd401e3834c.u
-rw-r--r--   1 root root    232 Nov  1 12:12 @4000000041860c4b0d88978c.u
-rw-r--r--   1 root root    232 Nov  1 12:14 @4000000041860dcb2913c51c.u
-rw-r--r--   1 root root    880 Nov  1 12:24 @4000000041860f3c0fed85b4.u
-rw-r--r--   1 root root    556 Nov  1 12:28 @400000004186107b2427b2fc.u
-rw-r--r--   1 root root     70 Nov  1 12:31 @400000004186119631f90994.u
-rw-r--r--   1 root root   1528 Nov  1 12:44 @40000000418613ac12d55f7c.u
-rw-r--r--   1 root root    539 Nov  1 12:45 @40000000418613e60e26d35c.u
-rw-r--r--   1 root root     70 Nov  1 12:45 @400000004186141c0d6d0cc4.u
-rw-r--r--   1 root root 387145 Nov  4 02:08 @400000004189735e1e359ea4.u
-rwxr--r--   1 root root 999101 Nov  7 20:07 @40000000418e644f389079f4.s
-rwxr--r--   1 root root 999079 Nov 12 07:17 @400000004194475728b8b884.s
-rw-r--r--   1 root root 109159 Nov 12 17:27 @400000004194d68a29b9cd04.u
-rw-r--r--   1 root root 650538 Nov 15 13:59 @400000004198a4a12eae477c.u
-rwxr--r--   1 root root 999067 Nov 19 10:15 @40000000419dab8f38bdc79c.s
-rw-r--r--   1 root root 509896 Nov 21 11:48 @4000000041a0649f299c560c.u
-rw-r--r--   1 root root 651025 Nov 23 23:36 @4000000041a3ada32807853c.u
-rwxr--r--   1 root root 999084 Nov 27 17:32 @4000000041a89dfb3792381c.s
-rwxr--r--   1 root root 999125 Dec  1 12:07 @4000000041ad97cf17f1b8fc.s
-rwxr--r--   1 root root 999035 Dec  5 06:00 @4000000041b287cb21ab6c44.s
-rwxr--r--   1 root root 999038 Dec  8 23:46 @4000000041b776230e1fa384.s
-rwxr--r--   1 root root 999005 Dec 12 17:36 @4000000041bc656b2a14c22c.s
-rwxr--r--   1 root root 999108 Dec 16 11:31 @4000000041c155df1b151524.s
-rwxr--r--   1 root root 999138 Dec 20 08:24 @4000000041c6700b12c4969c.s
-rwxr--r--   1 root root 999111 Dec 24 03:26 @4000000041cb7033173bc6dc.s
-rwxr--r--   1 root root 999037 Dec 27 23:31 @4000000041d07f1f296d6284.s
-rwxr--r--   1 root root 999158 Dec 31 17:36 @4000000041d571eb2a2511f4.s
-rwxr--r--   1 root root 999000 Jan  4 11:30 @4000000041da622329b53924.s
-rwxr--r--   1 root root 999016 Jan  8 05:22 @4000000041df51e305630164.s
-rwxr--r--   1 root root 999084 Jan 11 23:15 @4000000041e441df2cd0b1c4.s
-rwxr--r--   1 root root 999043 Jan 15 17:46 @4000000041e93ac3354569bc.s
-rwxr--r--   1 root root 999156 Jan 19 11:44 @4000000041ee2beb04296ff4.s
-rw-r--r--   1 root root     14 Sep 24 13:54 config
-rw-r--r--   1 root root 623649 Jan 21 19:50 current
-rw-------   1 root root      0 Feb 18  2004 lock
-rw-r--r--   1 root root      0 Feb 18  2004 state

home /var/log # cat kernel/config
s999999
n3
e*
home /var/log # ls -l kernel
total 85244
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2c834938e7c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2c926bf70f4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ca090e69ac.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ca2979f92c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2cb125221d4.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2cb338d3654.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2cc15c470ec.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2cc3507bcac.s
-rwxr--r--  1 log nofiles 999033 Dec 30 14:21 @4000000041d3f2cd1d3a894c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ce06d8867c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2ce31a6328c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2cf21db9464.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d013ece614.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d108a9b46c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d1355a0714.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2d21c7d442c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d3042d11a4.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2d327b09f24.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d40d9aa0bc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d42dbb1d7c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2d51c2a981c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d603c77204.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d62fa4f3ec.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d7157abdbc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d7364783f4.s
-rwxr--r--  1 log nofiles 999077 Dec 30 14:21 @4000000041d3f2d822bf800c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d90b6bd5f4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2d92c278144.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2da1226fedc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2da3328a054.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2db29c5b3e4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2dc1513030c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2dd090234ac.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2de0090ef84.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2de3058bb5c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2df1b4e788c.s
-rwxr--r--  1 log nofiles 999033 Dec 30 14:21 @4000000041d3f2e000758bcc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e02635af54.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e11198c0cc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e1334144ec.s
-rwxr--r--  1 log nofiles 999077 Dec 30 14:21 @4000000041d3f2e231987304.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e31e7658f4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e4106545f4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e5047c3f2c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e530f2828c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2e62a0d28dc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e7145b8e34.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e80949dc6c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e83754739c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2e9330e26fc.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2ea224c4fc4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2eb167f4cdc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ec142d080c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ed051f4974.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ed31c40744.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:21 @4000000041d3f2ee20f56e6c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2ef157779cc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2f017297b44.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2f039301554.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:21 @4000000041d3f2f11f45bacc.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:22 @4000000041d3f2f2114b835c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f2324744ec.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f31ab326d4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f40049c8ac.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f428fbb10c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:22 @4000000041d3f2f525b9bcdc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f60e865c64.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:22 @4000000041d3f2f70a4efc14.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f7312ac4e4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f8156eafcc.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:22 @4000000041d3f2f838780c0c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2f91e3592ec.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2fa043e3c2c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2fa2837ebdc.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2fb265967dc.s
-rwxr--r--  1 log nofiles 999077 Dec 30 14:22 @4000000041d3f2fc27bab914.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2fd11bd82f4.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2fd31ac1e2c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2fe1897e024.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f2ff0373627c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:22 @4000000041d3f2ff2fe60814.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f3001735566c.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f30037f0463c.s
-rwxr--r--  1 log nofiles 999100 Dec 30 14:22 @4000000041d3f3012025e584.s
-rwxr--r--  1 log nofiles 999010 Dec 30 14:22 @4000000041d3f3020587b3ec.s
-rwxr--r--  1 log nofiles 999065 Jan  7 08:47 @4000000041de309136219504.s
-rwxr--r--  1 log nofiles 999095 Jan 19 00:13 @4000000041ed8a261ba1343c.s
-rw-r--r--  1 log nofiles     14 Sep 24 13:38 config
-rw-r--r--  1 log nofiles 289258 Jan 21 19:11 current
-rw-------  1 log nofiles      0 Feb 20  2004 lock

-- 
			WBR, Alex.

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: runit 1.05: svlogd rotation

[Gerrit Pape]

On Fri, Jan 21, 2005 at 07:56:58PM +0200, Alex Efros wrote:
> Probably I don't understand svlogd documentation correctly or there some
> (documentation?) bugs lurking around... I noticed a huge number of rotated
> log files @*.s exists, much more than allowed by logdir/config file in 'n'
> directive.
> 
> In some config files I've set 'n0' to have no old rotated log files
> (I need only up to last 1MB of data in "current" file). But looks like 'n0'
> mean: don't remove log files at all. :) Ok, I will use 'n1' instead of 'n0'
> in the future.

Yes, 'n0' means not removing log files at all on log rotation, I'll
clarify this in the documentation.

> But there also a lot of logdirs with:
> - no 'n' directive at all (this should be equal to 'n10')
> - 'n2' directive
> - 'n3' directive
> - 'n5' directive
> And in most of these directories I see a LOT of @*.s files instead of 2-3-5.
> In some directories without 'n' directive also a lot of @*.s files exists.

svlogd does not automatically reduce the number of log files.  If there
are already more log files in the log directory than specified in the
config file, svlogd will still only remove the oldest one on rotation;
adds one and removes one.  You would need to remove older log files by
hand if you change the configuration.  Maybe I should clarify that in
the documentation too, I think this may be your problem.

> Also there exists a lot of @*.u files. I know @*.u isn't usual rotated
> log file, but 'incomplete' log file. I understand what it isn't correct
> to rotate such incomplete logs just like usual logs because in this case
> we can finish with only 10 incomplete log files with 100-200 bytes in
> each left in log directory instead of last 10MB of log data.

On log file rotation svlogd treats the @*.u and @*.s files the same, it
maintains a number of @* files with a 27 characters long filename.

Regards, Gerrit.