limit svlogd log dir size

limit svlogd log dir size

[Alex Efros]

Hi!

Is there exists way to limit size of svlogd's log directory, to guarantee
it will not become too big and result in 'disk full' error?

Usually svlogd rotate log files, but in some cases it will not, and amount
of log files become much bigger than set in 'n' option in ./config
(see example how this can happens below).

I see only way to fix this issue - set special cron script which will
remove unwanted log files, but that script probably will not be simple
because which files are "unwanted" in each directory depends in ./config
in that directory, so script should parse ./config, multiple values
on "n" and "s" options (substituting default values if options not exists)
to find total size allowed for '@*' + 'current' files and then remove
oldest files until size of left files will be less than calculated sum.
Sounds more like work for svlogd when it starting, not for cron script...


powerman@home ~/tmp $ mkdir log ; echo -e "n2\ns1" >log/config
powerman@home ~/tmp $ for i in $(seq 1 20); do perl -e '$|=1; print "x\n"; sleep 1; kill -9, $$' | svlogd log/; done
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
Killed
powerman@home ~/tmp $ ls -al log/
total 91
drwxr-xr-x   2 powerman users 1080 Feb 13 20:03 .
drwxr-xr-x  16 powerman users 2128 Feb 13 20:02 ..
-rw-r--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9db047d0a4c.u
-rwxr--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9db04c9cea4.s
-rw-r--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9dc04f87fc4.u
-rwxr--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9dc05450984.s
-rw-r--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9dd0573e59c.u
-rwxr--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9dd05c086cc.s
-rw-r--r--   1 powerman users    1 Feb 13 20:02 @4000000043f0c9de05f07c24.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9de063d1584.s
-rw-r--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9df06688a84.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9df06b53b54.s
-rw-r--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e006e455ec.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e007306a7c.s
-rw-r--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e1075ffa44.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e107abdc0c.s
-rw-r--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e207daeed4.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e2082699ec.s
-rw-r--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e30855d3c4.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e308a2693c.s
-rw-r--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e408e9651c.u
-rwxr--r--   1 powerman users    1 Feb 13 20:03 @4000000043f0c9e4091b61fc.s
-rw-r--r--   1 powerman users    6 Feb 13 20:02 config
-rw-r--r--   1 powerman users    1 Feb 13 20:03 current
-rw-------   1 powerman users    0 Feb 13 20:02 lock
powerman@home ~/tmp $ cat log/config 
n2
s1

:-(

-- 
			WBR, Alex.

Re: limit svlogd log dir size

[Gerrit Pape]

On Mon, Feb 13, 2006 at 08:16:12PM +0200, Alex Efros wrote:
> Is there exists way to limit size of svlogd's log directory, to guarantee
> it will not become too big and result in 'disk full' error?

Take a look at the 'N' config in the svlogd man page.

HTH, Gerrit.

Re: limit svlogd log dir size

[Alex Efros]

Hi!

On Tue, Feb 14, 2006 at 08:49:08AM +0000, Gerrit Pape wrote:
> > Is there exists way to limit size of svlogd's log directory, to guarantee
> > it will not become too big and result in 'disk full' error?
> Take a look at the 'N' config in the svlogd man page.

I know about 'N'. This isn't a solution. Yeah, maybe when disk will become
full svlogd will remove some log files. But what about all other software -
like MySQL? I don't think it's good idea to allow svlogd to fill all hdd
first and then fall into interesting endless loop: got 'disk full', remove
a couple of log files, work for a couple of minutes/seconds, got next
'disk full' error...

-- 
			WBR, Alex.

Re: limit svlogd log dir size

[Gerrit Pape]

On Tue, Feb 14, 2006 at 11:15:43AM +0200, Alex Efros wrote:
> On Tue, Feb 14, 2006 at 08:49:08AM +0000, Gerrit Pape wrote:
> > > Is there exists way to limit size of svlogd's log directory, to guarantee
> > > it will not become too big and result in 'disk full' error?
> > Take a look at the 'N' config in the svlogd man page.
> 
> I know about 'N'. This isn't a solution. Yeah, maybe when disk will become
> full svlogd will remove some log files. But what about all other software -
> like MySQL? I don't think it's good idea to allow svlogd to fill all hdd
> first and then fall into interesting endless loop: got 'disk full', remove
> a couple of log files, work for a couple of minutes/seconds, got next
> 'disk full' error...

Well, this normally really doesn't happen.  You constructed a case where
you made svlogd crash by sending it SIGKILL.  What should an application
do if it crashes?  It cannot know why, and at what point ist crashed,
best is to do nothing, not removing any data from before the crash at
least.

What's sending SIGKILL to the svlogd process on your system so that you
run into this problem?  You should fix that, svlogd behaves well on
SIGTERM, and shouldn't crash by itself.

Regards, Gerrit.