[9fans] skulker

[9fans] skulker

[Steve Simon]

I'am sure others must have done this.

A little script that runs as 3AM on sundays and tidies up stuff,
ie trimming logfiles, deleting old /mail/grey entries.

Maybe even deleting old *.[v86...] files in /sys/src
(maybe not as fossil/venti makes this less important).

I'am looking for ideas as to what I should tidy up
from cron.

-Steve

Re: [9fans] skulker

[Russ Cox]

> A little script that runs as 3AM on sundays and tidies up stuff,
> ie trimming logfiles, deleting old /mail/grey entries.
> 
> Maybe even deleting old *.[v86...] files in /sys/src
> (maybe not as fossil/venti makes this less important).
> 
> I'am looking for ideas as to what I should tidy up
> from cron.

Why bother?  You've already spent Venti space on them.

On the main Plan 9 file server, we try to be careful about
running mk clean when we're done in a directory, so that 
we don't need a cron job coming in and cleaning up after us.

We rotate the log files every January 1, so we have
/sys/log/mail.2003, etc.  It's actually useful to have the old
logs around, and it's not like deleting them would free up
space.

Russ

Re: [9fans] skulker

[Fco. J. Ballesteros]

> Why bother?  You've already spent Venti space on them.

True for /sys/src, but not for +t /usr/*/tmp directories, in our case.

> We rotate the log files every January 1, so we have
> /sys/log/mail.2003, etc.  It's actually useful to have the old

Do you put your logs into venti?
I was scared to do that, but perhaps I just don't really assume
that disks grow much faster than we can consume them...

Re: [9fans] skulker

[Russ Cox]

> > We rotate the log files every January 1, so we have
> > /sys/log/mail.2003, etc.  It's actually useful to have the old
> 
> Do you put your logs into venti?
> I was scared to do that, but perhaps I just don't really assume
> that disks grow much faster than we can consume them...

Sure.  And we put them on the WORM before Venti.
The biggest file is /sys/log/ipboot, but even that is 
only about 1GB/year, and boy does it compress well:
right now it's 690MB that gzips to 45MB.  That's not
exactly what Venti does, but it's close enough.

Russ

Re: [9fans] skulker

[C H Forsyth]

>>that disks grow much faster than we can consume them...

disks from some manufacturers consume themselves much faster
than we can consume them.

Re: [9fans] skulker

[andrey mirtchovski]

On Wed, 16 Jun 2004, Fco. J. Ballesteros wrote:

> > Why bother?  You've already spent Venti space on them.
> 
> True for /sys/src, but not for +t /usr/*/tmp directories, in our case.
> 

this reminded me to submit two patches (sources/patch hasn't been touched in
a long time!) i had almost forgotten about.

the first one makes a trivial change to /sys/lib/newuser which does a

    chmods +t /usr/$user/tmp 

after that directory has been created. 

the second is more controversial, so i'll ask 9fans' opinion first:

why create a whole new flag (-T) to 'ls' just to show files of type DMTMP,
when we can use the second char in the first field of an 'ls -l', currently reserved only for DMEXCL
files?

it is possible to have a file of both DMTMP and DMEXCL types but it seems
rare enough and we aren't guaranteed that a missing DMTMP will cause the
file to be stored to venti anyway.

the change is trivial:

    plan9% diff dirmodefmt.c /sys/src/libc/9sys/dirmodefmt.c 
    41,42d40
    <   else if(m & DMTMP)
    <       buf[1]='t';
    plan9% 

andrey

Re: [9fans] skulker

[Fco. J. Ballesteros]

> why create a whole new flag (-T) to 'ls' just to show files of type DMTMP,
> when we can use the second char in the first field of an 'ls -l', currently reserved only for DMEXCL
> files?

Why not just assume that we have another permission flag, and write
it like we do with any other one? eg talrwxr-x-rx

Would that break so many scripts and programs?

Re: [9fans] skulker

[Russ Cox]

rob and i talked about this when i added DMTMP, and
we decided that:

a) it wasn't worth putting another column in for
something that doesn't affect normal usage.

b) overloading an existing column is not possible
because of the chance of conflict.  you say it's rare
but what happens when it occurs?  by the same
logic we could get rid of the second and third x bit listing.
i don't even think it's rare: if you make your mail directory
+t (which is something people might reasonably do), then
the lock files would be +lt.

as far as the chmod +t /usr/$user/tmp is concerned,
sure, let's try it out.

your patch had lots of calgaryisms in it.  please try to
submit patches that only list the relevant changes.
when you don't, it gets in the way of the automation.

my apologies to everyone for falling behind on patch
applications.  i set up a reminder email via cron but it
has not been firing.

i've processed all the patches that were submitted as of
a few minutes ago.

russ

Re: [9fans] skulker

[rog]

> b) overloading an existing column is not possible
> because of the chance of conflict.  you say it's rare
> but what happens when it occurs?

i seem to remember that some mode bits on
unix ls used a different letter for this sort of thing.

like

t -> t
l -> l
t+l -> T (or something else)

not the most intuitive ever, but saves an ls option,
shows all the information, and is still simple to process
in shell scripts.

Re: [9fans] skulker

[William Josephson]

On Wed, Jun 16, 2004 at 06:51:26PM +0100, rog@vitanuova.com wrote:
> > b) overloading an existing column is not possible
> > because of the chance of conflict.  you say it's rare
> > but what happens when it occurs?
> 
> i seem to remember that some mode bits on
> unix ls used a different letter for this sort of thing.
> 
> like
> 
> t -> t
> l -> l
> t+l -> T (or something else)

Setuid and setgid bits:

	S -> S
	x -> x
	S+x  -> s

Since the common case is that x is set when s is,
except possibly on directories.

Re: [9fans] skulker

[Fco. J. Ballesteros]

> t+l -> T (or something else)

Just trying to learn.
Is there any big deal I'm missing in preserving
the number of characters?
We have a place for +a and a place for +l,
why not just another place for +t ?

Re: [9fans] skulker

[rog]

> We have a place for +a and a place for +l,
> why not just another place for +t ?

a) it's more space for something that's not much
used (and usually isn't relevant when using the file).

b) it's not uncommon to do something like:

	ls -l | grep '^d...x..x..x'

a change would break any scripts that do this..

Re: [9fans] skulker

[Fco. J. Ballesteros]

> a) it's more space for something that's not much
> used (and usually isn't relevant when using the file).
> 
> b) it's not uncommon to do something like:
> 
> 	ls -l | grep '^d...x..x..x'

I see. Although perhaps it would be easy to grep for such greps
and fix them.

thanks, I was assuming that breaking some scripts couldn't be
the real reason; but now I see that probably that's why.

Re: [9fans] skulker

[boyd, rounin]

> I see. Although perhaps it would be easy to grep for such greps
> and fix them.

worldwide?  on every plan 9 site?

what about awk, sed, etc ... ?

Re: [9fans] skulker

[Fco. J. Ballesteros]

> worldwide?  on every plan 9 site?
> 
> what about awk, sed, etc ... ?

Only for plan 9.

We'll know because things will break ;-)

Re: [9fans] skulker

[boyd, rounin]

> We'll know because things will break ;-)

Dies ist KAOS und wir tun dies nicht hier!  -- Siegfried

Re: [9fans] skulker

[Fco. J. Ballesteros]

> Dies ist KAOS und wir tun dies nicht hier!  -- Siegfried

Any German reader in the list? ☺

Re: [9fans] skulker

[Christoph Lohmann]

On Wed, 16 Jun 2004 22:33:46 +0200
"Fco. J. Ballesteros" <nemo@lsub.org> wrote:

> > Dies ist KAOS und wir tun dies nicht hier!  -- Siegfried
> 
> Any German reader in the list? ☺

Yes.

"This is KAOS and we don't do this here! -- Siegfried"

20h

Re: [9fans] skulker

[boyd, rounin]

> Dies ist KAOS und wir tun dies nicht hier!  -- Siegfried

esto es KAOS, nosotros no hacemos eso!  -- Siegfried

Re: [9fans] skulker

[Kenji Okamoto]

>> Dies ist KAOS und wir tun dies nicht hier!  -- Siegfried

Hmm,
"混沌の中では何をしても無駄" かな?

Kenji

Re: [9fans] skulker

[Kenji Okamoto]

>>> Dies ist KAOS und wir tun dies nicht hier!  -- Siegfried
> 
> Hmm,
> "混沌の中では何をしても無駄" かな?
> 
> Kenji
> 

I don't know why all lines have last 
...

Kenji

Re: [9fans] skulker

[Kenji Okamoto]

> I don't know why all lines have last 
> ...

Added 'AUTOMATICALLY' by our Univ's mail server, elmo.
Damn it!

Kenji

Re: [9fans] skulker

[andrey mirtchovski]

>> t+l -> T (or something else)
> 
> Just trying to learn.
> Is there any big deal I'm missing in preserving
> the number of characters?
> We have a place for +a and a place for +l,
> why not just another place for +t ?

I was following the rationale that the first character shows the file
type (DMDIR, DMAPPEND, etc) and the second shows the file mode
(DMEXCL, DMTMP).

while a file cannot be an append-only directory it is entirely
possible for it to be DMEXCL|DMTMP...

maybe packing characters/runes together in bitfield-like fashion
(charfields?) isn't such a good idea after all :)

andrey

Re: [9fans] skulker

[boyd, rounin]

> t+l -> T (or something else)

like the sysVile S bits?  what a dreadful idea that was.

> not the most intuitive ever, but saves an ls option,
> shows all the information, and is still simple to process
> in shell scripts.

it will break all the existing scripts that use this column
and when/should this happen it might not be 'good'.
 

Re: [9fans] skulker

[rog]

> like the sysVile S bits?  what a dreadful idea that was.

you're probably right.

i just thought i'd mention the possibility.

Re: [9fans] skulker

[andrey mirtchovski]

> your patch had lots of calgaryisms in it.  please try to
> submit patches that only list the relevant changes.
> when you don't, it gets in the way of the automation.

ouch, my bad -- i had assumed i used the newuser file from /n/sources
when i did the change . in the future feel free to reject anything
containing local information and leave the submitter (me) the job of
fixing it and submitting a proper patch :)

andrey

Re: [9fans] skulker

[Geoff Collyer]

/mail/grey should ideally be stored in an other file system or
something similar (ramfs, DMTMP).  It should be cleared out daily so
that legitimate callers get a chance to try again, even if their
calling behaviour is odd:

17 2 * * *	cpu	cd /mail/grey && rm -rf *.*.*.*

I store some of the logs in /sys/log in /n/other/log and do this nightly:

50 4 * * *	cpu	cd /n/other/log && chop cron cs dns ipboot runq timesync

where /rc/bin/chop is:

#!/bin/rc
# chop file ... - truncate append-only files
for (f) {
	chmod -a $f
	>$f
	chmod +a $f
}

In /cron/upas/cron, I run:

3,13,23,33,43,53 * * * *	cpu	/sys/lib/fax/faxtickle; echo -n reload >/mail/ratify/ctl

Once a month, I regenerate manual indices:

30 4 1 * *	cpu	cd /sys/man && mk indices

Once a night I ran a script that runs calendar, pull -n, and:

	du -af /n/other >/n/other/du/other

which serves as the basis for a very simple `locate' command.