[9fans] news fileserver

[9fans] news fileserver

[Steve Simon]

 Hi,

I am thinking of writing a news fileserver,
here are a few of my ideas, any comments anyone?

First the news cache:

Keep mail in a traditional newsgroup-directory
heirarchy in Eg. /lib/news/comp/os/plan9.

News items are only downloaded when
read but headers are downloaded when the
newsgroup-directory is listed.

Expiry of old messages is left to a seperate
shell script.

Newsgroups are subscribed to if their
newsgroup-directory exists

Each message is reprisented by a
message-directory, in the newsgroup-directory,
which identified by the message's ID.

Each message-directory contains a file called
header and optional file called body.

Now the fileserver:

The file server would bind this into
/n/news/comp/os/plan9 and create a virtual
heirarchy with similar to the cache but each
message-directory will contain files similar to
mail/fs (reply-to, message-id, lines, expiry,
etc...).

when the user lists a newsgroup-directory the
server will check for new messages and download
their headers.

cross-posted messages will generate a
single cache entries but multiple virtual
message-directories.

if any of the body specific (MIME) files in the
message-directory are opened then the message
body will be downloaded and split into its MIME
parts as required.

Messages will be canceled if a message headers
requests it.

Spam will be filtered using a rewrite(1)
like system, perhaps we should always scan the
NoCem groups (www.NoCem.org ?) and check their
certificates - I would be interested to know how
much spam slips past rewrite at the labs these
days and how much upkeep the rules file needs?

Maybe when the filesystem should try
to keep a idea  of when users last
accessed a given newsgroup (stat(2) of
/usr/$user/lib/news/comp.os.plan9) and hide the
virtual directories they have already read... The
more I think about this one the less I like it.
Maybe this should be a NUA (News User Agent)
problem (somone elses :-)

anything else?

-Steve

Re: [9fans] news fileserver

[bobf]

> Spam will be filtered using a rewrite(1)
> like system, perhaps we should always scan the
> NoCem groups (www.NoCem.org ?) and check their
> certificates - I would be interested to know how
> much spam slips past rewrite at the labs these
> days and how much upkeep the rules file needs?

our filters run only on mail and catch more than 95%
of the spam.  it should not be difficult to modify
the code (see scanmail(8)) to plumb it into a news system.

the patterns require constant upkeep,
but the cost is amortized over our 2,500+ recipients.