Re: Multiple article buffers

[Brian Edmonds <edmonds@cs.ubc.ca>]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

Lars> I thought I'd finally write that "multiple directory nnml" thing,
Lars> where nnml would look into sub-directories called "2101-2200", and
Lars> look at the overview file there and stuff.

I was thinking of this the other day, and it occurred to me that with
latency and such, it typically only takes a little longer to transfer
20k than it does 2k via ftp.  This led me to conclude that a scheme
(which I've dubbed nnbucket) much like nnml, but bucketing n messages
sequentially together (a la nnfolder?) would be more efficient for the
archives.  This way you'll have files called 1, n+1, 2n+1, 3n+1, etc.

As a .overview optimization, you can then split the .overview every m
buckets, so you'll have .overview.1, .overview.mn+1, .overview.mn+2,
etc.  You could of course just number sequentially in both these cases,
but the computer should have no trouble numbering in jump like this, and
it makes it more accessible to humans browsing the raw directory
structure.

This, I think, should make ftp access to archives much smoother, as the
buckets can be cached, so you get something like the asynch lookahead
for free, and the .overview files should stay reasonably bounded in
size.  Lastly, you don't get such an ugly proliferation of directories.

Now all it needs is a lisp god to implement it.   A perl script to
convert from nnml to nnbucket would also be a good idea.

Brian.