Re: overview file access when spooling and nnml/nnimap performance

[Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann)]

I think nnimap is even slower than nnml because the NOV parsing done
by Gnus is so fast.  In fact, nnimap now includes a NOV cache feature
which fetches overview info from the IMAP server only once.  After
that, the overview information is stored in a file in ~/News and
subsequently fetched from there.

I use a Cyrus server and I do server-side splitting with Sieve.  So
getting new mail is real fast -- O(n) where n is the number of
subscribed groups.  The splitting happens in the background.  That's a
nice feature of the Gnus/nnimap combination.

You're right, it seems strange that operations on large groups are so
slow.  Recently, I moved all articles from one 3500 message nnml group
to an 8000 message group, and that was real slow.  Towards the end,
moving got a lot faster.

I'm not sure how Gnus' speed could be improved here.  One could try to
read ever larger chunks from the end of the file until the beginning
of the last line is found.  And writing new lines could happen by
appending to the existing file.  Yes, that might be faster for large
overview files, but what's the size where it starts winning.

As to displaying large summary buffers, I find that I normally get by
with just displaying few messages.  I switched from total-expire to
auto-expire, and most old messages are marked as read.  So when
entering a group, I normally see just a few ticked messages, plus the
new ones, plus the old ones that are parents of the other shown ones.

Rather than displaying thousands of messages to search for an old one,
I use nnir to search for it.  (Should use nnir much more often,
though.)

kai
-- 
I like BOTH kinds of music.