Re: Quicker exit and re-enter of large groups

[Ken Raeburn <raeburn@cygnus.com>]

Kai Grossjohann <grossjohann@charly.informatik.uni-dortmund.de> writes:

> Well, suppose there's just one new message, then surely it can be
> inserted in $O(n)$ time, whereas sorting all messages needs $O(n \log
> n)$ time, no?

You might need to merge and/or reorder threads, though.

> The main time killer (at least for me, I sort messages by number and
> use threading but no other sorting -- no scoring or suchlike) is
> summary buffer generation which takes the most time.  Thus, having to
> generate only a few lines and just sticking them in at the right
> positions would be a huge improvement.  I'm talking about groups with
> several thousands of (ticked) messages.

I do likewise.  Last time I did some profiling, gnus-dd-mmm took a big
chunk of that time -- nearly a third.  It's called for every message.
Can we speed it up somehow?  Or, more to the point, cut the cost of a
large number of invocations, perhaps with some sort of caching?

I've still got code around for caching "fuzzy simplified" subjects to
speed up kill-same-subject and such operations, by doing caching the
first time the operation is done on an article, increasing its cost a
bit, and bypassing all the regexp code when the cache is already
primed, greatly cutting the cost on later uses.  My current version
operates by adding one field to the mail-header structure for Gnus to
store information, and a new obarray for each group.

Perhaps we could extend that to store parsed-time info for each
article, and preserve the info across two accesses to the group?
Rebuilding the summary buffer would still take time, but less than it
does now (at least, after the first time for the group).

(Lars had said long ago he'd include this code or something like it in
Red Gnus, but I think it fell off the tail end of his to-do list.  Or
maybe I get a low score these days because of rants about multi-
threaded emacs and stuff. :-)

If we had a .overview.el file, maybe we could cache some of this data
long-term for nnml....