Re: nnml/nnfolder marks faster

[Dan Christensen <jdc+news@uwo.ca>]

I have a few groups that have really large .mrk files, and this is
causing slow performance.  I've noticed this more since the new
.mrk code was introduced, and it is in particular noticeable when
I *exit* a group with a large .mrk file.  But I don't know if the
new .mrk code is causing any problems, or whether it just makes
them more visible.

As an example, my nnfolder:debian.big group has about 1000 articles
in it, and the .mrk file is somehow using about 840 bytes per article,
which seems really excessive:

-rw-------    1 jdc      jdc       4231285 Sep 12 13:56 Gnus/Mail/debian.big
-rw-r--r--    1 jdc      jdc        840761 Sep 12 14:46 Gnus/Mail/debian.big.mrk
-rw-------    1 jdc      jdc        310299 Sep 12 14:42 Gnus/Mail/debian.big.nov

Here are a few excerpts from the .mrk file which look suspicious:

There are screenfuls and screenfuls of "seen" marks that look like:

  (184755 . 184758) (184759 . 184762)

which should be merged into a single long list I would think.

Also, the score caching has stuff like:

  (score (185776 . -1) (185820 . -1) (185962 . -1) (185831 . -1)
  (185991 . -1)

which isn't even stored in numerical order (how is lookup done??).  
I also wonder if this could be stored in a collapsed form using ranges

  (score ((12345 . 15000) . -1)    

etc. (in cases where there are consecutive articles with the same
score.)

Hmm, looking further, it is the score entries that are taking up 99%
of the .mrk file, and most of them don't correspond to articles that
exist in the group!  Shouldn't score cache entries be removed when
the article is deleted from the group (e.g. by the expiry mechanism)?
I smell a bug.

Dan

-- 
Dan Christensen
jdc+news@uwo.ca