Re: [PATCH] nnimap performance improvement for large or old groups

[Simon Josefsson <jas@extundo.com>]

Daniel Pittman <daniel@rimspace.net> writes:

> Simon Josefsson <jas@extundo.com> writes:
>> Daniel Pittman <daniel@rimspace.net> writes:
>>
>>> One of the nagging problems with using nnimap as my primary mail store
>>> is that it takes absolutely forever to enter my INBOX, as well as
>>> causing XEmacs to allocate around 150MB of memory.
>>>
>>> This is because I have been using the same INBOX for a long while now,
>>> and the 'read' info range starts with '(1 . 695705)'
>>>
>>> The code in `nnimap-request-update-info-internal' called
>>> `gnus-uncompress-range' on this, resulting in a list containing around
>>> seven million numbers -- an awful lot of memory, and time spent working
>>> through it.
>>>
>>> To address this I rewrote the code in that routine to work with
>>> compressed ranges, rather than uncompressed, which gives me a huge
>>> performance improvement (almost instant entry, vs ten to fifteen
>>> seconds) and reduces the memory use significantly.
>>>
>>>
>>> I don't think this is too performance-inefficient to include as is, and
>>> it doesn't seem to adversely effect entry into small groups or anything
>>> like that.
>>
>> Your patch seem correct.  Installed, thanks!
>>
>> It should probably be tested for a while before being included in
>> v5-10 though, the logic seem somewhat complicated and while I
>> convinced myself that it does the same thing, I might be mistaken.
>
> I completely agree that this should have long and thorough testing, for
> all that I did very carefully check the way that it worked, and made
> sure to verify every step of the new computation manually.

There are some subtle situations where a message is marked unseen on
the server (by another client), which should be propagated back into
gnus, and also when an article is removed.

But it looks OK to me.  So if nobody complains, let's move it to v5-10
in 3 months.

> Oh, but it surely is *nice* to have that in there, though. :)

Yup.