Re: nnimap slow on big groups

Re: nnimap slow on big groups

[Nuutti Kotivuori]

Kevin Greiner wrote:
> Nuutti Kotivuori <naked@iki.fi> writes:
>> 'nnimap' seems to be rather slow when selecting a large group - in
>> this case, around 65000 messages.
>>
>> A trivial poking around from me showed that it requests the entire
>> SEEN list from the group upon entry, while showing the "Updating
>> info for ..." message. Probably there are other operations that are
>> happening as well.
>>
>> Is there anything that can be done to alleviate the problem?
>
> 1) Agentize the server so that the 65K headers can be cached locally.
> 2) Set the agent-predicate to false so that the articles will not be
> stored locally.

I will try this later on when I get around to it, but I am pretty sure
it will not solve my problem.

The problem is not that 65k headers would have to be sent over the
network. I wouldn't even want to have them locally if I can avoid it.

The problem is that nnimap checks if the read article ranges
correspond to the 'seen' list on the server, and so fetches only the
article numbers of 65k seen articles - but even that is excessive.

-- Naked

Re: nnimap slow on big groups

[Nuutti Kotivuori]

Simon Josefsson wrote:
> Nuutti Kotivuori <naked@iki.fi> writes:
>> The problem is that nnimap checks if the read article ranges
>> correspond to the 'seen' list on the server, and so fetches only
>> the article numbers of 65k seen articles - but even that is
>> excessive.
>
> I think that is by design, and there isn't, AFAIK, any simple way of
> fixing it.  Changing the design is one option, but that is typically
> non-trivial.

Okay, this doesn't completely alleviate the problem, but:

,----[ nnimap-request-update-info-internal ]
| ;; seen might lack articles marked as read by other
| ;; imap clients! we correct this
| seen (append seen (imap-search "SEEN"))
`----

What will happen if this is not done? Will it mean that those articles
will appear unread in Gnus until read in Gnus - or will it just mean
that those articles show up in the summary buffer but when their flags
are fetched, they will be shown as seen anyway? Or will there be
spontaneous internal combustion later on?

That place is the only place that bothers me in daily use - and I can
live with either option 1 or 2 - and I might try fixing option 3 if it
so happens. Or if none of those work too well, I might change the seen
to search only the latest week by arrival times, which would be quite
okay by me.

Ofcourse this does not remove the fact that the whole read range is
first uncompressed, then differenced and then compressed again, which
again on a 65k article list is not neglible, but it is not too bad.

-- Naked