Re: nnimap issues

[Lars Magne Ingebrigtsen <larsi@gnus.org>]

Simon Josefsson <simon@josefsson.org> writes:

> The 'UID FETCH 1:*' command is an O(n) operation (both in terms of
> computation on client and server side but also on bandwidth) on the size
> of the mailbox, and for large mailboxes this becomes really painful.  It
> can results in MB's of data being pushed every time I press 'g'.

It only requests the last 100 flags...

> The old nnimap used the STATUS command to look for new mail instead.
> There are controversies over whether this is a good idea or not, but
> older IMAP RFCs support it and as far as I know, it is the fastest and
> most reliable way to check for new mail.  Alternatively, doing a 'UID
> FETCH 1,*' should be an O(1) operation.
>
> Generally, why is it downloading all the flags on all messages over and
> over again?  It should have all that cached anyway, except for possibly
> new messages.

Before starting working on this, I was trolling the gnus-bug group, and
it seemed like the most constant complaint was "why is nnimap showing 30
unread messages while there's really only 2"?  I think the only way to
actually tell how many new messages have actually arrived when UIDNEXT
jumps from X to X+N is to do a UID FETCH X:*, isn't it?  (Please correct
me if I'm mistaken -- I'm not an IMAP expert).

So Gnus requests (- X 100):*, to just double-check whether any of the
most recent messages have changed their flags, too.

So the primary reason for the flag fetch isn't really to get the flags
(although that's a nice side-effect), but to find out how many new
articles there are in the mail box.

If there's a faster way to do this, I'd love to hear it.

(Oh, there's QRESYNC, which might speed stuff up, but I haven't
implemented that yet.)

> 2) Nnimap no longer auto-discover that my server supports STARTTLS and
> instead tried to use direct TLS, which obviously fails since my IMAP
> server doesn't speak native TLS on the vanilla IMAP port.  I had to add
> 'nnimap-stream starttls' to my select method manually.  STARTTLS is the
> recommended way to do TLS with IMAP so it would be nice to make this
> work automatically.

Yes, it should check the capabilites and auto-STARTTLS if it's
available, I guess?

> 3) When imap-log is non-nil, the *imap-log* buffer no longer contains
> the output from the server, which makes debugging harder.

I was planning on getting rid of the log buffer totally when it works
perfectly.  Any day now!  :-)

> 4) Nnimap mail splitting is not working any more.  All my e-mail sits in
> INBOX and there is no filtering going on.  I don't see anything in
> GNUS-NEWS about any change in this area, so it must be a bug. :-)

:-)  I've now added a mention, but it's mostly related to the new
nnimap-inbox variable, and using the standard nnmail-split-methods.

>
> 5) Selecting a IMAP group results in two SELECT operations?  From
> *imap-log*:
>
> 09:34:53 2012 SELECT "INBOX.yxa-staff"
> 09:34:54 2013 SELECT "INBOX.yxa-staff"

That looks like a bug.  I'll fix it...

> 6) Deleting messages doesn't compress the article list, which for large
> operations may result in too long lines:
>
> 09:35:05 2016 UID STORE 6309,6310,6311,6312,6313,6314,6315 +FLAGS.SILENT (\Deleted)
> 09:35:06 2017 UID EXPUNGE 6309,6310,6311,6312,6313,6314,6315
> 09:35:08 2018 UID STORE 6309:6315 +FLAGS.SILENT (\Seen)

Yup; fixed.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen