nnimap issues

[Simon Josefsson <simon@josefsson.org>]

Hi!  I just downloaded the latest Gnus and it found the new nnimap stuff
(thanks for working on that!) and I noticed some issues:

1) Getting new mail is really really slow!  Looking at *imap-log*
explains why, there are tons of commands like this:

...
09:27:32 1706 EXAMINE "INBOX.grisslan"
09:27:32 1707 UID FETCH 1:* FLAGS
09:27:32 1708 EXAMINE "INBOX.mail-bounce"
09:27:32 1709 UID FETCH 1935:* FLAGS
09:27:32 1710 EXAMINE "INBOX.spamassassin"
09:27:32 1711 UID FETCH 180267:* FLAGS
09:27:32 1712 EXAMINE "INBOX.2009"
09:27:32 1713 UID FETCH 30541:* FLAGS
09:27:32 1714 EXAMINE "INBOX.sent-2009"
09:27:32 1715 UID FETCH 1763:* FLAGS
...

The 'UID FETCH 1:*' command is an O(n) operation (both in terms of
computation on client and server side but also on bandwidth) on the size
of the mailbox, and for large mailboxes this becomes really painful.  It
can results in MB's of data being pushed every time I press 'g'.

The old nnimap used the STATUS command to look for new mail instead.
There are controversies over whether this is a good idea or not, but
older IMAP RFCs support it and as far as I know, it is the fastest and
most reliable way to check for new mail.  Alternatively, doing a 'UID
FETCH 1,*' should be an O(1) operation.

Generally, why is it downloading all the flags on all messages over and
over again?  It should have all that cached anyway, except for possibly
new messages.

2) Nnimap no longer auto-discover that my server supports STARTTLS and
instead tried to use direct TLS, which obviously fails since my IMAP
server doesn't speak native TLS on the vanilla IMAP port.  I had to add
'nnimap-stream starttls' to my select method manually.  STARTTLS is the
recommended way to do TLS with IMAP so it would be nice to make this
work automatically.

3) When imap-log is non-nil, the *imap-log* buffer no longer contains
the output from the server, which makes debugging harder.

4) Nnimap mail splitting is not working any more.  All my e-mail sits in
INBOX and there is no filtering going on.  I don't see anything in
GNUS-NEWS about any change in this area, so it must be a bug. :-)

5) Selecting a IMAP group results in two SELECT operations?  From
*imap-log*:

09:34:53 2012 SELECT "INBOX.yxa-staff"
09:34:54 2013 SELECT "INBOX.yxa-staff"
09:34:54 2014 UID FETCH 6309:6315 (UID RFC822.SIZE BODYSTRUCTURE BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups)])
09:34:56 2015 UID FETCH 6309 BODY.PEEK[]

6) Deleting messages doesn't compress the article list, which for large
operations may result in too long lines:

09:35:05 2016 UID STORE 6309,6310,6311,6312,6313,6314,6315 +FLAGS.SILENT (\Deleted)
09:35:06 2017 UID EXPUNGE 6309,6310,6311,6312,6313,6314,6315
09:35:08 2018 UID STORE 6309:6315 +FLAGS.SILENT (\Seen)

Thanks,
/Simon