Re: nnimap vs dbmail

[Simon Josefsson <simon@josefsson.org>]

James Cloos <cloos+math_uh-ding@jhcloos.com> writes:

> I'm looking into converting over to using imap as the backstore for my
> mail and am looking at using dbmail for that.
>
> I'm seeing some buglets in nnimap.  
>
> As an example, the unread article counts in the Group buffer are almost
> always incorrect.  And errors often get printed to the minibuffer.
>
> One issue is that dbmail gives each article a server-unique UID rather
> than a just user-unique or folder-unique.  So they are never contiguous.
> nnimap seems to expect contiguous UIDs and goes searching for them,
> causing IMAP errors.

Actually, it is Gnus that assumes contiguous article numbers.  When
there are "holes" in the ranges, the Group buffer unread article count
estimate (yes, it is documented as an estimate) will be wrong.  This
happens for nntp, nnml etc too.

> ,----< 155 UID FETCH 3665,3667 (UID RFC822.SIZE BODY BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups)]) >
> | 155 BAD invalid message range specified
> `----
>
> So why fetch 3665 and 3667 if they were not listed in any of the UID
> searches?

They may be UNSEEN DELETED, but I agree that it should be able to figure
out that these do not exist.  I suspect that the reason nnimap asked
about this is that Gnus asked about those articles.  Running Gnus and
nnimap under edebug will help to understand why information about those
articles are requested.

>
> ,----< 156 UID FETCH 3665,3667 (UID RFC822.SIZE BODY BODY.PEEK[HEADER.FIELDS (Subject From Date Message-Id References In-Reply-To Xref To Newsgroups)]) >
> | 156 BAD invalid message range specified
> `----
>
> Twice, even.

Huh, that's weird.  It might indicate that something non-normal (i.e., a
bug) is happening.

> ,----< 157 UID STORE 3665,3667 +FLAGS (\Seen) >
> | 157 BAD invalid message range specified
> `----
>
> And dbmail doesn't like storing flags for non-extant UIDs.

RFC 3501 says:

6.4.8.  UID Command
...
      A non-existent unique identifier is ignored without any error
      message generated.  Thus, it is possible for a UID FETCH command
      to return an OK without any data or a UID COPY or UID STORE to
      return an OK without performing any operations.

I'd say this is a dbmail bug.

> This time the errors did not prevent me from seeing new mail, but that
> isn't always the case.  Some new mail is inaccessible in gnus because of
> such errors.  A test shows that one of those errors is probably due to
> this line in list/imap.el:
>
> ,----[ from gnus/lisp/imap.el (imap-parse-body) ]
> | (assert (eq (char-after) ?\)) nil "In imap-parse-body")
> `----
>
> Are those (assert)s necessary?  They effectively lose mail.

Can you quote the content of the ' *nnimap* foo' buffer when that
happens?  Alternatively, try '(setq imap-log t)' and quote the contents
from *imap-log* related to that parse assertion.

The reason for the assert is that imap.el cannot parse what the server
is sending.  I think there are only two reasons for that happening:
imap.el is buggy or the server is buggy.  The effects of either will be
that it appears as if a message is missing (or even worse consequences).
I'm not sure what could be done to avoid this.

/Simon