IMAP Performance

IMAP Performance

[Johan Parin]

I don't know what I'm doing wrong (I certainly hope I'm doing
something wrong), but I have severe performance problem with moving
messages to IMAP folders.

Experiment :  100 1-line messages created.

Gnus
  Move between IMAP folders :           18 1/2 min
  Copy between IMAP folders :           12 min

Netscape/Win
  Copy between IMAP folders :           about 2 seconds

(the folders are on the same IMAP server)

What is the problem here? It surely shouldn't be this slow. The IMAP
server responds like this:

* OK tor Solstice (tm) Internet Mail Server (tm) IMAP4 service @ 2.0 - at Sun, 15 Apr 2001 20:08:39 +0200 (MET DST)

Any clue as to what is going on here?


Johan
-- 
Johan Parin <Johan.Parin@abc.se>
"Lisp is a programmable programming language" - John Foderaro

Re: IMAP Performance

[simon]

Johan Parin <Johan.Parin@abc.se> writes:

> I don't know what I'm doing wrong (I certainly hope I'm doing
> something wrong), but I have severe performance problem with moving
> messages to IMAP folders.
> 
> Experiment :  100 1-line messages created.
> 
> Gnus
>   Move between IMAP folders :           18 1/2 min
>   Copy between IMAP folders :           12 min
> 
> Netscape/Win
>   Copy between IMAP folders :           about 2 seconds
> 
> (the folders are on the same IMAP server)

You can evaluate `(setq imap-log "*imap-log*")' to log what is being
sent to and received from the server, and then observe at least a few
mailbox (re-)selects per moved message, making things slow.  The next
step would be to optimize nnimap for batch moving/copying and sending
patches. :)

Without modifying code, perhaps you could use splitting to move
messages were you want them? (And then leave them there.)  I know it
doesn't solve the real problem.

Re: IMAP Performance

[Russ Allbery]

Johan Parin <Johan.Parin@abc.se> writes:

> I don't know what I'm doing wrong (I certainly hope I'm doing something
> wrong), but I have severe performance problem with moving messages to
> IMAP folders.

> Experiment :  100 1-line messages created.

> Gnus
>   Move between IMAP folders :           18 1/2 min
>   Copy between IMAP folders :           12 min

> Netscape/Win
>   Copy between IMAP folders :           about 2 seconds

> (the folders are on the same IMAP server)

Moving and copying messages is rather slow in nnml too.  I'd love to see
that fixed.  It really shouldn't take hours, or even minutes, to move a
few thousand messages between nnml groups if one can use rename() in the
underlying file system (and Gnus can always fall back to a copy if it
can't).

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>

Re: IMAP Performance

[Karl Kleinpaste]

Russ Allbery <rra@stanford.edu> writes:
> Moving and copying messages is rather slow in nnml too.  I'd love to see
> that fixed.  It really shouldn't take hours, or even minutes, to move a
> few thousand messages between nnml groups if one can use rename() in the
> underlying file system

Messages can't simply be renamed because e.g. Xref headers must be rebuilt.

Re: IMAP Performance

[Paul Jarc]

Karl Kleinpaste <karl@charcoal.com> writes:
> Russ Allbery <rra@stanford.edu> writes:
> > Moving and copying messages is rather slow in nnml too.  I'd love to see
> > that fixed.  It really shouldn't take hours, or even minutes, to move a
> > few thousand messages between nnml groups if one can use rename() in the
> > underlying file system

nnmaildir uses add-name-to-file and delete-file.  Well, sometimes -
nnmaildir-request-move-article binds nnmaildir--file to the article
filename in a (let), puts the message in a buffer, and proceeds to
eval the foorm passed by Gnus.  nnmaildir-request-accept-article then
checks nnmaildir--file and tries add-name-to-file before falling back
to copying the buffer.  But I think sometimes Gnus arranges things so
that -request-accept-article isn't called until after
-request-move-article returns, so nnmaildir--file no longer holds the
filename.  This ought to be solved by fixing the backend interface, I
think.

> Messages can't simply be renamed because e.g. Xref headers must be rebuilt.

nnmaildir generates Xrefs on the fly, so this isn't a problem.  At
present, the generates Xrefs are fairly useless, but is fixable.


paul

Re: IMAP Performance

[Russ Allbery]

Karl Kleinpaste <karl@charcoal.com> writes:
> Russ Allbery <rra@stanford.edu> writes:

>> Moving and copying messages is rather slow in nnml too.  I'd love to
>> see that fixed.  It really shouldn't take hours, or even minutes, to
>> move a few thousand messages between nnml groups if one can use
>> rename() in the underlying file system

> Messages can't simply be renamed because e.g. Xref headers must be
> rebuilt.

Hurm.  Is there anything besides Xref?  I don't use Xref in mail at all; I
have nnml configured not to crosspost (I don't find that behavior useful).

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>