From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/86857 Path: news.gmane.org!not-for-mail From: gallego@cri.ensmp.fr (Emilio =?utf-8?Q?Jes=C3=BAs?= Gallego Arias) Newsgroups: gmane.emacs.gnus.general Subject: Re: duplicates with offlineimap+dovecot Date: Sun, 14 Feb 2016 05:05:38 +0100 Organization: CRI ParisTech Message-ID: <87k2m8ndgt.fsf@cri.ensmp.fr> References: <878u34uyk4.fsf@free.fr> <84bn7q9m9i.fsf@cri.ensmp.fr> <87wpqd5u46.fsf@gnus.org> <87si11nz3m.fsf@cri.ensmp.fr> <87r3glp923.fsf@gnus.org> <87io1wfbvp.fsf@cri.ensmp.fr> <87twlcvh9v.fsf@cri.ensmp.fr> <87io1sdnpt.fsf@gnus.org> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: ger.gmane.org 1455422807 3361 80.91.229.3 (14 Feb 2016 04:06:47 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Sun, 14 Feb 2016 04:06:47 +0000 (UTC) Cc: ding@gnus.org To: Lars Ingebrigtsen Original-X-From: ding-owner+M35079@lists.math.uh.edu Sun Feb 14 05:06:35 2016 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from lists1.math.uh.edu ([129.7.128.208]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1aUnxC-0006vR-HU for ding-account@gmane.org; Sun, 14 Feb 2016 05:06:34 +0100 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by lists1.math.uh.edu with smtp (Exim 4.85) (envelope-from ) id 1aUnwY-0007Hj-Du; Sat, 13 Feb 2016 22:05:54 -0600 Original-Received: from mx1.math.uh.edu ([129.7.128.32]) by lists1.math.uh.edu with esmtps (TLSv1.2:AES128-GCM-SHA256:128) (Exim 4.85) (envelope-from ) id 1aUnwV-0007H8-4w for ding@lists.math.uh.edu; Sat, 13 Feb 2016 22:05:51 -0600 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx1.math.uh.edu with esmtps (TLSv1.2:DHE-RSA-AES128-SHA:128) (Exim 4.85) (envelope-from ) id 1aUnwP-0003sQ-Rj for ding@lists.math.uh.edu; Sat, 13 Feb 2016 22:05:51 -0600 Original-Received: from jiboia.ensmp.fr ([194.214.158.137]) by quimby.gnus.org with esmtp (Exim 4.80) (envelope-from ) id 1aUnwM-0000AT-EF; Sun, 14 Feb 2016 05:05:42 +0100 Original-Received: from rochefort (164.9.67.86.rev.sfr.net [86.67.9.164]) (authenticated bits=0) by jiboia.ensmp.fr (8.15.2/8.15.1/JMMC-22/Oct/2013) with ESMTPSA id u1E45dKd005553 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128 verify=NO); Sun, 14 Feb 2016 05:05:39 +0100 X-Url: https://www.cri.ensmp.fr/people/gallego/ In-Reply-To: <87io1sdnpt.fsf@gnus.org> (Lars Ingebrigtsen's message of "Sun, 14 Feb 2016 13:34:22 +1100") User-Agent: Gnus/5.130016 (Ma Gnus v0.16) Emacs/25.1.50 (gnu/linux) X-Miltered: at jiboia.ensmp.fr with ID 56BFFD13.000 by Joe's j-chkmail (http : // j-chkmail dot ensmp dot fr)! X-j-chkmail-Auth: USER-ID emilio.gallego_arias X-j-chkmail-Enveloppe: 56BFFD13.000 from 164.9.67.86.rev.sfr.net/164.9.67.86.rev.sfr.net/86.67.9.164/rochefort/ X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:86857 Archived-At: Hi Lars, Lars Ingebrigtsen writes: > gallego@cri.ensmp.fr (Emilio Jes?FFFAs Gallego Arias) writes: > >>> *first press of g* >>> 21:02:59 [domain] 9020 UID MOVE 4063:4064,4067,4069:4072 "folder1" >>> * OK [COPYUID 1424475231 4063:4064,4067,4069:4072 26:32] Moved UIDs. >>> * VANISHED 4062,4066,4068:4072 >>> * 3 RECENT >>> 9020 OK [HIGHESTMODSEQ 5193] Move completed. >> >> I interpret this as "only 4062,4066,4068:4072" got moved. > > I can't find anything in the IMAP MOVE standard that says something like > that. And it only talks about having to be careful about streaming > other commands after MOVE as a MOVE may lead to articles being > renumbered. And the VANISHED stuff is from unrelated QRESYNC support, > if I'm reading the QRESYNC RFC correctly. Note that gnus is currently streaming MOVE commands, however I'm not sure the RFC means to allow this. > And, as you see from the VANISHED above there, most of the stuff it says > is vanished is stuff that we're not even trying to move (4062, 4066). I'm sorry, I trimmed too much of the log, if you look at my previous mail you'll see: > 9019 UID MOVE 4062,4066,4068 "unrelated2" > 9020 UID MOVE 4063:4064,4067,4069:4072 "folder1" > * OK [COPYUID 1424475218 4062,4066,4068 376:378] Moved UIDs. > * OK [COPYUID 1424475231 4063:4064,4067,4069:4072 26:32] Moved UIDs. > * VANISHED 4062,4066,4068:4072 Thus yeah, we ordered the move of 4062, etc... and the server: a) confirms the move for all the messages. b) doesn't confirm the expunge from INBOX for all of them! Thus, it is clear that 4063 was moved but not expunged from the mailbox; hence the duplicated msg... > So I don't know what's going on on the server... I have no clue either but the server is a 100% vanilla Debian Dovecot. I have this problem with several other servers not administered by me so if there's a server problem here is not only on my server. After some more experiments, I believe the bug can be reproduced as follows: install a Debian stable with Dovecot, then have Gnus to issue multiple (pipelined) MOVE commands with QRESYNC enabled. I'd be happy to get in touch with dovecot upstream if we deem it OK. Regarding the log, note that the RFC says: Because a MOVE applies to a set of messages, it might fail partway through the set. Regardless of whether the command is successful in moving the entire set, each individual message SHOULD either be moved or unaffected. The server MUST leave each message in a state where ^^^^ it is in at least one of the source or target mailboxes (no message can be lost or orphaned). The server SHOULD NOT leave any message in ^^^^^^^^^^ both mailboxes (it would be bad for a partial failure to result in a bunch of duplicate messages). This is true even if the server returns a tagged NO response to the command. So Dovecot here appears to be violating the "SHOULD NOT"; it effectively leaves the message in the two mailboxes (even if it notifies us that it was not VANISHED from INBOX). However: > RFC6851: > The server may send EXPUNGE (or VANISHED) responses before the tagged > response, so the client cannot safely send more commands with message > sequence number arguments while the server is processing MOVE or UID > MOVE. Doesn't that mean that gnus should not stream the two MOVES? Thanks again for the help! Regards, E.