From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/81943 Path: news.gmane.org!not-for-mail From: peder@news.klingenberg.no (Peder O. Klingenberg) Newsgroups: gmane.emacs.gnus.general Subject: Re: initial sync, over and over Date: Thu, 14 Jun 2012 12:27:07 +0200 Organization: Persons in a Position to Know, inc. Message-ID: References: <87ehpjow9u.fsf@maru.md5i.com> NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain X-Trace: dough.gmane.org 1339684837 18678 80.91.229.3 (14 Jun 2012 14:40:37 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Thu, 14 Jun 2012 14:40:37 +0000 (UTC) To: ding@gnus.org Original-X-From: ding-owner+M30213@lists.math.uh.edu Thu Jun 14 16:40:36 2012 Return-path: Envelope-to: ding-account@gmane.org Original-Received: from util0.math.uh.edu ([129.7.128.18]) by plane.gmane.org with esmtp (Exim 4.69) (envelope-from ) id 1SfBDr-0007I2-F5 for ding-account@gmane.org; Thu, 14 Jun 2012 16:40:31 +0200 Original-Received: from localhost ([127.0.0.1] helo=lists.math.uh.edu) by util0.math.uh.edu with smtp (Exim 4.63) (envelope-from ) id 1SfBDY-00068P-04; Thu, 14 Jun 2012 09:40:12 -0500 Original-Received: from mx2.math.uh.edu ([129.7.128.33]) by util0.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.63) (envelope-from ) id 1Sf7OX-00050w-Hu for ding@lists.math.uh.edu; Thu, 14 Jun 2012 05:35:17 -0500 Original-Received: from quimby.gnus.org ([80.91.231.51]) by mx2.math.uh.edu with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) (envelope-from ) id 1Sf7OS-0003ky-36 for ding@lists.math.uh.edu; Thu, 14 Jun 2012 05:35:17 -0500 Original-Received: from plane.gmane.org ([80.91.229.3]) by quimby.gnus.org with esmtp (Exim 4.72) (envelope-from ) id 1Sf7OP-0005KN-JO for ding@gnus.org; Thu, 14 Jun 2012 12:35:09 +0200 Original-Received: from list by plane.gmane.org with local (Exim 4.69) (envelope-from ) id 1Sf7OL-00041P-FM for ding@gnus.org; Thu, 14 Jun 2012 12:35:05 +0200 Original-Received: from luna.netfonds.no ([80.91.225.79]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 14 Jun 2012 12:35:05 +0200 Original-Received: from peder by luna.netfonds.no with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 14 Jun 2012 12:35:05 +0200 X-Injected-Via-Gmane: http://gmane.org/ Original-Lines: 67 Original-X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: luna.netfonds.no User-Agent: Gnus/5.130004 (Ma Gnus v0.4) Emacs/24.0.94 (gnu/linux) Cancel-Lock: sha1:s5SJR36rbsCIOGle/4q3o3WMHI8= X-Spam-Score: -1.9 (-) List-ID: Precedence: bulk Xref: news.gmane.org gmane.emacs.gnus.general:81943 Archived-At: Michael Welsh Duggan writes: > On one particular IMAP server (Exchange), I am always getting an initial > sync of 7 groups. Which seems rather non-initial, at least after the > first time. Is this expected? I saw something similar a while back. I analyzed the problem a bit, but never got to the point of writing code to fix it, because I found a workaround that seemed to solve it. Recently, I've seen an increase of groups that seem to need an initial sync again, so it might be worth looking into a proper fix. I haven't looked into why the problem reappears. I'll share what analysis I did, including the workaround, in case someone feels the urge to delve into it. I saw that the unexist mark didn't exist in the info objects for the groups that are repeatedly initial-synced. And the reason unexist isn't set is because permanent-flags isn't set. And permanent-flags doesn't get set because the nnimap-update-info function receives 'not-scanned in its marks-parameter. The reason for _that_ again is that EXAMINE is used instead of SELECT to, well, examine the group. Using EXAMINE to look at a group in IMAP apparently causes the server to claim that every group is read-only (observed). nnimap-parse-flags ignores the PERMANENTFLAGS sent by the server and sets permanent-flags to 'not-scanned. This causes nnimap-parse-flags to not store permanent-flags in the group parameters. Gnus decides to use EXAMINE instead of SELECT if we've seen the group before, determined by the presence of 'uidvalidity in gnus-info-params. So basically, if we've seen the group before, we've not seen the group before and need to do an initial sync. This seems a bit contradictory. To sum up: * nnimap-retrieve-group-data-early finds uidvalidity in the group parameters => * EXAMINE is used instead of SELECT => * nnimap-update-info does not set permanent-flags in the group parameters => * nnimap-update-info believes the group is read-only and doesn't update unexists => * nnimap-retrieve-group-data-early believes it needs to do an initial sync. I found two workarounds. One was to manually move a message into one of the affected group (B m). This was too slow for me (I had around 50 of the afflicted groups), so I forced nnimap-retrieve-group-data-early to issue SELECT commands in all cases, and pressed 'g' in the group buffer. The first 'g' did initial sync again, but this time, unexist was updated, so further 'g's didn't. I then changed nnimap-retrieve-group-data-early back the way it was, on the assumption that the speedup from EXAMINE was worth keeping. What I don't understand is how/why the permanent-flags group parameter got lost. It should have been set the first time (the _real_ first time) the group was seen. This may be the real root cause. Anyway, that was as far as I got. I'm not 100% sure of my analysis, I haven't really looked at the gnus code before this, so it needs to be checked. Still, I hope it's a useful starting point. ...Peder... -- This must be Thursday. I never could get the hang of Thursdays.