latest sources do not client-split IMAP anymore

latest sources do not client-split IMAP anymore

[Tibor Simko]

Hello:

After upgrading from git sources as of 05-Dec-2010 to the latest git
sources as of 14-Dec-2010, i.e. today, the client-side IMAP splitting
stopped working for me.

The last known good state is:

   be87151 Clarify comment (13-Dec-2010)

The first commit leading to troubles is:

   3595cdc (gnus-1): Don't create the nndrafts group twice

This one actually leads to gnus startup troubles for me; the startup
troubles are gone in subsequent commits, so 18c050f works apparently
well again, but then the client-side IMAP splitting is not performed and
the incoming mail stays in INBOX.  (unlike with be87151)

Best regards
-- 
Tibor Simko

Re: latest sources do not client-split IMAP anymore

[Lars Magne Ingebrigtsen]

Tibor Simko <tibor.simko@cern.ch> writes:

> After upgrading from git sources as of 05-Dec-2010 to the latest git
> sources as of 14-Dec-2010, i.e. today, the client-side IMAP splitting
> stopped working for me.

It seems to work for me, and I can't remember touching any
nnimap-relevant code the last few days...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: latest sources do not client-split IMAP anymore

[Tibor Simko]

[-- Attachment #1: Type: text/plain, Size: 249 bytes --]

On Tue, 14 Dec 2010, Lars Magne Ingebrigtsen wrote:
> It seems to work for me, and I can't remember touching any
> nnimap-relevant code the last few days...

It's due to recent changes in `gnus-setup-news'.  The attached patch
fixes things for me.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0001-gnus-start.el-gnus-setup-news-add-back-gnus-read-act.patch --]
[-- Type: text/x-diff, Size: 1444 bytes --]

From 9a6117fb06879598a9171abe4f21762dffc3eae4 Mon Sep 17 00:00:00 2001
From: Tibor Simko <tibor.simko@cern.ch>
Date: Wed, 15 Dec 2010 02:59:09 +0100
Subject: [PATCH] gnus-start.el (gnus-setup-news): add back gnus-read-active-file parts

* Add back parts of code removed in cc7d0e0 related to gnus-read-active-file;
  this fixes the problem of IMAP client-side splitting that was working fine
  in my setup up to be87151 but stopped working after 3595cdc.
---
 lisp/gnus-start.el |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)

diff --git a/lisp/gnus-start.el b/lisp/gnus-start.el
index d9e4327..194bfc1 100644
--- a/lisp/gnus-start.el
+++ b/lisp/gnus-start.el
@@ -995,8 +995,17 @@ If LEVEL is non-nil, the news will be set up at level LEVEL."
     (when (or (null gnus-read-active-file)
 	      (eq gnus-read-active-file 'some))
       (gnus-update-active-hashtb-from-killed))
+
+    ;; Read the active file and create `gnus-active-hashtb'.
+    ;; If `gnus-read-active-file' is nil, then we just create an empty
+    ;; hash table.  The partial filling out of the hash table will be
+    ;; done in `gnus-get-unread-articles'.
+    (and gnus-read-active-file
+         (not level)
+         (gnus-read-active-file nil dont-connect))
     (unless gnus-active-hashtb
       (setq gnus-active-hashtb (gnus-make-hashtable 4096)))
+
     ;; Initialize the cache.
     (when gnus-use-cache
       (gnus-cache-open))
-- 
1.7.2.3


[-- Attachment #3: Type: text/plain, Size: 30 bytes --]


Best regards
-- 
Tibor Simko

Re: latest sources do not client-split IMAP anymore

[Lars Magne Ingebrigtsen]

Tibor Simko <tibor.simko@cern.ch> writes:

> It's due to recent changes in `gnus-setup-news'.  The attached patch
> fixes things for me.

[...]

> +         (gnus-read-active-file nil dont-connect))

So reading the active file(s) makes nnimap split its mail for you, while
nothing else does this?  That's odd...  hm...

If you eval this

(gnus-read-active-file nil nil)

is IMAP mail split?

I'm unable to reproduce the problem with git Gnus.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: latest sources do not client-split IMAP anymore

[Tibor Simko]

On Wed, 15 Dec 2010, Lars Magne Ingebrigtsen wrote:
> If you eval this
>
> (gnus-read-active-file nil nil)
>
> is IMAP mail split?

If I launch gnus-git-master and I eval this, then no, it is not split.

If I launch gnus-git-master-with-the-cited-patch and I eval this, then
yes, it is split.

> I'm unable to reproduce the problem with git Gnus.

I'm using MS Exchange 2010 and a setup like:

(setq gnus-select-method '(nnnil "")
      gnus-secondary-select-methods '((nnimap "foo-user1"
                                        (nnimap-address "imap.foo.com")
                                        (nnimap-split-methods default)
                                        ;; etc
                                      (nnimap "foo-user2"
                                        (nnimap-address "imap.foo.com")
                                        ;; etc, no split
                                     
That is, two different users on the same IMAP server, one is client-side
split and one is not.  The splitting looks like:

(setq nnmail-split-methods 'nnmail-split-fancy)
(setq nnmail-split-fancy
   '(|  ;; to boot, I don't like duplicates:
        ("Gnus-Warning" "This is a duplicate" "mail.spam.duplicates")
        ;; etc
        ("List-ID" "ding\\.gnus\\.org" "mail.app.emacs.gnus")
        ;; etc
        ;; by default we do not trust the rest
        "mail.personal.misc"))

As I mentioned in the patch commit message, this used to work well until
and including be87151 (13-Dec-2010), was broken by the following two
commits (3595cdc, cc7d0e0) and works again after the cited patch.

Best regards
--
Tibor Simko

Re: latest sources do not client-split IMAP anymore

[Lars Magne Ingebrigtsen]

Tibor Simko <tibor.simko@cern.ch> writes:

>> (gnus-read-active-file nil nil)
>>
>> is IMAP mail split?
>
> If I launch gnus-git-master and I eval this, then no, it is not split.
>
> If I launch gnus-git-master-with-the-cited-patch and I eval this, then
> yes, it is split.

That's even more incomprehensible.  Your patch just put

(gnus-read-active-file nil nil)

into `gnus-setup-news'.  The patch doesn't do anything to
`gnus-read-active-file' itself, so I don't really understand this at
all. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: latest sources do not client-split IMAP anymore

[Tibor Simko]

On Thu, 16 Dec 2010, Lars Magne Ingebrigtsen wrote:
> That's even more incomprehensible.  Your patch just put
>
> (gnus-read-active-file nil nil)
>
> into `gnus-setup-news'.

Yes, `(gnus-read-active-file nil nil)' helps in that place.  Note that
it used to be there since 1997, see commit b9299e2.

Maybe the internal function call order during `gnus-setup-news' is
important?  If I eval the above after gnus started, then it does not
help anymore.  Would you like me to edebug something more concretely?

Best regards
-- 
Tibor Simko

Re: latest sources do not client-split IMAP anymore

[Lars Magne Ingebrigtsen]

Tibor Simko <tibor.simko@cern.ch> writes:

> Maybe the internal function call order during `gnus-setup-news' is
> important?  If I eval the above after gnus started, then it does not
> help anymore.

It's must be something like that.

> Would you like me to edebug something more concretely?

Do an `M-x debug-on-entry RET nnimap-request-scan RET' and mail me the
backtrace, so that I can see what the call path for the mail splitting
is. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: latest sources do not client-split IMAP anymore

[Tibor Simko]

On Thu, 16 Dec 2010, Lars Magne Ingebrigtsen wrote:
> Do an `M-x debug-on-entry RET nnimap-request-scan RET' and mail me the
> backtrace, so that I can see what the call path for the mail splitting
> is.

Done.

Best regards
-- 
Tibor Simko

Re: latest sources do not client-split IMAP anymore

[Richard Riley]

Is splitting working for you now?


On Thu, Dec 16, 2010 at 10:39 PM, Tibor Simko <tibor.simko@cern.ch> wrote:
> On Thu, 16 Dec 2010, Lars Magne Ingebrigtsen wrote:
>> Do an `M-x debug-on-entry RET nnimap-request-scan RET' and mail me the
>> backtrace, so that I can see what the call path for the mail splitting
>> is.
>
> Done.
>
> Best regards
> --
> Tibor Simko
>
>
>

Re: latest sources do not client-split IMAP anymore

[Tibor Simko]

On Sat, 18 Dec 2010, Richard Riley wrote:
> Is splitting working for you now?

It was working well until and including be87151 (13-Dec-2010), later it
required that patch.  The investigations showed that the problem may be
related to the fact that I've been using:

(setq gnus-secondary-select-methods '((nnimap "imap.cern.ch"
                                        (nnimap-address "imap.cern.ch")
                                        (nnimap-stream ssl)
                                        (nnimap-server-port 993)
                                        (nnimap-logout-timeout 10)
                                        (nnimap-inbox "INBOX")
                                        (nnimap-split-methods default)
                                        (nnir-search-engine imap))
                                      (nnimap "cds"
                                        (nnimap-address "imap.foo.com")
                                        ;; etc, different user, no split

but also:

(setq gnus-message-archive-method '(nnimap "imap.cern.ch"))

IOW, the same `imap.cern.ch' method is used in the archive method, but
with different parameters, as it were, and that Gnus may have perhaps
troubles unifying the two.  Or not.

I have not investigated this hypothesis further, since I had switched in
the meantime to a gnus <-> dovecot <-> offlineimap <-> exchange setup,
as discussed in another `gcc-mark-as-read sometimes broken on Exchange'
thread.

Best regards
-- 
Tibor Simko