furthermore! (nnimap splitting con't)

furthermore! (nnimap splitting con't)

[Eric Abrahamsen]

I swear I'm cursed, I'm the only person on this list who has ever had
imap splitting problems. If you search the archives for "imap split",
it's all me.

So now that I've made a little progress fixing splitting, I'm trying to
manually respool some of the mis-split articles into their proper
groups. `gnus-summary-respool-article' blows up with the backtrace
below.

The problem is that when gnus-summary-move-article calls
gnus-request-move-article, it passes along a nil "to-newsgroup"
parameter. Ie, it requests a move, but doesn't say where to move to.

From what I can tell, the problem is around gnus-sum.el:9962. There's a
branch that checks "(and (not to-newsgroup) (not select-method))", but
no branch that checks "(and (not to-newsgroup) select-method)", which
according to the docstring is where respooling would happen. I tried
changing the former to the latter, thinking that the
gnus-read-move-group-name call might do the spooling for me, but it just
prompted me for a group name, I guess because my gnus-move-split-methods
is nil. I'd never even heard of that variable. Should it be set to
something by default?

Anyway, I don't know if I'm the only person with this problem, or just
the only person trying to respool imap messages, but looking at the code
I just don't see how it would work...

Sadly,
Eri

Debugger entered--Lisp error: (wrong-type-argument stringp nil)
  decode-coding-string(nil utf-8)
  nnimap-decode-gnus-group(nil)
  nnimap-request-accept-article(nil "Girzel" t)
  gnus-request-accept-article(nil (nnimap "Girzel" ETC) t t)
  eval((gnus-request-accept-article nil (quote (nnimap "Girzel" ETC)) t t))
  nnimap-request-move-article(103 "mail.misc" "Girzel" (gnus-request-accept-article nil (quote (nnimap "Girzel" ETC)) t t) t nil)
  gnus-request-move-article(103 "nnimap+Girzel:mail.misc" "Girzel" (gnus-request-accept-article nil (quote (nnimap "Girzel" ETC)) t t) t nil)
  gnus-summary-move-article(nil nil (nnimap "Girzel" ETC))
  gnus-summary-respool-article(nil (nnimap "Girzel" ETC))
  call-interactively(gnus-summary-respool-article nil nil)

Re: furthermore! (nnimap splitting con't)

[Steinar Bang]

>>>>> Eric Abrahamsen <eric@ericabrahamsen.net>:

> I swear I'm cursed, I'm the only person on this list who has ever had
> imap splitting problems. If you search the archives for "imap split",
> it's all me.

Yes, well... I use procmail with a file that has survived and mutatet
since the mid-ninties, to split incoming into IMAP folders visited by
nnimap.

(I'm aware that there are supposedly better mail filtering out there.  I
had something sieve'y working in my cyrus years, but I happily ditched
that when I ditched cyrus for dovecot.  But it just seems like too much
work...)

Re: furthermore! (nnimap splitting con't)

[Eric Abrahamsen]

Steinar Bang <sb@dod.no> writes:

>>>>>> Eric Abrahamsen <eric@ericabrahamsen.net>:
>
>> I swear I'm cursed, I'm the only person on this list who has ever had
>> imap splitting problems. If you search the archives for "imap split",
>> it's all me.
>
> Yes, well... I use procmail with a file that has survived and mutatet
> since the mid-ninties, to split incoming into IMAP folders visited by
> nnimap.
>
> (I'm aware that there are supposedly better mail filtering out there.  I
> had something sieve'y working in my cyrus years, but I happily ditched
> that when I ditched cyrus for dovecot.  But it just seems like too much
> work...)

If it wasn't for the fact that most of my accounts are Gmail accounts,
I'd definitely be doing my splitting on the server. But Gmail filters
just seem a little weak compared to fancy splitting...

Re: furthermore! (nnimap splitting con't)

[Rainer M Krug]

[-- Attachment #1: Type: text/plain, Size: 1819 bytes --]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Steinar Bang <sb@dod.no> writes:
>
>>>>>>> Eric Abrahamsen <eric@ericabrahamsen.net>:
>>
>>> I swear I'm cursed, I'm the only person on this list who has ever had
>>> imap splitting problems. If you search the archives for "imap split",
>>> it's all me.
>>
>> Yes, well... I use procmail with a file that has survived and mutatet
>> since the mid-ninties, to split incoming into IMAP folders visited by
>> nnimap.
>>
>> (I'm aware that there are supposedly better mail filtering out there.  I
>> had something sieve'y working in my cyrus years, but I happily ditched
>> that when I ditched cyrus for dovecot.  But it just seems like too much
>> work...)
>
> If it wasn't for the fact that most of my accounts are Gmail accounts,
> I'd definitely be doing my splitting on the server. But Gmail filters
> just seem a little weak compared to fancy splitting...
>

What about imapfilter [1] ? From the README:

,----
| Description
| 
|   IMAPFilter is a mail filtering utility. It connects to remote mail servers
|   using the Internet Message Access Protocol (IMAP), sends searching queries to
|   the server and processes mailboxes based on the results. It can be used to
|   delete, copy, move, flag, etc. messages residing in mailboxes at the same or
|   different mail servers. The 4rev1 and 4 versions of the IMAP protocol are
|   supported.
| 
|   IMAPFilter uses the Lua programming language as a configuration and extension
|   language.
`----

I use it for basic filtering, but I am sure you can do very fancy things
with it.

But it is always nice to not use to many tools.

Cheers,

Rainer
 


Footnotes: 
[1]  https://github.com/lefcha/imapfilter

-- 
Rainer M. Krug

email: RMKrug<at>gmail<dot>com

[-- Attachment #2: Type: application/pgp-signature, Size: 494 bytes --]

Re: furthermore! (nnimap splitting con't)

[Lars Ingebrigtsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> From what I can tell, the problem is around gnus-sum.el:9962. There's a
> branch that checks "(and (not to-newsgroup) (not select-method))", but
> no branch that checks "(and (not to-newsgroup) select-method)", which
> according to the docstring is where respooling would happen.

Uhm, yeah.  I'm getting the same when I try to respool.  I'll get
fixin'...

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: furthermore! (nnimap splitting con't)

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Uhm, yeah.  I'm getting the same when I try to respool.  I'll get
> fixin'...

Fixed now in bzr Emacs and soon in Ma Gnus.

-- 
(domestic pets only, the antidote for overdose, milk.)
  bloggy blog http://lars.ingebrigtsen.no/

Re: furthermore! (nnimap splitting con't)

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Uhm, yeah.  I'm getting the same when I try to respool.  I'll get
>> fixin'...
>
> Fixed now in bzr Emacs and soon in Ma Gnus.

Thanks! I'll look out for it. Doesn't look like it's landed in Ma Gnus
just yet.

Re: furthermore! (nnimap splitting con't)

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Uhm, yeah.  I'm getting the same when I try to respool.  I'll get
>> fixin'...
>
> Fixed now in bzr Emacs and soon in Ma Gnus.

Okay, I've tried this out, and it's not quite there, but nearly!

The problem is in nnimap-request-accept-article. First of all,
apparently nothing in my setup is loading nnml.el, so nnml-active-number
is undefined. Either a require or an autoload, or something...

Second, when nnmail-article-group is called, it is accessing a value of
nnmail-split-methods that hasn't been let to the nnimap equivalents. So
in a bunch of places there's this:

(let ((nnmail-split-methods
			(cond
			 ((eq nnimap-split-methods 'default)
			  nnmail-split-methods)
			 (nnimap-split-methods
			  nnimap-split-methods)
			 (nnimap-split-fancy
			  'nnmail-split-fancy)))
		       (nnmail-split-fancy (or nnimap-split-fancy
					       nnmail-split-fancy)))
      (do-thing)


And that's needed here as well. With those changes in place, all seems
well.

Thanks,
Eric

Re: furthermore! (nnimap splitting con't)

[Eric Abrahamsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Uhm, yeah.  I'm getting the same when I try to respool.  I'll get
>> fixin'...
>
> Fixed now in bzr Emacs and soon in Ma Gnus.

Also! Respooling creates group directories under my ~/Mail directory.
The first time I respool to a particular group, it tells me "Creating
mail directory /home/eric/Mail/$GROUPNAME/". That directory stays empty,
so far as I can tell.

My imap messages are stored under "~/.mail", which is a value specified
in the server parameters, via the nnimap-shell-program call to dovecot.

Eric

Re: furthermore! (nnimap splitting con't)

[Eric Abrahamsen]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Lars Ingebrigtsen <larsi@gnus.org> writes:
>>
>>> Uhm, yeah.  I'm getting the same when I try to respool.  I'll get
>>> fixin'...
>>
>> Fixed now in bzr Emacs and soon in Ma Gnus.
>
> Okay, I've tried this out, and it's not quite there, but nearly!
>
> The problem is in nnimap-request-accept-article. First of all,
> apparently nothing in my setup is loading nnml.el, so nnml-active-number
> is undefined. Either a require or an autoload, or something...
>
> Second, when nnmail-article-group is called, it is accessing a value of
> nnmail-split-methods that hasn't been let to the nnimap equivalents. So
> in a bunch of places there's this:
>
> (let ((nnmail-split-methods
> 			(cond
> 			 ((eq nnimap-split-methods 'default)
> 			  nnmail-split-methods)
> 			 (nnimap-split-methods
> 			  nnimap-split-methods)
> 			 (nnimap-split-fancy
> 			  'nnmail-split-fancy)))
> 		       (nnmail-split-fancy (or nnimap-split-fancy
> 					       nnmail-split-fancy)))
>       (do-thing)
>
>
> And that's needed here as well. With those changes in place, all seems
> well.
>
> Thanks,
> Eric

Anyhoo, here's a patch for the above.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: 0002-Allow-nnimap-respooling-requests-to-function-correct.patch --]
[-- Type: text/x-diff, Size: 1960 bytes --]

From 62b55b56bed9e542e6f024f5307de196adf52d87 Mon Sep 17 00:00:00 2001
From: Eric Abrahamsen <eric@ericabrahamsen.net>
Date: Tue, 18 Mar 2014 09:50:53 +0800
Subject: [PATCH 2/2] Allow nnimap respooling requests to function correctly.

---
 lisp/nnimap.el | 23 +++++++++++++++++------
 1 file changed, 17 insertions(+), 6 deletions(-)

diff --git a/lisp/nnimap.el b/lisp/nnimap.el
index 4e07495..1b7a623 100644
--- a/lisp/nnimap.el
+++ b/lisp/nnimap.el
@@ -48,6 +48,7 @@
 
 (autoload 'auth-source-forget+ "auth-source")
 (autoload 'auth-source-search "auth-source")
+(autoload 'nnml-active-number "nnml")
 
 (nnoo-declare nnimap)
 
@@ -1118,11 +1119,21 @@ If LIMIT, first try to limit the search to the N last articles."
   (unless group
     ;; We're respooling.  Find out where mail splitting would place
     ;; this article.
-    (setq group
-	  (caar
-	   (nnmail-article-group
-	    `(lambda (group)
-	       (nnml-active-number group ,server))))))
+    (let ((nnmail-split-methods
+	   (cond
+	    ((eq nnimap-split-methods 'default)
+	     nnmail-split-methods)
+	    (nnimap-split-methods
+	     nnimap-split-methods)
+	    (nnimap-split-fancy
+	     'nnmail-split-fancy)))
+	  (nnmail-split-fancy (or nnimap-split-fancy
+				  nnmail-split-fancy)))
+      (setq group
+	    (caar
+	     (nnmail-article-group
+	      `(lambda (group)
+		 (nnml-active-number group ,server)))))))
   (setq group (nnimap-decode-gnus-group group))
   (when (nnimap-change-group nil server)
     (nnmail-check-syntax)
@@ -1157,7 +1168,7 @@ If LIMIT, first try to limit the search to the N last articles."
 		    (or (nnimap-find-uid-response "APPENDUID" (car result))
 			(nnimap-find-article-by-message-id
 			 group server message-id
-                         nnimap-request-articles-find-limit))))))))))
+			 nnimap-request-articles-find-limit))))))))))
 
 (defun nnimap-process-quirk (greeting-match type data)
   (when (and (nnimap-greeting nnimap-object)
-- 
1.9.0