Re: nnimap and searching articles by message-id

Re: nnimap and searching articles by message-id

[Reiner Steib]

On Sat, Sep 19 2009, Peter wrote:

> It turns out that the IMAP server I am using 

Which IMAP server software is it?

> requires the message-id be stripped of the enclosing angle brackets
> when searching for messages like so:
>
> UID SEARCH HEADER Message-Id "xxxx.yyyy@foobar.com"
>
> Not sure who is not compliant here: Gnus/nnimap or the IMAP server?

Cc-ing the developer list.  Maybe someone there knows.

> I found that gnus-summary-refer-article and gnus-article-refer-article
> could be modified to suppress the angle brackets.  That doesn't seem to
> be the right solution however as I would think the fix needs to be made in
> nnimap.el instead.  Can anybody suggest a proper way to resolve the
> problem while allowing for article referencing by message-id when
> reading news through nntp and email through imap?

Here's a preliminary, untested patch.  Does it work for you?

--8<---------------cut here---------------start------------->8---
--- nnimap.el	30 Jan 2009 18:32:29 +0100	7.59
+++ nnimap.el	19 Sep 2009 11:09:02 +0200	
@@ -252,6 +252,15 @@
   :type 'boolean
   :group 'nnimap)
 
+(defcustom nnimap-search-mid-strip-angles t ;; nil
+  "Strip angles around Message-ID search.
+
+Some non-compliant (???) IMAP servers require to strip the angles
+around Message-ID when doing an UID SEARCH."
+  :version "23.2" ;; No Gnus 0.12
+  :group 'nnimap
+  :type '(choice boolean))
+
 (defvoo nnimap-need-unselect-to-notice-new-mail t
   "Unselect mailboxes before looking for new mail in them.
 Some servers seem to need this under some circumstances.")
@@ -654,7 +663,10 @@
 			   articles)))))
       (mapcar (lambda (msgid)
 		(imap-search
-		 (format "HEADER Message-Id \"%s\"" msgid)))
+		 (format "HEADER Message-Id \"%s\""
+			 (if nnimap-search-mid-strip-angles
+			     (gnus-replace-in-string msgid "[<>]" "")
+			   msgid))))
 	      articles))))
 
 (defun nnimap-group-overview-filename (group server)
--8<---------------cut here---------------end--------------->8---

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: nnimap and searching articles by message-id

[janko hrasko]

--- On Sat, 9/19/09, Reiner Steib <reinersteib+gmane@imap.cc> wrote:

> > It turns out that the IMAP server I am using 
> 
> Which IMAP server software is it?

It is a part of Oracle Beehive collaboration suite.

> 
> > requires the message-id be stripped of the enclosing
> angle brackets
> > when searching for messages like so:
> >
> > UID SEARCH HEADER Message-Id "xxxx.yyyy@foobar.com"
> >
> > Not sure who is not compliant here: Gnus/nnimap or the
> IMAP server?
> 
> Cc-ing the developer list.  Maybe someone there
> knows.

It would be good to know.  What do people think?  A quick scan through the RFC2822 - Internet Message Format reveals the following, but I am not sure if that is enough to convince anyone. 

   Semantically, the angle bracket characters are not part of the
   msg-id; the msg-id is what is contained between the two angle bracket
   characters.

Thanks for the patch.  It didn't work for what I needed (e.g. fetch the parent article) until I applied analogous changes to the other occurrence of search by message-id in nnimap.el: nnimap-request-article-part.

Thanks,
--Peter


      

Re: nnimap and searching articles by message-id

[Ted Zlatanov]

On Mon, 21 Sep 2009 16:25:11 -0700 (PDT) janko hrasko <jhrasko1@yahoo.com> wrote: 

jh> --- On Sat, 9/19/09, Reiner Steib <reinersteib+gmane@imap.cc> wrote:

>> > It turns out that the IMAP server I am using 
>> 
>> Which IMAP server software is it?

jh> It is a part of Oracle Beehive collaboration suite.

>> 
>> > requires the message-id be stripped of the enclosing
>> angle brackets
>> > when searching for messages like so:
>> >
>> > UID SEARCH HEADER Message-Id "xxxx.yyyy@foobar.com"
>> >
>> > Not sure who is not compliant here: Gnus/nnimap or the
>> IMAP server?
>> 
>> Cc-ing the developer list.  Maybe someone there
>> knows.

jh> It would be good to know.  What do people think?  A quick scan through the RFC2822 - Internet Message Format reveals the following, but I am not sure if that is enough to convince anyone. 

jh>    Semantically, the angle bracket characters are not part of the
jh>    msg-id; the msg-id is what is contained between the two angle bracket
jh>    characters.

jh> Thanks for the patch.  It didn't work for what I needed (e.g. fetch
jh> the parent article) until I applied analogous changes to the other
jh> occurrence of search by message-id in nnimap.el:
jh> nnimap-request-article-part.

Can you and Reiner put together a patch?  Should be a simple one.

I think stripping off the angle brackets MAY cost some efficiency in
other IMAP servers (maybe they hash by the full message ID), but I don't
know.  The SEARCH reference in RFC 3501
(http://www.ietf.org/rfc/rfc3501.txt) says that search by any field is
by contained string, so as far as the standard is concerned stripping
off the angle brackets is harmless.  I would make it a user option
unless we're sure Courier, Dovecot, Exchange, and UW IMAP will work the
same way with and without the change.

Ted

Re: nnimap and searching articles by message-id

[Peter]

Here is what I am running with now.  It works fine for me, although I am not
sure what is the effect of the change in nnimap-retrieve-which-headers that
Reiner first proposed.

*** /scratch/ppovinec/emacs23/share/emacs/23.0.96/lisp/gnus/nnimap.el.orig	Sat
Jun 20 21:37:54 2009
--- /scratch/ppovinec/emacs23/share/emacs/23.0.96/lisp/gnus/nnimap.el	Fri Sep 25
16:28:31 2009
***************
*** 252,257 ****
--- 252,266 ----
    :type 'boolean
    :group 'nnimap)
  
+ (defcustom nnimap-search-mid-strip-angles t ;; nil
+   "Strip angles around Message-ID search.
+ 
+ Some non-compliant (???) IMAP servers require to strip the angles
+ around Message-ID when doing an UID SEARCH."
+   :version "23.2" ;; No Gnus 0.12
+   :group 'nnimap
+   :type '(choice boolean))
+ 
  (defvoo nnimap-need-unselect-to-notice-new-mail t
    "Unselect mailboxes before looking for new mail in them.
  Some servers seem to need this under some circumstances.")
***************
*** 654,660 ****
  			   articles)))))
        (mapcar (lambda (msgid)
  		(imap-search
! 		 (format "HEADER Message-Id \"%s\"" msgid)))
  	      articles))))
  
  (defun nnimap-group-overview-filename (group server)
--- 663,672 ----
  			   articles)))))
        (mapcar (lambda (msgid)
  		(imap-search
! 		 (format "HEADER Message-Id \"%s\""
! 			 (if nnimap-search-mid-strip-angles
! 			     (gnus-replace-in-string msgid "[<>]" "")
! 			   msgid))))
  	      articles))))
  
  (defun nnimap-group-overview-filename (group server)
***************
*** 950,956 ****
    (when (nnimap-possibly-change-group group server)
      (let ((article (if (stringp article)
  		       (car-safe (imap-search
! 				  (format "HEADER Message-Id \"%s\"" article)
  				  nnimap-server-buffer))
  		     article)))
        (when article
--- 962,971 ----
    (when (nnimap-possibly-change-group group server)
      (let ((article (if (stringp article)
  		       (car-safe (imap-search
!                                   (format "HEADER Message-Id \"%s\""
!                                           (if nnimap-search-mid-strip-angles
!                                               (gnus-replace-in-string article
"[<>]" "")
!                                             article))
  				  nnimap-server-buffer))
  		     article)))
        (when article

Re: nnimap and searching articles by message-id

[Ted Zlatanov]

On Fri, 25 Sep 2009 23:41:06 +0000 (UTC) Peter <jhrasko1@yahoo.com> wrote: 

P> Here is what I am running with now.  It works fine for me, although I am not
P> sure what is the effect of the change in nnimap-retrieve-which-headers that
P> Reiner first proposed.

[patch omitted]

Does anyone have a problem with this?  Reiner, should it go in?

Ted