gnus IMAP vs. Exchange Server 2007

gnus IMAP vs. Exchange Server 2007

[Mat Marcus]

Hi,

At work, we recently "upgraded" from MS Exchange Server 2003 to 2007. Suddenly
gnus stopped working as an IMAP client. As Thunderbird also had problems I
decided to do a little debugging. After viewing the *imap-log* I noticed that in
the course of the seesion, gnus sends the command 

124 FETCH 1,* UID

to which MS Exchange Server 2007

124 BAD The specified message set is invalid.

Things go downhill from there. I also manually telneted in and issued the same
command, and received the same result. My questions are:

1) is 124 FETCH 1,* UID a valid IMAP command for non-empty mailboxes?
2) if so, is this an MS Exchange Server 2007 bug?
3) in either case is there a patch workaround for gnus? I am currently running
gnus "5.11" that comes with emacs 22.1.1. 

Sorry if this has already been covered--I am not an expert in IMAP protocols,
Exchange Server, or gnus.

Thanks,
Mat

Re: gnus IMAP vs. Exchange Server 2007

[Vitaly Mayatskikh]

Mat Marcus <mat-lists@emarcus.org> writes:

> At work, we recently "upgraded" from MS Exchange Server 2003 to 2007. Suddenly
> gnus stopped working as an IMAP client. As Thunderbird also had problems I
> decided to do a little debugging. After viewing the *imap-log* I noticed that in
> the course of the seesion, gnus sends the command 
>
> 124 FETCH 1,* UID
>
> to which MS Exchange Server 2007
>
> 124 BAD The specified message set is invalid.
>
> Things go downhill from there. I also manually telneted in and issued the same
> command, and received the same result. My questions are:
>
> 1) is 124 FETCH 1,* UID a valid IMAP command for non-empty mailboxes?
> 2) if so, is this an MS Exchange Server 2007 bug?
> 3) in either case is there a patch workaround for gnus? I am currently running
> gnus "5.11" that comes with emacs 22.1.1. 
>
> Sorry if this has already been covered--I am not an expert in IMAP protocols,
> Exchange Server, or gnus.

(setq imap-enable-exchange-bug-workaround t)

"Documentation:
Send FETCH UID commands as *:* instead of *.
Enabling this appears to be required for some servers (e.g.,
Microsoft Exchange) which otherwise would trigger a response 'BAD
The specified message set is invalid.'."

-- 
wbr, Vitaly

Re: gnus IMAP vs. Exchange Server 2007

[David Engster]

Vitaly Mayatskikh <v.mayatskih@gmail.com> writes:
> Mat Marcus <mat-lists@emarcus.org> writes:
>> 3) in either case is there a patch workaround for gnus? I am currently running
>> gnus "5.11" that comes with emacs 22.1.1. 

> (setq imap-enable-exchange-bug-workaround t)

Let me add that this variable does not exist in Gnus 5.11. You will have
to update to the latest NoGnus (0.10, from May 2008), or to CVS.

-David

Re: gnus IMAP vs. Exchange Server 2007

[Mat Marcus]

On Tue, Sep 23, 2008 at 11:21 PM, Vitaly Mayatskikh
<v.mayatskih@gmail.com> wrote:
> Mat Marcus <mat-lists@emarcus.org> writes:
>
>> At work, we recently "upgraded" from MS Exchange Server 2003 to 2007. Suddenly
>> gnus stopped working as an IMAP client. As Thunderbird also had problems I
>> decided to do a little debugging. After viewing the *imap-log* I noticed that in
>> the course of the seesion, gnus sends the command
>>
>> 124 FETCH 1,* UID
>>
>> to which MS Exchange Server 2007
>>
>> 124 BAD The specified message set is invalid.
>>
>> Things go downhill from there. I also manually telneted in and issued the same
>> command, and received the same result. My questions are:
>>
>> 1) is 124 FETCH 1,* UID a valid IMAP command for non-empty mailboxes?
>> 2) if so, is this an MS Exchange Server 2007 bug?
>> 3) in either case is there a patch workaround for gnus? I am currently running
>> gnus "5.11" that comes with emacs 22.1.1.
>>
>> Sorry if this has already been covered--I am not an expert in IMAP protocols,
>> Exchange Server, or gnus.
>
> (setq imap-enable-exchange-bug-workaround t)
>
> "Documentation:
> Send FETCH UID commands as *:* instead of *.
> Enabling this appears to be required for some servers (e.g.,
> Microsoft Exchange) which otherwise would trigger a response 'BAD
> The specified message set is invalid.'."

Thanks for the reply. This variable is just what I need. C-h v
indicates that it is not present in the version of gnus that I have
(see 3, above). Do I just grab an arbitrary nightly snapshot  and
replace my lisp/gnus folder? Or will this introduce more
"instabilities" than it is worth? Or am I better off downloading and
compiling a newer emacs? If so, which version will suffice? Is there
an FAQ about this?

Thanks again for your patient replies,
Mat

Re: gnus IMAP vs. Exchange Server 2007

[Vitaly Mayatskikh]

"Mat Marcus" <mat-lists@emarcus.org> writes:

> Thanks for the reply. This variable is just what I need. C-h v
> indicates that it is not present in the version of gnus that I have
> (see 3, above). Do I just grab an arbitrary nightly snapshot  and
> replace my lisp/gnus folder? Or will this introduce more
> "instabilities" than it is worth? Or am I better off downloading and
> compiling a newer emacs? If so, which version will suffice? Is there
> an FAQ about this?

Perhaps, the simplest way for you will be redefining all functions with
imap-enable-exchange-bug-workaround in your ~/.emacs. You have to
redefine these functions (take them from cvs or nightly snapshot):

gnus/nnimap.el:
  nnimap-find-minmax-uid

net/imap.el:
  imap-message-copyuid-1
  imap-message-appenduid-1

-- 
wbr, Vitaly

Re: gnus IMAP vs. Exchange Server 2007

[Reiner Steib]

On Wed, Sep 24 2008, Mat Marcus wrote:

> Vitaly Mayatskikh <v.mayatskih@gmail.com> wrote:
>> Mat Marcus <mat-lists@emarcus.org> writes:
[...]
>>> 3) in either case is there a patch workaround for gnus? I am
>>> currently running gnus "5.11" that comes with emacs 22.1.1.
[...]
>> (setq imap-enable-exchange-bug-workaround t)
>>
>> "Documentation:
>> Send FETCH UID commands as *:* instead of *.
>> Enabling this appears to be required for some servers (e.g.,
>> Microsoft Exchange) which otherwise would trigger a response 'BAD
>> The specified message set is invalid.'."
>
> Thanks for the reply. This variable is just what I need. C-h v
> indicates that it is not present in the version of gnus that I have
> (see 3, above). Do I just grab an arbitrary nightly snapshot  and
> replace my lisp/gnus folder? 

No.

There are some options:

- install the current development version (No Gnus) completely as
  described in the top-level README.

- Insert this code (e.g. in ~/.gnus.el)
  *before* setting `imap-enable-exchange-bug-workaround':

--8<---------------cut here---------------start------------->8---
;; Workaround.  Don't use with Gnus 5.12 or later.

(require 'imap)
(unless (boundp 'imap-enable-exchange-bug-workaround)

  (defvar imap-enable-exchange-bug-workaround nil
    "Send FETCH UID commands as *:* instead of *.
Enabling this appears to be required for some servers (e.g.,
Microsoft Exchange) which otherwise would trigger a response 'BAD
The specified message set is invalid.'.

BACKPORT from No Gnus!")

  (defun imap-message-copyuid-1 (mailbox)
    (if (imap-capability 'UIDPLUS)
	(list (nth 0 (imap-mailbox-get-1 'copyuid mailbox))
	      (string-to-number (nth 2 (imap-mailbox-get-1 'copyuid mailbox))))
      (let ((old-mailbox imap-current-mailbox)
	    (state imap-state)
	    (imap-message-data (make-vector 2 0)))
	(when (imap-mailbox-examine-1 mailbox)
	  (prog1
	      (and (imap-fetch
		    (if imap-enable-exchange-bug-workaround "*:*" "*") "UID")
		   (list (imap-mailbox-get-1 'uidvalidity mailbox)
			 (apply 'max (imap-message-map
				      (lambda (uid prop) uid) 'UID))))
	    (if old-mailbox
		(imap-mailbox-select old-mailbox (eq state 'examine))
	      (imap-mailbox-unselect)))))))

  (defun imap-message-appenduid-1 (mailbox)
    (if (imap-capability 'UIDPLUS)
	(imap-mailbox-get-1 'appenduid mailbox)
      (let ((old-mailbox imap-current-mailbox)
	    (state imap-state)
	    (imap-message-data (make-vector 2 0)))
	(when (imap-mailbox-examine-1 mailbox)
	  (prog1
	      (and (imap-fetch
		    (if imap-enable-exchange-bug-workaround "*:*" "*") "UID")
		   (list (imap-mailbox-get-1 'uidvalidity mailbox)
			 (apply 'max (imap-message-map
				      (lambda (uid prop) uid) 'UID))))
	    (if old-mailbox
		(imap-mailbox-select old-mailbox (eq state 'examine))
	      (imap-mailbox-unselect)))))))

  (message "Workaround `imap-enable-exchange-bug-workaround' added."))

(setq imap-enable-exchange-bug-workaround t)

--8<---------------cut here---------------end--------------->8---

- install the following patch (and recompile imap.el):

--8<---------------cut here---------------start------------->8---
--- imap.el	2 Mar 2008 16:55:00 -0000	7.44
+++ imap.el	2 Apr 2008 14:46:57 -0000	7.45
@@ -440,6 +440,12 @@
 The function should take two arguments, the first the IMAP tag and the
 second the status (OK, NO, BAD etc) of the command.")
 
+(defvar imap-enable-exchange-bug-workaround nil
+  "Send FETCH UID commands as *:* instead of *.
+Enabling this appears to be required for some servers (e.g.,
+Microsoft Exchange) which otherwise would trigger a response 'BAD
+The specified message set is invalid.'.")
+
 \f
 ;; Utility functions:
 
@@ -1774,7 +1780,8 @@
 	  (imap-message-data (make-vector 2 0)))
       (when (imap-mailbox-examine-1 mailbox)
 	(prog1
-	    (and (imap-fetch "*" "UID")
+	    (and (imap-fetch
+		  (if imap-enable-exchange-bug-workaround "*:*" "*") "UID")
 		 (list (imap-mailbox-get-1 'uidvalidity mailbox)
 		       (apply 'max (imap-message-map
 				    (lambda (uid prop) uid) 'UID))))
@@ -1818,7 +1825,8 @@
 	  (imap-message-data (make-vector 2 0)))
       (when (imap-mailbox-examine-1 mailbox)
 	(prog1
-	    (and (imap-fetch "*" "UID")
+	    (and (imap-fetch
+		  (if imap-enable-exchange-bug-workaround "*:*" "*") "UID")
 		 (list (imap-mailbox-get-1 'uidvalidity mailbox)
 		       (apply 'max (imap-message-map
 				    (lambda (uid prop) uid) 'UID))))
--8<---------------cut here---------------end--------------->8---

> Or will this introduce more "instabilities" than it is worth? Or am
> I better off downloading and compiling a newer emacs? If so, which
> version will suffice? 

It's only in the development version of Emacs (to become Emacs 23).

Unfortunately we didn't apply this patch to v5-10 so that it is not
included in the recently released Emacs 22.3.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: gnus IMAP vs. Exchange Server 2007

[Steinar Bang]

>>>>> "Mat Marcus" <mat-lists@emarcus.org>:

> Thanks for the reply. This variable is just what I need. C-h v
> indicates that it is not present in the version of gnus that I have
> (see 3, above). Do I just grab an arbitrary nightly snapshot and
> replace my lisp/gnus folder?

I wouldn't have done that.  I would have unpacked the nightly snapshot,
or perhaps just checked out the current CVS in a separate place, and add
the lisp subdirectory to load-path and the info subdirectory to
Info-directory-list.

Below are the fragments from my ~/.emacs to make the CVS version of Gnus
be the Gnus that emacs uses.  They are a bit complicated, because they
are trying to handle both emacs and XEmacs, and emacs on windows
(ie. they can be simplified a lot).


(defvar windows-emacs (string-match "mingw" (emacs-version))
  "Hold a numerical value if this is an Emacs running on Windows, and
nil if this isn't windows")

(defvar cvs-workspace
  (if windows-emacs
      (expand-file-name "~/workspacecvs/")
    (expand-file-name "~/cvs/"))
   "The place where I check out CVS workspaces.  On linux I like this to 
be lowercase \"cvs\" for shortness.  But on Win32, that name conflicts 
with the \"CVS\" directory in my CVS-versioned home directory, so it has 
to be something different.")

;; CVS version of Gnus:
(let ((cvs-gnus-directory
       (if (string-match "XEmacs" emacs-version)
	   (concat cvs-workspace "gnus.xemacs")
	   (concat cvs-workspace "gnus"))))
  (add-to-list 'load-path (concat cvs-gnus-directory "/lisp"))
  (let ((cvs-gnus-info-dir (concat cvs-gnus-directory "/texi")))
    (if (boundp 'Info-directory-list)
	(add-to-list 'Info-directory-list cvs-gnus-info-dir)
      (setq Info-directory-list (append
				 (list cvs-gnus-info-dir)
				 Info-default-directory-list)))))

Re: gnus IMAP vs. Exchange Server 2007

[Mat Marcus]

Thanks for all the replies. The suggested patches and the strategies
for upgrading my gnus were most educational. I will work on applying
them now.

 - Mat