Imap Problem

Imap Problem

[Jake Colman]

I believe that this has been mentioned several times in the past but I'll do
it again since it's gotton worse.

I use nnimap as my secondary-select-method to access email from an MS
Exchange Server.  Periodcally, but quite often, accessing the server will
simply hang.  I can C-g and the server buffer will list the server as
'denied'.  I can close the connection and reopen but it will hang again.  The
only solution is to exit XEmacs and restart.  This has gotton worse since
I've updated to most of the latest packages and has not been helped by an
upgrade from XEmacs 21.1.12 to XEmacs 21.4.3.

Does anyone else have this problem and can suggest a solution?

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Kai Großjohann]

When the IMAP connection hangs, does it help to kill the ` *nnimap
FOO' buffer?  Or is it named ` *server nnimap FOO'?  (Note the leading
space.) 
kai
-- 
~/.signature: No such file or directory

Re: Imap Problem

[Mats Lidell]

>>>>> Jake wrote:

Jake> Does anyone else have this problem and can suggest a solution?

Yes! You are not alone ;-) No! No solution ;-(

There might be more situations that can lead to this behavior. Such as
the imap connection timing out. What I have found useful is to quite
often check for new mail and by that avoid the timeout.

One other problem, that I have tried to track down, is that nnimap
while connecting looks in the wrong buffer for the reply. Now this
might sound strange but somehow the current buffer is changed. I have
verified this. So nnimap might be looking for the IMAP responce in the
*Group* buffer. 

This happens every time I start gnus in its own window. My workaround
is to start with M-x gnus. So I won't get a new window. My guess is
that you can get similar problems when imap has timedout and it tries
to reconnect.

By looking at the nnimap code I can't however figure out how this can
ever happen and obviously the code does work for a lot so it is a
really hard bug to find. I guess someone who gets this problem must
look into it ;-) OK. Someday ...

Yours
-- 
%% Mats

Re: Imap Problem

[Jake Colman]

>>>>> "ML" == Mats Lidell <Mats.Lidell@contactor.se> writes:

    ML> Yes! You are not alone ;-) No! No solution ;-(

Great!  Misery just loves company...

    ML> There might be more situations that can lead to this behavior. Such
    ML> as the imap connection timing out. What I have found useful is to
    ML> quite often check for new mail and by that avoid the timeout.

Yes, that's exactly what I do.  Unfortunately, if you don't do it quite often
enough you're screwed and have to restart your entire XEmacs session.

    ML> By looking at the nnimap code I can't however figure out how this can
    ML> ever happen and obviously the code does work for a lot so it is a
    ML> really hard bug to find. I guess someone who gets this problem must
    ML> look into it ;-) OK. Someday ...

There are enough of use with this problem that you'd think someone, somewhere
would try to run it down.  I'd try to look into but I have no idea where to
begin.  In case this helps, since upgrading to 21.4.3 and the latest
packages, I can make this happen consistently if I just do the following:

1) Start XEmacs.
2) C-x f .emacs
2) C-x 5 b
3) M-x gnus in the new frame

The only way I can successfully start gnus is if I start gnus from the
initial frame and create my additional frames only after gnus is up and
running.


-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Kai Großjohann]

On 01 Jun 2001, Jake Colman wrote:

> The only way I can successfully start gnus is if I start gnus from
> the initial frame and create my additional frames only after gnus is
> up and running.

What's interesting is that for a few years, I've been doing M-x
kai-mail RET to start Gnus, and I've never had Emacs hang on me (not
because of Gnus, anyway).

(defun kai-mail ()
  (interactive)
  (select-frame
   (or (car
	(filtered-frame-list
	 (lambda (x)
	   (string= "--Mail--"
		    (cdr (assoc 'name (frame-parameters x)))))))
       (make-frame '((name . "--Mail--")))))
  (other-frame 0)
  (if (string-match "^slowfox\\>" (system-name))
      (progn (require 'gnus-agent) (gnus-unplugged))
    (gnus)))

Strange.  Seems to be an XEmacs-only problem.

kai
-- 
~/.signature: No such file or directory

Re: Imap Problem

[Mats Lidell]

>>>>> Jake wrote:

Jake> There are enough of use with this problem that you'd think
Jake> someone, somewhere would try to run it down.  I'd try to look
Jake> into but I have no idea where to begin.

Start by applying this patch to imap.el, or just add the message line,
and verify that you get the same behavior as I do. Then we know it is
the same problem.

*** imap.el	Fri Jun  1 17:02:46 2001
--- imap.el.~1~	Thu Jan  4 06:16:06 2001
***************
*** 1830,1836 ****
  
  (defun imap-parse-greeting ()
    "Parse a IMAP greeting."
-   (message (concat "Looking for IMAP greeting in " (buffer-name))) ;;; XXX
    (cond ((looking-at "\\* OK ")
  	 (setq imap-state 'nonauth))
  	((looking-at "\\* PREAUTH ")
--- 1830,1835 ----

Jake> In case this helps, since upgrading to 21.4.3 and the latest
Jake> packages, I can make this happen consistently if I just do the
Jake> following:

I have had this a long time with different version of XEmacs and it is
present both on NT and on Linux.

Jake> 1) Start XEmacs.
Jake> 2) C-x f .emacs
Jake> 2) C-x 5 b
Jake> 3) M-x gnus in the new frame

With the patch applied my guess is that you will in the Message buffer
se one line that reads: 
  "Looking for IMAP greeting in  *nnimap* <your-server>"
followed by a number of lines: (Depending on how fast to hit C-G to
break the loop)
  "Looking for IMAP greeting in *Group* <your-server>"

What is going on you might ask? (And if I could tell you the whole
store we would probably have solved the question.) 

The short answer is that imap is waiting for the authorisation reply
from the server and when it looks in the process buffer the first time
the reply hasn't arrived yet. So it loops and comes back again only
now to be looking-at in the wrong buffer. Since the *Group* buffer
isn't known to contain an IMAP authorisation message the test will
fail etc.

This is a excerpt from the Message log when I just successfully
managed to connect (by using the M-x gnus-in-a-single-frame-method as
outlined earlier.)
----------------------------------------------------------------------
imap: Connecting to localhost...
Looking for IMAP greeting in  *nnimap* contactor
Looking for IMAP greeting in  *nnimap* contactor
Waiting for response from localhost...done
imap: Connecting to localhost...done
----------------------------------------------------------------------

Notice that this time we had to look twice in the process buffer to
find the reply and that both times we looked in the process
buffer. That's nice and how it should be. 

This also could explain why it works sometimes. With a fast
IMAP-server or a slow XEmacs the reply could arrive so that it is in
the process buffer when we look into it the first time.

OK. To be complete. Here is how it looks when it failes.
----------------------------------------------------------------------
imap: Connecting to localhost...
Looking for IMAP greeting in  *nnimap* contactor
Looking for IMAP greeting in *Group*
Looking for IMAP greeting in *Group*
imap: Connecting to localhost...failed
----------------------------------------------------------------------

This is what I describe above but it is probably easier to understand
when you see it.

Yours
-- 
%% Mats
PS. I really don't know anything about the internal behavior of XEmacs
process buffers so there might be other reasons for what we are
seeing. The consequences remains the same though. DS.

Re: Imap Problem

[Jake Colman]

Nope!  Deleting the ' *nnimap truman' buffer has no effect.  :-(


    KG> When the IMAP connection hangs, does it help to kill the ` *nnimap
    KG> FOO' buffer?  Or is it named ` *server nnimap FOO'?  (Note the
    KG> leading space.)  kai
    KG> --
    KG> ~/.signature: No such file or directory

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Jake Colman]

Kai,

I love that defun.  I just implemented it for myself and it works like a
charm -- except that gnus always hangs!  This would be perfect for me if it
wasn;t for that minor detail.  It is quite consistent, however.  I will
always hang if I use your technique of starting gnus in a new frame.  If I
start it in the initial frame after immediately starting my session, gnus
works fine (that is, until it hangs due to some timeout)

...Jake



    KG> On 01 Jun 2001, Jake Colman wrote:
    >> The only way I can successfully start gnus is if I start gnus from the
    >> initial frame and create my additional frames only after gnus is up
    >> and running.

    KG> What's interesting is that for a few years, I've been doing M-x
    KG> kai-mail RET to start Gnus, and I've never had Emacs hang on me (not
    KG> because of Gnus, anyway).

    KG> (defun kai-mail ()
    KG>   (interactive) (select-frame
    KG>    (or (car
    KG> 	(filtered-frame-list
    KG> 	 (lambda (x)
    KG> 	   (string= "--Mail--"
    KG> 		    (cdr (assoc 'name (frame-parameters x)))))))
    KG>        (make-frame '((name . "--Mail--")))))
    KG>   (other-frame 0) (if (string-match "^slowfox\\>" (system-name))
    KG>       (progn (require 'gnus-agent) (gnus-unplugged))
    KG>     (gnus)))

    KG> Strange.  Seems to be an XEmacs-only problem.

    KG> kai
    KG> --
    KG> ~/.signature: No such file or directory

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Jake Colman]

Mats,

Bingo!

Here's what I get what I connect to 'truman', my MS Exchange IMAP Server:

Opening nnimap server on truman...
imap: Connecting to truman...
Looking for IMAP greeting in  *nnimap* truman
Looking for IMAP greeting in *Group*
Looking for IMAP greeting in *Group*
imap: Connecting to truman...failed

The 'failed' message is after I pressed C-g.

So now that we have a complete diagnosis of the problem, how can it get
fixed?

Is Simon reading this?

...Jake



>>>>> "ML" == Mats Lidell <Mats.Lidell@contactor.se> writes:

>>>>> Jake wrote:
    Jake> There are enough of use with this problem that you'd think someone,
    Jake> somewhere would try to run it down.  I'd try to look into but I
    Jake> have no idea where to begin.

    ML> Start by applying this patch to imap.el, or just add the message
    ML> line, and verify that you get the same behavior as I do. Then we know
    ML> it is the same problem.

    ML> *** imap.el Fri Jun 1 17:02:46 2001
    ML> --- imap.el.~1~ Thu Jan 4 06:16:06 2001
    ML> ***************
    ML> *** 1830,1836 ****
  
    ML>   (defun imap-parse-greeting ()
    ML>     "Parse a IMAP greeting."
    ML> - (message (concat "Looking for IMAP greeting in " (buffer-name)))
    ML>     ;;; XXX
    ML>     (cond ((looking-at "\\* OK ")
    ML>   	 (setq imap-state 'nonauth))
    ML>   	((looking-at "\\* PREAUTH ")
    ML> --- 1830,1835 ----

    Jake> In case this helps, since upgrading to 21.4.3 and the latest
    Jake> packages, I can make this happen consistently if I just do the
    Jake> following:

    ML> I have had this a long time with different version of XEmacs and it
    ML> is present both on NT and on Linux.

    Jake> 1) Start XEmacs.
    Jake> 2) C-x f .emacs
    Jake> 2) C-x 5 b
    Jake> 3) M-x gnus in the new frame

    ML> With the patch applied my guess is that you will in the Message
    ML> buffer se one line that reads:
    ML>   "Looking for IMAP greeting in *nnimap* <your-server>"
    ML> followed by a number of lines: (Depending on how fast to hit C-G to
    ML> break the loop)
    ML>   "Looking for IMAP greeting in *Group* <your-server>"

    ML> What is going on you might ask? (And if I could tell you the whole
    ML> store we would probably have solved the question.)

    ML> The short answer is that imap is waiting for the authorisation reply
    ML> from the server and when it looks in the process buffer the first
    ML> time the reply hasn't arrived yet. So it loops and comes back again
    ML> only now to be looking-at in the wrong buffer. Since the *Group*
    ML> buffer isn't known to contain an IMAP authorisation message the test
    ML> will fail etc.

    ML> This is a excerpt from the Message log when I just successfully
    ML> managed to connect (by using the M-x gnus-in-a-single-frame-method as
    ML> outlined earlier.)
    ML> ----------------------------------------------------------------------
    ML> imap: Connecting to localhost...  Looking for IMAP greeting in
    ML> *nnimap* contactor Looking for IMAP greeting in *nnimap* contactor
    ML> Waiting for response from localhost...done imap: Connecting to
    ML> localhost...done
    ML> ----------------------------------------------------------------------

    ML> Notice that this time we had to look twice in the process buffer to
    ML> find the reply and that both times we looked in the process
    ML> buffer. That's nice and how it should be.

    ML> This also could explain why it works sometimes. With a fast
    ML> IMAP-server or a slow XEmacs the reply could arrive so that it is in
    ML> the process buffer when we look into it the first time.

    ML> OK. To be complete. Here is how it looks when it failes.
    ML> ----------------------------------------------------------------------
    ML> imap: Connecting to localhost...  Looking for IMAP greeting in
    ML> *nnimap* contactor Looking for IMAP greeting in *Group* Looking for
    ML> IMAP greeting in *Group* imap: Connecting to localhost...failed
    ML> ----------------------------------------------------------------------

    ML> This is what I describe above but it is probably easier to understand
    ML> when you see it.

    ML> Yours
    ML> --
    ML> %% Mats
    ML> PS. I really don't know anything about the internal behavior of
    ML> XEmacs process buffers so there might be other reasons for what we
    ML> are seeing. The consequences remains the same though. DS.

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Jake Colman]

Mats,

I agree with your analysis.  I delved into the code and I can also see no
reason why it should it be switching buffers. Since this is so repeatable, I
am hoping that someone will jump on this...

Anyone....?

...Jake


>>>>> "ML" == Mats Lidell <Mats.Lidell@contactor.se> writes:

>>>>> Jake wrote:
    Jake> There are enough of use with this problem that you'd think someone,
    Jake> somewhere would try to run it down.  I'd try to look into but I
    Jake> have no idea where to begin.

    ML> Start by applying this patch to imap.el, or just add the message
    ML> line, and verify that you get the same behavior as I do. Then we know
    ML> it is the same problem.

    ML> *** imap.el Fri Jun 1 17:02:46 2001
    ML> --- imap.el.~1~ Thu Jan 4 06:16:06 2001
    ML> ***************
    ML> *** 1830,1836 ****
  
    ML>   (defun imap-parse-greeting ()
    ML>     "Parse a IMAP greeting."
    ML> - (message (concat "Looking for IMAP greeting in " (buffer-name)))
    ML>     ;;; XXX
    ML>     (cond ((looking-at "\\* OK ")
    ML>   	 (setq imap-state 'nonauth))
    ML>   	((looking-at "\\* PREAUTH ")
    ML> --- 1830,1835 ----

    Jake> In case this helps, since upgrading to 21.4.3 and the latest
    Jake> packages, I can make this happen consistently if I just do the
    Jake> following:

    ML> I have had this a long time with different version of XEmacs and it
    ML> is present both on NT and on Linux.

    Jake> 1) Start XEmacs.
    Jake> 2) C-x f .emacs
    Jake> 2) C-x 5 b
    Jake> 3) M-x gnus in the new frame

    ML> With the patch applied my guess is that you will in the Message
    ML> buffer se one line that reads:
    ML>   "Looking for IMAP greeting in *nnimap* <your-server>"
    ML> followed by a number of lines: (Depending on how fast to hit C-G to
    ML> break the loop)
    ML>   "Looking for IMAP greeting in *Group* <your-server>"

    ML> What is going on you might ask? (And if I could tell you the whole
    ML> store we would probably have solved the question.)

    ML> The short answer is that imap is waiting for the authorisation reply
    ML> from the server and when it looks in the process buffer the first
    ML> time the reply hasn't arrived yet. So it loops and comes back again
    ML> only now to be looking-at in the wrong buffer. Since the *Group*
    ML> buffer isn't known to contain an IMAP authorisation message the test
    ML> will fail etc.

    ML> This is a excerpt from the Message log when I just successfully
    ML> managed to connect (by using the M-x gnus-in-a-single-frame-method as
    ML> outlined earlier.)
    ML> ----------------------------------------------------------------------
    ML> imap: Connecting to localhost...  Looking for IMAP greeting in
    ML> *nnimap* contactor Looking for IMAP greeting in *nnimap* contactor
    ML> Waiting for response from localhost...done imap: Connecting to
    ML> localhost...done
    ML> ----------------------------------------------------------------------

    ML> Notice that this time we had to look twice in the process buffer to
    ML> find the reply and that both times we looked in the process
    ML> buffer. That's nice and how it should be.

    ML> This also could explain why it works sometimes. With a fast
    ML> IMAP-server or a slow XEmacs the reply could arrive so that it is in
    ML> the process buffer when we look into it the first time.

    ML> OK. To be complete. Here is how it looks when it failes.
    ML> ----------------------------------------------------------------------
    ML> imap: Connecting to localhost...  Looking for IMAP greeting in
    ML> *nnimap* contactor Looking for IMAP greeting in *Group* Looking for
    ML> IMAP greeting in *Group* imap: Connecting to localhost...failed
    ML> ----------------------------------------------------------------------

    ML> This is what I describe above but it is probably easier to understand
    ML> when you see it.

    ML> Yours
    ML> --
    ML> %% Mats
    ML> PS. I really don't know anything about the internal behavior of
    ML> XEmacs process buffers so there might be other reasons for what we
    ML> are seeing. The consequences remains the same though. DS.


-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Simon Josefsson]

Mats Lidell <Mats.Lidell@contactor.se> writes:

> OK. To be complete. Here is how it looks when it failes.

Thanks (to both of you) for your very detailed bug report!

A guess would be that this is the same bug as nntp.el has been
including a workaround for:

	    (nntp-accept-response)
	    ;; On some Emacs versions the preceding function has a
	    ;; tendency to change the buffer.  Perhaps.  It's quite
	    ;; difficult to reproduce, because it only seems to happen
	    ;; once in a blue moon.

It might be `accept-process-output' causing this.  Does the attached
patch do anything useful?

Index: imap.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/imap.el,v
retrieving revision 6.6
diff -u -u -w -r6.6 imap.el
--- imap.el	2001/04/16 00:00:45	6.6
+++ imap.el	2001/05/31 23:21:52
@@ -480,6 +480,7 @@
 				       "^\\(Authenticat.*\\)" nil t))
 				  (setq response (match-string 1)))))
 	      (accept-process-output process 1)
+	      (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
 	      (sit-for 1))
 	    (and imap-log
 		 (with-current-buffer (get-buffer-create imap-log)
@@ -539,6 +540,7 @@
 				   "^\\(Authenticat.*\\)" nil t)
 				  (setq response (match-string 1)))))
 	      (accept-process-output process 1)
+	      (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
 	      (sit-for 1))
 	    (and imap-log
 		 (with-current-buffer (get-buffer-create imap-log)
@@ -586,6 +588,7 @@
 			(forward-line -1)
 			(not (imap-parse-greeting)))
 	      (accept-process-output process 1)
+	      (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
 	      (sit-for 1))
 	    (and imap-log
 		 (with-current-buffer (get-buffer-create imap-log)
@@ -616,6 +619,7 @@
 		  (goto-char (point-min))
 		  (not (imap-parse-greeting)))
 	(accept-process-output process 1)
+	(set-buffer buffer) ;; XXX "blue moon" nntp.el bug
 	(sit-for 1))
       (and imap-log
 	   (with-current-buffer (get-buffer-create imap-log)
@@ -652,6 +656,7 @@
 		      (goto-char (point-min))
 		      (not (imap-parse-greeting)))
 	    (accept-process-output process 1)
+	    (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
 	    (sit-for 1))
 	  (and imap-log
 	       (with-current-buffer (get-buffer-create imap-log)
@@ -689,6 +694,7 @@
 		  (goto-char (point-min))
 		  (not (imap-parse-greeting)))
 	(accept-process-output process 1)
+	(set-buffer buffer) ;; XXX "blue moon" nntp.el bug
 	(sit-for 1))
       (and imap-log
 	   (with-current-buffer (get-buffer-create imap-log)

Re: Imap Problem

[Mats Lidell]

>>>>> Simon wrote:

Simon> It might be `accept-process-output' causing this.  Does the
Simon> attached patch do anything useful?

Well... something. This time it looked like this.

----------------------------------------------------------------------
imap: Connecting to localhost...
Looking for greeting in  *nnimap* contactor
Looking for greeting in *Group*
Looking for greeting in *Group*
Looking for greeting in *Group*
Looking for greeting in *Group*
Looking for greeting in *Group*
Looking for greeting in *Group*
Looking for greeting in  *nnimap* contactor
Waiting for response from localhost...done
----------------------------------------------------------------------

It found the right buffer first after I selected the frame by clicking
in it with the mouse. I tried it a couple of times. It turned out the
same every time. (Quite interesting don't you think.)

But the patch gave a hint. If the current buffer is changed without us
knowing it why not set the buffer back again. Simon tried it but why
not set it just before calling imap-parse-greeting so we know we are
looking at the right thing. This lead to the following patch.

*** imap.el	Sat Jun  2 20:59:18 2001
--- imap.el.~32~	Sat Jun  2 19:24:15 2001
***************
*** 616,622 ****
  	 (process (open-network-stream name buffer server port)))
      (when process
        (while (and (memq (process-status process) '(open run))
- 		  (set-buffer buffer) ;; XXX "blue moon" nntp.el bug
  		  (goto-char (point-min))
  		  (not (imap-parse-greeting)))
  	(accept-process-output process 1)
--- 616,621 ----
***************
*** 1842,1848 ****
  
  (defun imap-parse-greeting ()
    "Parse a IMAP greeting."
-   (message (concat "Looking for greeting in " (buffer-name)))
    (cond ((looking-at "\\* OK ")
  	 (setq imap-state 'nonauth))
  	((looking-at "\\* PREAUTH ")
--- 1841,1846 ----

... and the right respons to!
----------------------------------------------------------------------
imap: Connecting to localhost...
Looking for greeting in  *nnimap* contactor
Looking for greeting in  *nnimap* contactor
Waiting for response from localhost...done
----------------------------------------------------------------------

I tried it a couple of times and worked every time. Both starting gnus
in new frame and starting gnus in existing frame with more frames
around worked.

Please note that I have only tried it _together_ with Simons patch and
that it only affects imap-network-open.

If you feel adventurous you might want to look into the affairs of
process-status. ;-)

Yours
-- 
%% Mats

Re: Imap Problem

[Mats Lidell]

>>>>> I wrote:

I> Please note that I have only tried it _together_ with Simons patch
I> and that it only affects imap-network-open.

Well I tried my fix twice _without_ Simons patch and it actually workes
fine.

Yours
-- 
%% Mats

Re: Imap Problem

[Simon Josefsson]

Mats Lidell <Mats.Lidell@contactor.se> writes:

> But the patch gave a hint. If the current buffer is changed without us
> knowing it why not set the buffer back again. Simon tried it but why
> not set it just before calling imap-parse-greeting so we know we are
> looking at the right thing. This lead to the following patch.

Thanks!  I modified all streams similarily and committed it.

> If you feel adventurous you might want to look into the affairs of
> process-status. ;-)

I think it's actually `accept-process-output' that changes the buffer
-- this is why the buffer is correct the first time but changes after
the while loop body has run once.  The code 

Anyway it's an XEmacs bug, but I'm not able to reproduce it so it's
hard to debug.  Maybe you could test the following patch with the old
Gnus imap.el and see if it does anything.

Index: event-stream.c
===================================================================
RCS file: /usr/CVSroot/XEmacs/xemacs/src/event-stream.c,v
retrieving revision 1.55
diff -u -u -w -r1.55 event-stream.c
--- event-stream.c	2001/05/31 12:45:36	1.55
+++ event-stream.c	2001/06/02 21:03:58
@@ -2526,7 +2526,7 @@
   int timeout_id = -1;
   int timeout_enabled = 0;
   int done = 0;
-  struct buffer *old_buffer = current_buffer;
+  /*  struct buffer *old_buffer = current_buffer;*/
   int count;
 
   /* We preserve the current buffer but nothing else.  If a focus
@@ -2637,7 +2637,7 @@
 
   Fdeallocate_event (event);
   UNGCPRO;
-  current_buffer = old_buffer;
+  /*  current_buffer = old_buffer;*/
   return result;
 }
 

Re: Imap Problem

[Jake Colman]

Simon,

Does your commit take into account the fact that Matt's patch works even
without your original patch?  Do I just need to apply Matt's patch?  Or, even
better, I suppose I can just do a cvs update of pgnus.

...Jake


>>>>> "SJ" == Simon Josefsson <simon@josefsson.org> writes:

    SJ> Mats Lidell <Mats.Lidell@contactor.se> writes:
    >> But the patch gave a hint. If the current buffer is changed without us
    >> knowing it why not set the buffer back again. Simon tried it but why
    >> not set it just before calling imap-parse-greeting so we know we are
    >> looking at the right thing. This lead to the following patch.

    SJ> Thanks!  I modified all streams similarily and committed it.

    >> If you feel adventurous you might want to look into the affairs of
    >> process-status. ;-)

    SJ> I think it's actually `accept-process-output' that changes the buffer
    SJ> -- this is why the buffer is correct the first time but changes after
    SJ> the while loop body has run once.  The code

    SJ> Anyway it's an XEmacs bug, but I'm not able to reproduce it so it's
    SJ> hard to debug.  Maybe you could test the following patch with the old
    SJ> Gnus imap.el and see if it does anything.

    SJ> Index: event-stream.c
    SJ> ===================================================================
    SJ> RCS file: /usr/CVSroot/XEmacs/xemacs/src/event-stream.c,v retrieving
    SJ> revision 1.55 diff -u -u -w -r1.55 event-stream.c
    SJ> --- event-stream.c 2001/05/31 12:45:36 1.55
    SJ> +++ event-stream.c 2001/06/02 21:03:58
    SJ> @@ -2526,7 +2526,7 @@
    SJ>    int timeout_id = -1; int timeout_enabled = 0; int done = 0;
    SJ> - struct buffer *old_buffer = current_buffer;
    SJ> + /* struct buffer *old_buffer = current_buffer;*/
    SJ>    int count;
 
    SJ>    /* We preserve the current buffer but nothing else.  If a focus
    SJ> @@ -2637,7 +2637,7 @@
 
    SJ>    Fdeallocate_event (event); UNGCPRO;
    SJ> - current_buffer = old_buffer;
    SJ> + /* current_buffer = old_buffer;*/
    SJ>    return result;
    SJ>  }
 


-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Jake Colman]

Simon,

I updated pgnus and, of course, this includes your patch.  It's too early to
tell whether this patch fixed the problem since I have to wait for a timeout
to see if it reconnects properly.  I am having a related problem, however,
and I don't know if it's as a result of your patch or not.

If I start gnus in another frame, a sure way of triggering the bug which
you've hopefully patched, the password prompting is broken.  Instead of
accepting my CR as a end-of-line and passing the password onwards, it eats
my CR and displays it as a '.'.  If I start gnus in the my initial frame, the
password prompting works correctly.  Can this be a result of something else I
may have downloaded when I did my cvs update or might this be as a result of
what you did.

What I will probably do now is back out my cvs update, restore the version I
had been using up until yesterday, and manually apply Matt's version of your
patch. If that works, I'll post a bug report with respect to password
handling.


...Jake


>>>>> "SJ" == Simon Josefsson <simon@josefsson.org> writes:

    SJ> Mats Lidell <Mats.Lidell@contactor.se> writes:
    >> But the patch gave a hint. If the current buffer is changed without us
    >> knowing it why not set the buffer back again. Simon tried it but why
    >> not set it just before calling imap-parse-greeting so we know we are
    >> looking at the right thing. This lead to the following patch.

    SJ> Thanks!  I modified all streams similarily and committed it.

    >> If you feel adventurous you might want to look into the affairs of
    >> process-status. ;-)

    SJ> I think it's actually `accept-process-output' that changes the buffer
    SJ> -- this is why the buffer is correct the first time but changes after
    SJ> the while loop body has run once.  The code

    SJ> Anyway it's an XEmacs bug, but I'm not able to reproduce it so it's
    SJ> hard to debug.  Maybe you could test the following patch with the old
    SJ> Gnus imap.el and see if it does anything.

    SJ> Index: event-stream.c
    SJ> ===================================================================
    SJ> RCS file: /usr/CVSroot/XEmacs/xemacs/src/event-stream.c,v retrieving
    SJ> revision 1.55 diff -u -u -w -r1.55 event-stream.c
    SJ> --- event-stream.c 2001/05/31 12:45:36 1.55
    SJ> +++ event-stream.c 2001/06/02 21:03:58
    SJ> @@ -2526,7 +2526,7 @@
    SJ>    int timeout_id = -1; int timeout_enabled = 0; int done = 0;
    SJ> - struct buffer *old_buffer = current_buffer;
    SJ> + /* struct buffer *old_buffer = current_buffer;*/
    SJ>    int count;
 
    SJ>    /* We preserve the current buffer but nothing else.  If a focus
    SJ> @@ -2637,7 +2637,7 @@
 
    SJ>    Fdeallocate_event (event); UNGCPRO;
    SJ> - current_buffer = old_buffer;
    SJ> + /* current_buffer = old_buffer;*/
    SJ>    return result;
    SJ>  }
 


-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Jake Colman]

Sorry to follow-up my own postings but we have a problem.

I went back to my prior version of pgnus, prior to Simon's commits, and
manually applied on my Matt's one line change to imap.el.  With this patch
applied, I cannot start gnus in its own frame.  if I do, the password
prompting is messed up and it eats my CR instead of processing the password.
So this problem is specifically related to Matt's portion of the patch and
was not caused by whatever else Simon patched.

I'l let my gnus rest over the course of the day and see if it continues to
work.  In the meantime, I am working with latest cvs version of gnus.

...Jake
g

>>>>> "SJ" == Simon Josefsson <simon@josefsson.org> writes:

    SJ> Mats Lidell <Mats.Lidell@contactor.se> writes:
    >> But the patch gave a hint. If the current buffer is changed without us
    >> knowing it why not set the buffer back again. Simon tried it but why
    >> not set it just before calling imap-parse-greeting so we know we are
    >> looking at the right thing. This lead to the following patch.

    SJ> Thanks!  I modified all streams similarily and committed it.

    >> If you feel adventurous you might want to look into the affairs of
    >> process-status. ;-)

    SJ> I think it's actually `accept-process-output' that changes the buffer
    SJ> -- this is why the buffer is correct the first time but changes after
    SJ> the while loop body has run once.  The code

    SJ> Anyway it's an XEmacs bug, but I'm not able to reproduce it so it's
    SJ> hard to debug.  Maybe you could test the following patch with the old
    SJ> Gnus imap.el and see if it does anything.

    SJ> Index: event-stream.c
    SJ> ===================================================================
    SJ> RCS file: /usr/CVSroot/XEmacs/xemacs/src/event-stream.c,v retrieving
    SJ> revision 1.55 diff -u -u -w -r1.55 event-stream.c
    SJ> --- event-stream.c 2001/05/31 12:45:36 1.55
    SJ> +++ event-stream.c 2001/06/02 21:03:58
    SJ> @@ -2526,7 +2526,7 @@
    SJ>    int timeout_id = -1; int timeout_enabled = 0; int done = 0;
    SJ> - struct buffer *old_buffer = current_buffer;
    SJ> + /* struct buffer *old_buffer = current_buffer;*/
    SJ>    int count;
 
    SJ>    /* We preserve the current buffer but nothing else.  If a focus
    SJ> @@ -2637,7 +2637,7 @@
 
    SJ>    Fdeallocate_event (event); UNGCPRO;
    SJ> - current_buffer = old_buffer;
    SJ> + /* current_buffer = old_buffer;*/
    SJ>    return result;
    SJ>  }
 

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Mats Lidell]

>>>>> Jake wrote:

Jake> I went back to my prior version of pgnus, prior to Simon's
Jake> commits, and manually applied on my Matt's one line change to
Jake> imap.el.  With this patch applied, I cannot start gnus in its
Jake> own frame.  if I do, the password prompting is messed up and it
Jake> eats my CR instead of processing the password.  So this problem
Jake> is specifically related to Matt's portion of the patch and was
Jake> not caused by whatever else Simon patched.

OOPS! Quite right. I use .authinfo and with no prompt during startup I
thus didn't get this problem. With no .authinfo I see this problem
too. (at least on NT.) Back to the drawing board!?

Could someone please explain how setting the buffer when looking for
the imap response could possibly affect how passwords are read from
the minibuffer?

Yours
-- 
%% Mats

Re: Imap Problem

[Nevin Kapur]

Mats Lidell <Mats.Lidell@contactor.se> writes:

> Could someone please explain how setting the buffer when looking for
> the imap response could possibly affect how passwords are read from
> the minibuffer?

FWIW, I see the same problem when prompted for to enter my GPG
passphrase. I don't use IMAP. When this happens (like right now), I
can reproduce the problem by evaluating

(read-passwd "Pass:")

in the *scratch* buffer, but I can't reproduce the situation that
brings it about. I brought this up on the ding list and
comp.emacs.xemacs but didn't get any solutions. I suspect this is an
XEmacs 21.4.x bug.

-- 
Nevin

Re: Imap Problem

[Jake Colman]

This is not a 21.4.3 problem.  I can reproduce it exactly the same way using
21.1.12.


    NK> Mats Lidell <Mats.Lidell@contactor.se> writes:
    >> Could someone please explain how setting the buffer when looking for
    >> the imap response could possibly affect how passwords are read from
    >> the minibuffer?

    NK> FWIW, I see the same problem when prompted for to enter my GPG
    NK> passphrase. I don't use IMAP. When this happens (like right now), I
    NK> can reproduce the problem by evaluating

    NK> (read-passwd "Pass:")

    NK> in the *scratch* buffer, but I can't reproduce the situation that
    NK> brings it about. I brought this up on the ding list and
    NK> comp.emacs.xemacs but didn't get any solutions. I suspect this is an
    NK> XEmacs 21.4.x bug.

    NK> --
    NK> Nevin

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Simon Josefsson]

Jake Colman <colman@ppllc.com> writes:

> Does your commit take into account the fact that Matt's patch works even
> without your original patch?  Do I just need to apply Matt's patch?  Or, even
> better, I suppose I can just do a cvs update of pgnus.

A cvs update should work, with no patches applied.

Re: Imap Problem

[Simon Josefsson]

Jake Colman <colman@ppllc.com> writes:

> I went back to my prior version of pgnus, prior to Simon's commits, and
> manually applied on my Matt's one line change to imap.el.  With this patch
> applied, I cannot start gnus in its own frame.  if I do, the password
> prompting is messed up and it eats my CR instead of processing the password.
> So this problem is specifically related to Matt's portion of the patch and
> was not caused by whatever else Simon patched.

Does the problem also disappear if you remove the patch?  (Just to
single out any other factors.)

I suspect something else is wrong, maybe a `read-passwd' bug, since
the password has always been read from within the process buffer, and
adding an extra `set-buffer' to change to that buffer shouldn't harm
anything.  (Famous last words.)

Re: Imap Problem

[Simon Josefsson]

Jake Colman <colman@ppllc.com> writes:

> This is not a 21.4.3 problem.  I can reproduce it exactly the same way using
> 21.1.12.

Maybe a XEmacs+Windows problem then?  `read-passwd' seem to do all
sorts of weird system specific stuff, grabbing the keyboard and such.

Re: Imap Problem

[Jake Colman]

>>>>> "SJ" == Simon Josefsson <jas@extundo.com> writes:

    SJ> Jake Colman <colman@ppllc.com> writes:
    >> This is not a 21.4.3 problem.  I can reproduce it exactly the same way
    >> using 21.1.12.

    SJ> Maybe a XEmacs+Windows problem then?  `read-passwd' seem to do all
    SJ> sorts of weird system specific stuff, grabbing the keyboard and such.


Simon,

I'm running XEmacs under Solaris 7 and Mat has the same problem with XEmacs
under WinNT.  Are you not seeing the same problem?

...Jake



-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Jake Colman]

>>>>> "SJ" == Simon Josefsson <jas@extundo.com> writes:

    SJ> Jake Colman <colman@ppllc.com> writes:
    >> I went back to my prior version of pgnus, prior to Simon's commits,
    >> and manually applied on my Matt's one line change to imap.el.  With
    >> this patch applied, I cannot start gnus in its own frame.  if I do,
    >> the password prompting is messed up and it eats my CR instead of
    >> processing the password.  So this problem is specifically related to
    >> Matt's portion of the patch and was not caused by whatever else Simon
    >> patched.

    SJ> Does the problem also disappear if you remove the patch?  (Just to
    SJ> single out any other factors.)

    SJ> I suspect something else is wrong, maybe a `read-passwd' bug, since
    SJ> the password has always been read from within the process buffer, and
    SJ> adding an extra `set-buffer' to change to that buffer shouldn't harm
    SJ> anything.  (Famous last words.)

If I remove the patch, I cannot start up gnus in another frame anyway since I
it will hang!  That's what the patch is for!!

-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Simon Josefsson]

Jake Colman <colman@ppllc.com> writes:

>    >> This is not a 21.4.3 problem.  I can reproduce it exactly the same way
>    >> using 21.1.12.
> 
>    SJ> Maybe a XEmacs+Windows problem then?  `read-passwd' seem to do all
>    SJ> sorts of weird system specific stuff, grabbing the keyboard and such.
> 
> I'm running XEmacs under Solaris 7 and Mat has the same problem with XEmacs
> under WinNT.  Are you not seeing the same problem?

No.  XEmacs 21.4, RedHat 7.1.  But I wasn't able to reproduce the
original problem either, so I'm probably doing something wrong.
Simply clicking on the toolbar Gnus icon (thus starting Gnus in
another frame) trigger the bug?

Re: Imap Problem

[Jake Colman]

>>>>> "SJ" == Simon Josefsson <jas@extundo.com> writes:

    SJ> Jake Colman <colman@ppllc.com> writes:
    >> >> This is not a 21.4.3 problem.  I can reproduce it exactly the same
    >> >> way using 21.1.12.
    >>
    SJ> Maybe a XEmacs+Windows problem then?  `read-passwd' seem to do all
    SJ> sorts of weird system specific stuff, grabbing the keyboard and such.
    >>
    >> I'm running XEmacs under Solaris 7 and Mat has the same problem with
    >> XEmacs under WinNT.  Are you not seeing the same problem?

    SJ> No.  XEmacs 21.4, RedHat 7.1.  But I wasn't able to reproduce the
    SJ> original problem either, so I'm probably doing something wrong.
    SJ> Simply clicking on the toolbar Gnus icon (thus starting Gnus in
    SJ> another frame) trigger the bug?

Starting gnus via the following defuun will trigger the problem:

(defun jnc-gnus ()
  (interactive)
  (select-frame
   (or (car
	(filtered-frame-list
	 (lambda (x)
	   (string= "--Mail--"
		    (cdr (assoc 'name (frame-parameters x)))))))
       (make-frame '((name . "--Mail--")))))
  (other-frame 0)
  (if (string-match "^slowfox\\>" (system-name))
      (progn (require 'gnus-agent) (gnus-unplugged))
    (gnus)))

Alternatively, simply create a new frame via 'C-x 5 b' and then start gnus in
that new frame.

With 21.4.3 under Solaris 7 I will consistently hang with the symptom that
has now been corrected via your patch.  However, password handling will not
work.  If I start gnus in the intial frame, password handling will work fine
and the original problem (using latest ognus with your patches) works fine
too.  And this is true even of I move gnus into its own frame after it is up
and running.


-- 
Jake Colman                     

Principia Partners LLC                  Phone: (201) 946-0300
Harborside Financial Center               Fax: (201) 946-0320
902 Plaza II                           Beeper: (800) 928-4640
Jersey City, NJ 07311                  E-mail: colman@ppllc.com
                                       E-mail: jcolman@jnc.com
                                          web: http://www.ppllc.com

Re: Imap Problem

[Mats Lidell]

>>>>> Simon wrote:

Simon> No.  XEmacs 21.4, RedHat 7.1.  But I wasn't able to reproduce the
Simon> original problem either, so I'm probably doing something wrong.

Or right ;-) 

I'm having the problem with XEmacs "21.5 (beta1) \"anise\" XEmacs
Lucid" and gnus-version "Oort Gnus v0.04". RedHat 7.1. I have it on NT
XEmacs 21.4.3(?) with whatever version of GNUS that comes with the
network setup. (I'm on the Linux-box now hence the question mark.) I
have had the problem for a while to. I really don't think the version
of XEmacs matters that much maybe because the bug is in code that
hasn't changed thru the versions?

Simon> Simply clicking on the toolbar Gnus icon (thus starting Gnus in
Simon> another frame) trigger the bug?

I get it by doing this yes!

(Current status: With the patch and .autinfo I'm not able to reproduce
the problem. Without .authinfo I get the password problem.)

Yours
-- 
%% Mats

imap problem

[walter.franzini-M4bTUdDEyImonA0d6jMUrA]

I've recently (~10 days) updated gnus to latest cvs version and since
this event gnus has started to behave in a strange way.

Sometime while getting new mail from an imap server it seems
to enter an endless loop and eating too much memory (>= 150MB).  This
also happen while saving a message in an imap folder.  Sometime C-g is
enough to stop the loop, sometime 'kill -9' is the only solution.

Can someone help me to solve my problem?  Thanks.
-- 
Walter Franzini