Process imap not running

Process imap not running

[Ralf Angeli]

[Second try, since the first message sent through Gmane hasn't shown up
for two days.]

Hi,

the error related to the following stack trace usually occurs more than
three times a day for me:

Debugger entered--Lisp error: (error "Process imap not running")
  process-send-string(#<process imap> "37 EXAMINE \"INBOX\"
\n")
  imap-send-command-1("37 EXAMINE \"INBOX\"")
  imap-send-command("EXAMINE \"INBOX\"" nil)
  imap-send-command-wait("EXAMINE \"INBOX\"")
  imap-mailbox-unselect(" *nnimap* <name>")
  nnimap-before-find-minmax-bugworkaround()
  nnimap-retrieve-groups(("Gesendet" "INBOX") "<name>")
  gnus-retrieve-groups(("Gesendet" "INBOX") (nnimap "<name>" (nnimap-address "<server>") (nnimap-stream ssl)))
  gnus-read-active-file-2(("Gesendet" "INBOX") (nnimap "<name>" (nnimap-address "<server>") (nnimap-stream ssl)))
  gnus-read-active-file-1((nnimap "<name>" (nnimap-address "<server>") (nnimap-stream ssl)) nil)
  gnus-read-active-file()
  gnus-group-get-new-news(nil)
  call-interactively(gnus-group-get-new-news nil nil)

This happens with a SSL-enabled connection to a remote IMAP server on
Gnus 5.13 in a recent Emacs from the emacs-unicode-2 branch.  The real
server specifics were replaced with "<name>" and "<server>" in the above
stack trace.

I'm not sure if and how this could be fixed.  Perhaps something like
`imap-open' could be called if the process is found to have died?
`imap-open-1' will reset `imap-current-mailbox' and
`imap-current-message', so I am not sure if this could have adverse
directions if this is done when a certain mailbox and folder is selected
during a restart of the process.

-- 
Ralf

Re: Process imap not running

[Knut Anders Hatlen]

Ralf Angeli <angeli@caeruleus.net> writes:

> [Second try, since the first message sent through Gmane hasn't shown up
> for two days.]
>
> Hi,
>
> the error related to the following stack trace usually occurs more than
> three times a day for me:
>
> Debugger entered--Lisp error: (error "Process imap not running")
>   process-send-string(#<process imap> "37 EXAMINE \"INBOX\"
> \n")
>   imap-send-command-1("37 EXAMINE \"INBOX\"")
>   imap-send-command("EXAMINE \"INBOX\"" nil)
>   imap-send-command-wait("EXAMINE \"INBOX\"")
>   imap-mailbox-unselect(" *nnimap* <name>")
>   nnimap-before-find-minmax-bugworkaround()
>   nnimap-retrieve-groups(("Gesendet" "INBOX") "<name>")
>   gnus-retrieve-groups(("Gesendet" "INBOX") (nnimap "<name>" (nnimap-address "<server>") (nnimap-stream ssl)))
>   gnus-read-active-file-2(("Gesendet" "INBOX") (nnimap "<name>" (nnimap-address "<server>") (nnimap-stream ssl)))
>   gnus-read-active-file-1((nnimap "<name>" (nnimap-address "<server>") (nnimap-stream ssl)) nil)
>   gnus-read-active-file()
>   gnus-group-get-new-news(nil)
>   call-interactively(gnus-group-get-new-news nil nil)
>
> This happens with a SSL-enabled connection to a remote IMAP server on
> Gnus 5.13 in a recent Emacs from the emacs-unicode-2 branch.

Hi Ralf,

I used to have the same problem, but I managed to work around it with a
simple fix. See this article for the details:

http://article.gmane.org/gmane.emacs.gnus.general/63692

-- 
Knut Anders

Re: Process imap not running

[Andreas Jaeger]

On Jan 10, 2008 1:19 PM, Knut Anders Hatlen <kahatlen@gmail.com> wrote:
> [...]
> Hi Ralf,
>
> I used to have the same problem, but I managed to work around it with a
> simple fix. See this article for the details:
>
> http://article.gmane.org/gmane.emacs.gnus.general/63692

This works fine for me.  I would love to see this - or a superior
solution - committed to the gnus CVS.

Thanks,
Andreas

Re: Process imap not running

[Ralf Angeli]

* Knut Anders Hatlen (2008-01-10) writes:

> Ralf Angeli <angeli@caeruleus.net> writes:
>
>> the error related to the following stack trace usually occurs more than
>> three times a day for me:
>>
>> Debugger entered--Lisp error: (error "Process imap not running")
>>   process-send-string(#<process imap> "37 EXAMINE \"INBOX\"
[...]
> I used to have the same problem, but I managed to work around it with a
> simple fix. See this article for the details:
>
> http://article.gmane.org/gmane.emacs.gnus.general/63692

Thanks!  I've been using this patch for three days now and haven't got
the error during this time.

-- 
Ralf

Re: Process imap not running

[Reiner Steib]

On Mon, Jan 14 2008, Ralf Angeli wrote:

> * Knut Anders Hatlen (2008-01-10) writes:
>> Ralf Angeli <angeli@caeruleus.net> writes:
>>> Debugger entered--Lisp error: (error "Process imap not running")
>>>   process-send-string(#<process imap> "37 EXAMINE \"INBOX\"
> [...]
>> I used to have the same problem, but I managed to work around it with a
>> simple fix. See this article for the details:
>>
>> http://article.gmane.org/gmane.emacs.gnus.general/63692
[ Date: Tue, 05 Sep 2006 23:06:33 +0200
  Message-ID: <86d5aa81ly.fsf@ugle.hatlen.net> ]

> Thanks!  I've been using this patch for three days now and haven't got
> the error during this time.

Does anyone see a problem with it?  Simon?  (There was no response to
the original message.)

--- lisp/imap.el	26 Jun 2006 12:08:33 -0000	7.29
+++ lisp/imap.el	4 Sep 2006 17:45:39 -0000
@@ -1142,7 +1142,16 @@
        (buffer-live-p buffer)
        (with-current-buffer buffer
 	 (and imap-process
-	      (memq (process-status imap-process) '(open run))))))
+	      (memq (process-status imap-process) '(open run))
+	      (imap-ping-server)))))
+
+(defun imap-ping-server (&optional buffer)
+  "Ping the imap server in BUFFER with a noop command. Return non-nil
+if the server responds, and nil if it does not respond. If BUFFER is
+nil, the current buffer is used."
+  (condition-case ()
+      (imap-ok-p (imap-send-command-wait "NOOP" buffer))
+    (error nil)))

 (defun imap-authenticate (&optional user passwd buffer)
   "Authenticate to server in BUFFER, using current buffer if nil.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Process imap not running

[Simon Josefsson]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Mon, Jan 14 2008, Ralf Angeli wrote:
>
>> * Knut Anders Hatlen (2008-01-10) writes:
>>> Ralf Angeli <angeli@caeruleus.net> writes:
>>>> Debugger entered--Lisp error: (error "Process imap not running")
>>>>   process-send-string(#<process imap> "37 EXAMINE \"INBOX\"
>> [...]
>>> I used to have the same problem, but I managed to work around it with a
>>> simple fix. See this article for the details:
>>>
>>> http://article.gmane.org/gmane.emacs.gnus.general/63692
> [ Date: Tue, 05 Sep 2006 23:06:33 +0200
>   Message-ID: <86d5aa81ly.fsf@ugle.hatlen.net> ]
>
>> Thanks!  I've been using this patch for three days now and haven't got
>> the error during this time.
>
> Does anyone see a problem with it?  Simon?  (There was no response to
> the original message.)

Looks fine to me, please apply.  It adds some unnecessary round-trips,
but I guess we can live with those.  Maybe there could be a variable to
turn them off, but we can always add that later if people want it.

/Simon

> --- lisp/imap.el	26 Jun 2006 12:08:33 -0000	7.29
> +++ lisp/imap.el	4 Sep 2006 17:45:39 -0000
> @@ -1142,7 +1142,16 @@
>         (buffer-live-p buffer)
>         (with-current-buffer buffer
>  	 (and imap-process
> -	      (memq (process-status imap-process) '(open run))))))
> +	      (memq (process-status imap-process) '(open run))
> +	      (imap-ping-server)))))
> +
> +(defun imap-ping-server (&optional buffer)
> +  "Ping the imap server in BUFFER with a noop command. Return non-nil
> +if the server responds, and nil if it does not respond. If BUFFER is
> +nil, the current buffer is used."
> +  (condition-case ()
> +      (imap-ok-p (imap-send-command-wait "NOOP" buffer))
> +    (error nil)))
>
>  (defun imap-authenticate (&optional user passwd buffer)
>    "Authenticate to server in BUFFER, using current buffer if nil.
>
> Bye, Reiner.
> -- 
>        ,,,
>       (o o)
> ---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Process imap not running

[Reiner Steib]

On Mon, Jan 14 2008, Simon Josefsson wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>> On Mon, Jan 14 2008, Ralf Angeli wrote:
>>> * Knut Anders Hatlen (2008-01-10) writes:
[...]
>>>> http://article.gmane.org/gmane.emacs.gnus.general/63692
>> [ Date: Tue, 05 Sep 2006 23:06:33 +0200
>>   Message-ID: <86d5aa81ly.fsf@ugle.hatlen.net> ]
[...]
>> Does anyone see a problem with it?  Simon?  (There was no response to
>> the original message.)
>
> Looks fine to me, please apply.  

Done.

> It adds some unnecessary round-trips, but I guess we can live with
> those.  Maybe there could be a variable to turn them off, but we can
> always add that later if people want it.

I added the variable `imap-ping-server', defaulting to t.  Please
adjust if necessary.

Should the function `imap-ping-server' also add debug output to
`imap-debug-buffer'?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Process imap not running

[Ted Zlatanov]

On Tue, 15 Jan 2008 00:27:48 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Mon, Jan 14 2008, Simon Josefsson wrote:

>> It adds some unnecessary round-trips, but I guess we can live with
>> those.  Maybe there could be a variable to turn them off, but we can
>> always add that later if people want it.

RS> I added the variable `imap-ping-server', defaulting to t.  Please
RS> adjust if necessary.

I think it should be an integer, representing the minimum number of
seconds before it's triggered (or nil to turn it off).

The default should be nil.  When the IMAP server is far away, pinging
the server introduces a painful delay because of the TCP round trip.

The situation is similar with dropped SSH sessions, where OpenSSH allows
for the ServerAliveInterval parameter but doesn't turn it on by default
except when in batch mode.

RS> Should the function `imap-ping-server' also add debug output to
RS> `imap-debug-buffer'?

Yes, please.

Ted

Re: Process imap not running

[Reiner Steib]

On Tue, Jan 15 2008, Ted Zlatanov wrote:

> On Tue, 15 Jan 2008 00:27:48 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
>
> RS> On Mon, Jan 14 2008, Simon Josefsson wrote:
>
>>> It adds some unnecessary round-trips, but I guess we can live with
>>> those.  Maybe there could be a variable to turn them off, but we can
>>> always add that later if people want it.
>
> RS> I added the variable `imap-ping-server', defaulting to t.  Please
> RS> adjust if necessary.
>
> I think it should be an integer, representing the minimum number of
> seconds before it's triggered (or nil to turn it off).

Is this feasible?  Would someone like to implement this?

> The default should be nil.  When the IMAP server is far away, pinging
> the server introduces a painful delay because of the TCP round trip.

Could you explain how much delay it cause (and when)?  As it seems to
fix a problem for several users, it would be nice to set it to t.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Process imap not running

[Ted Zlatanov]

On Wed, 16 Jan 2008 23:50:44 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Tue, Jan 15 2008, Ted Zlatanov wrote:
>> On Tue, 15 Jan 2008 00:27:48 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
>> 
RS> On Mon, Jan 14 2008, Simon Josefsson wrote:
>> 
>>>> It adds some unnecessary round-trips, but I guess we can live with
>>>> those.  Maybe there could be a variable to turn them off, but we can
>>>> always add that later if people want it.
>> 
RS> I added the variable `imap-ping-server', defaulting to t.  Please
RS> adjust if necessary.
>> 
>> I think it should be an integer, representing the minimum number of
>> seconds before it's triggered (or nil to turn it off).

RS> Is this feasible?  Would someone like to implement this?

I could do it eventually, but it's really not too hard.  Just make a
buffer-local timer in the imap buffer and check it every time if the
variable is not t.

>> The default should be nil.  When the IMAP server is far away, pinging
>> the server introduces a painful delay because of the TCP round trip.

RS> Could you explain how much delay it cause (and when)?  As it seems to
RS> fix a problem for several users, it would be nice to set it to t.

Remote TCP connections require an acknowledgment for every packet.  This
is alleviated by some optimizations to TCP itself (for example, you're
allowed to send more packets before you get an acknowledgment to the
first one), but it's a session-based reliable protocol so eventually you
have to wait for a confirmation back to know your packet made it.  If I
understand it correctly, the connection to the IMAP server is
synchronous, so any new commands in the queue will have to wait for the
ping to complete (since we don't start sending new IMAP commands until
the current one completes).  Thus a server 80ms away will introduce a
80ms delay twice per ping.  When you're on a slow link, or the network
is congested/miconfigured, or the server is far away, the delay will be
noticeable.

Of course, if the ping can happen in the background, that would not
block.  But we'd need parallel IMAP sessions or support for multiple
queued commands in one IMAP session, and imap.el doesn't support either
AFAIK.

Ted

Re: Process imap not running

[Reiner Steib]

On Thu, Jan 17 2008, Ted Zlatanov wrote:

> On Wed, 16 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
> RS> On Tue, Jan 15 2008, Ted Zlatanov wrote:
>>> On Tue, 15 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
> RS> I added the variable `imap-ping-server', defaulting to t.  Please
> RS> adjust if necessary.
>>> 
>>> I think it should be an integer, representing the minimum number of
>>> seconds 

... better use milliseconds or allow floats?

>>> before it's triggered (or nil to turn it off).
>
> RS> Is this feasible?  Would someone like to implement this?
>
> I could do it eventually, 

Please do.

> but it's really not too hard.  Just make a buffer-local timer in the
> imap buffer and check it every time if the variable is not t.

>>> The default should be nil.  When the IMAP server is far away, pinging
>>> the server introduces a painful delay because of the TCP round trip.
>
> RS> Could you explain how much delay it cause (and when)?  As it seems to
> RS> fix a problem for several users, it would be nice to set it to t.

... or some reasonable integer?

> Remote TCP connections require an acknowledgment for every packet.  [...]

My question aimed at your personal experience, i.e. which Gnus/IMAP
commands are noticeably slower (getting new articles, displaying
articles, ..., more or less all commands?) and how long the delay is
approximately (several seconds?).

> Of course, if the ping can happen in the background, that would not
> block.  But we'd need parallel IMAP sessions or support for multiple
> queued commands in one IMAP session, and imap.el doesn't support
> either AFAIK.

That would be nicer.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: Process imap not running

[Ted Zlatanov]

On Thu, 17 Jan 2008 19:33:59 +0100 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 

RS> On Thu, Jan 17 2008, Ted Zlatanov wrote:
>> On Wed, 16 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
RS> On Tue, Jan 15 2008, Ted Zlatanov wrote:
>>>> On Tue, 15 Jan 2008 Reiner Steib <reinersteib+gmane@imap.cc> wrote: 
RS> I added the variable `imap-ping-server', defaulting to t.  Please
RS> adjust if necessary.
>>>> 
>>>> I think it should be an integer, representing the minimum number of
>>>> seconds 

RS> ... better use milliseconds or allow floats?

OK.

>>>> before it's triggered (or nil to turn it off).
>> 
RS> Is this feasible?  Would someone like to implement this?
>> 
>> I could do it eventually, 

RS> Please do.

I'll put it on my TODO list.

>> but it's really not too hard.  Just make a buffer-local timer in the
>> imap buffer and check it every time if the variable is not t.

>>>> The default should be nil.  When the IMAP server is far away, pinging
>>>> the server introduces a painful delay because of the TCP round trip.
>> 
RS> Could you explain how much delay it cause (and when)?  As it seems to
RS> fix a problem for several users, it would be nice to set it to t.

RS> ... or some reasonable integer?

>> Remote TCP connections require an acknowledgment for every packet.  [...]

RS> My question aimed at your personal experience, i.e. which Gnus/IMAP
RS> commands are noticeably slower (getting new articles, displaying
RS> articles, ..., more or less all commands?) and how long the delay is
RS> approximately (several seconds?).

(sorry for the discourse)

I haven't noticed it personally, since I use IMAP servers that are
nearby on fast links.  Generally it wouldn't be directly noticeable
since the delay is small per command on the average, but on the whole
the users will have a slower experience, and in the worst case (on a
slow link to a distant server) it will surely be unpleasant.

Ted