Gnus: Buffer has a running process; kill it? (y or n)

Gnus: Buffer has a running process; kill it? (y or n)

[白い熊]

Sent this to emacs-devel list by mistake originally, no worder no reactions :O)

When I run the current CVS version of Emacs and connect to my IMAP mail server 
with Gnus, I get a message on startup:

Buffer has a running process; kill it? (y or n) 

When answering y, for every Gnus instance you run you end up with one hanging 
process, and when exiting Emacs after
closing Gnus you get the prompt:

Active processes exist; kill them and exit anyway? (y or n) 

and *Process List* shows:

imap<1> run      (Killed) gnutls-cli -s xxxx.xxxx.xxx -p 143

one line for each process.

*Messages* shows:

imap: Connecting to xxxx.xxxx.xxx...
Waiting for response from xxxx.xxxx.xxx...done
imap: Reconnecting with stream `starttls'...
Opening STARTTLS connection to `xxxx.xxxx.xxx:143'...
Buffer has a running process; kill it? (y or n) 
imap: Reconnecting with stream `starttls'...failed
imap: Connecting to xxxx.xxxx.xxx...done

If answering n to the prompt to kill the active process, everything is the 
same, just upon Emacs exit *Process List* shows:

imap<1> run      *temp* gnutls-cli -s xxxx.xxxx.xxx -p 143

for each hanging process of each Gnus instance started in the Emacs session.

When running Ubuntu's emacs-snapshot, which is of version 20090320, *Messages* 
shows the following:

imap: Connecting to xxxx.xxxx.xxx...
Waiting for response from xxxx.xxxx.xxx...done
imap: Reconnecting with stream `starttls'...
Opening STARTTLS connection to `xxxx.xxxx.xxx:143'...
imap: Reconnecting with stream `starttls'...failed
imap: Connecting to xxxx.xxxx.xxx...done

but no message on startup and there is no process hanging on exit. So something 
is also failing, but no processes
hanging remain.

How can I fix this?

Best regards,

白い熊

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Tassilo Horn]

"白い熊" <ding_gnus.org@sumou.com> writes:

> When I run the current CVS version of Emacs and connect to my IMAP
> mail server with Gnus, I get a message on startup:
>
> Buffer has a running process; kill it? (y or n) 
>
> When answering y, for every Gnus instance you run you end up with one
> hanging process, and when exiting Emacs after closing Gnus you get the
> prompt:

I think I don't quite understand what you are doing.  Do you run several
instances of emacs, and start gnus in several of them connecting to the
same IMAP server?

If that's the case, then I think you should change the way you use
Gnus, i.e. start it only once in one single emacs instance.

If you really feel the need to have Gnus open in more than one emacs
instance, then you should start the second one as a slave.  Have a look
at the manual.

,----[ (info "(gnus)Slave Gnusae") ]
| You might want to run more than one Emacs with more than one Gnus at
| the same time.  If you are using different `.newsrc' files (e.g., if
| you are using the two different Gnusae to read from two different
| servers), that is no problem whatsoever.  You just do it.
| 
|    The problem appears when you want to run two Gnusae that use the
| same `.newsrc' file.
`----

When you start gnus several time because you configured lets say your
browser to start emacs and gnus when clicking on mailto:-links, then you
should use the emacs server, so that the gnus running in the existing
emacs process is used.  Have a look at the emacs manual.

,----[ (info "(emacs)Emacs Server") ]
| Various programs such as `mail' can invoke your choice of editor to
| edit a particular piece of text, such as a message that you are
| sending.  By convention, most of these programs use the environment
| variable `EDITOR' to specify which editor to run.  If you set `EDITOR'
| to `emacs', they invoke Emacs--but in an inconvenient way, by starting
| a new Emacs process.  This is inconvenient because the new Emacs
| process doesn't share buffers, a command history, or other kinds of
| information with any existing Emacs process.
`----

HTH,
Tassilo

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Steinar Bang]

>>>>> Tassilo Horn <tassilo@member.fsf.org>:

> I think I don't quite understand what you are doing.  Do you run several
> instances of emacs, and start gnus in several of them connecting to the
> same IMAP server?

> If that's the case, then I think you should change the way you use
> Gnus, i.e. start it only once in one single emacs instance.

> If you really feel the need to have Gnus open in more than one emacs
> instance, then you should start the second one as a slave.  Have a look
> at the manual.

Note: starting gnus-slave in the other emacsen works perfectly well for
the above use case.  No imap collisons that I have experienced.

I've been running a gnus-slave to do Gmane spam report approval in, for
years.  And have had no collisions with the regular Gnus process.

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Vegard Vesterheim]

On Wed, 14 Oct 2009 08:57:00 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote:

> If you really feel the need to have Gnus open in more than one emacs
> instance, then you should start the second one as a slave.  Have a look
> at the manual.
>
> ,----[ (info "(gnus)Slave Gnusae") ]
> | You might want to run more than one Emacs with more than one Gnus at
> | the same time.  If you are using different `.newsrc' files (e.g., if
> | you are using the two different Gnusae to read from two different
> | servers), that is no problem whatsoever.  You just do it.
> | 
> |    The problem appears when you want to run two Gnusae that use the
> | same `.newsrc' file.
> `----

Hmm, maybe this could finally solve my frustration with using
Gnus/IMAP on several different computers (at work, home, on travel
etc). The problem is of course that Gnus displays incorrect unread
counts whenever I move from one emacs instance to another.

I could configure all my Gnusae to store its newsrc files with tramp
(possibly over IMAP), and configure one of them to be master and the
others to be slaves.

Would this work without having to restart Gnus whenever I want to read
mail?

 - Vegard V -

newsrc stored with Tramp or IMAP (was: Gnus: Buffer has a running process; kill it? (y or n))

[Ted Zlatanov]

On Wed, 14 Oct 2009 10:16:34 +0200 Vegard Vesterheim <vegard.vesterheim@uninett.no> wrote: 

VV> I could configure all my Gnusae to store its newsrc files with tramp
VV> (possibly over IMAP), and configure one of them to be master and the
VV> others to be slaves.

VV> Would this work without having to restart Gnus whenever I want to read
VV> mail?

No, Gnus wouldn't know that the other instance has modified the newsrc
file, and it wouldn't be able to easily synchronize changes with the
modified version.

The Tramp solution is good for cases where the file is small and
self-contained.  Unfortunately the newsrc file has too many dependencies
on things in memory.

I think the proper solution will be to allow saving and reading the
newsrc file in a more structured way, by group name.  Basically just
gnus-newsrc-alist needs to be broken up.  All the other variables
(gnus-topic-topology, gnus-topic-alist, gnus-server-alist, etc.) can be
stuffed into a single entry.  gnus-newsrc-file-version can be used to
indicate this more structured newsrc file.

With the imap-hash library I uploaded recently, this is pretty easy.
You'd have one entry per group name, plus one "global" entry.  To see if
the entry has been updated, you'd check the message date and compare it
to your own (so you'd need to track the last update of a group from the
newsrc, similar to how gnus-newsrc-last-checked-date works).  Merging
marks and counts per group is much easier than merging globally.

This work can obviously be generalized to use any storage mechanism,
from a remote RDBMS to a non-relational database like CouchDB or
Cassandra to Amazon's S3.  Furthermore, other Gnus structured data can
use it: score files, splitting rules, and the registry.  So I think it's
worth pursuing.  On my own, I'd probably get to it sometime next year
but if someone else is interested please feel free...

Ted

Re: Gnus: Buffer has a running process; kill it? (y or n)

[白い熊]

From: Tassilo Horn <tassilo <at> member.fsf.org>
Date: 2009-10-14 06:57:00 GMT (18 hours and 46 minutes ago)

>I think I don't quite understand what you are doing.  Do you run several
>instances of emacs, and start gnus in several of them connecting to the
>same IMAP server?

Hi, thanks for the input. However this is not the case. This happens every time I use gnus, from a single instance on a
single computer. I mention the number of sessions, as that happens when I start Gnus, then exit it, then from the same
Emacs session start Gnus again, then exit again, the running-trailing sessions accumulate.

The relevant part of ~/.emacs, of how I have the IMAP access set up:

;; Gnus with IMAP over ssh
(setq gnus-select-method '(nnimap "xxxx.xxxx.xxx" (nnimap-address "xxxx.xxxx.xxx")))
;; Tree view for groups
(add-hook 'gnus-group-mode-hook 'gnus-topic-mode)
;;Threads
(setq gnus-summary-thread-gathering-function 'gnus-gather-threads-by-subject)
;; Change email address for work folder.  This is one of the most interesting features of Gnus.  I plan on adding custom
.sigs soon for different mailing lists.
(setq gnus-posting-styles
      '((".*"
         (name "xxxxxxxxxxxxxxxx")
	 ("Gcc" "INBOX")
         (address "xxxxxxxxxxxxxxxxx"))))
(setq gnus-message-archive-method
      '(nnfolder "archive"
		 (nnfolder-inhibit-expiry t)
		 (nnfolder-directory "~/News/archive")
		 (nnfolder-active-file "~/News/archive/active")))
;; To display "Name" <email address>
(setq message-from-style 'angles)
;; To sort reverse-chronologically
(setq gnus-thread-sort-functions 'gnus-thread-sort-by-most-recent-date)
;; Caching
(setq gnus-use-cache t)
(setq gnus-cache-enter-articles '(read ticked dormant))
(setq gnus-cache-remove-articles '(unread))
;; To delete files also from the server, so that they are not kept
;; in the cache and stick around forever, the following
;; workaround is necessary.
;; This will delete the expired/deleted emails and remove them
;; from the local cache. The items will be deleted from the IMAP
;; server and locally, thus disappear everywhere.
(defun gnus-imap-cache-delete ()
       (interactive)
       (save-excursion
	 (setq gnus-use-cache 'passive)
	 (gnus-summary-mark-as-expirable -1))
       (gnus-cache-remove-article)
       (setq gnus-use-cache t))
(add-hook 'gnus-summary-mode-hook
	  (lambda ()
	    (local-set-key (kbd "E") 'gnus-imap-cache-delete)))
;; Debugging
(setq smtpmail-debug-info t)
(setq smtpmail-debug-verb t)

(setq message-send-mail-function 'smtpmail-send-it
      send-mail-function 'smtpmail-send-it
      smtpmail-default-smtp-server "xxxx.xxxx.xxx"
      smtpmail-smtp-server "xxxx.xxxx.xxx"
      smtpmail-local-domain nil
      smtpmail-debug-info t)
(setq smtpmail-auth-credentials (expand-file-name "xxxxxxxxxxx"))
;; Must be placed after declaring the smtpmail-... variables, otherwise they have no effect
(require 'smtpmail)
;; Has to be set after `(require 'smtpmail)' to set login as the method, as `cram-md5'
(setq smtpmail-auth-supported '(login))

What could be causing the problem?

白い熊

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Tassilo Horn]

"白い熊" <ding_gnus.org@sumou.com> writes:

Hi!

>>I think I don't quite understand what you are doing.  Do you run
>>several instances of emacs, and start gnus in several of them
>>connecting to the same IMAP server?
>
> Hi, thanks for the input. However this is not the case. This happens
> every time I use gnus, from a single instance on a single computer. I
> mention the number of sessions, as that happens when I start Gnus,
> then exit it, then from the same Emacs session start Gnus again, then
> exit again, the running-trailing sessions accumulate.

Strange, normally Gnus should terminate all its runnig processes when
quitting...

> The relevant part of ~/.emacs, of how I have the IMAP access set up:
>
> [...]
>
> What could be causing the problem?

Your settings look all sane, I would say.

Hm, what emacs version are you using?  Maybe some devel snapshot?  Then
I'd try with a stable release...

Bye,
Tassilo

Re: Gnus: Buffer has a running process; kill it? (y or n)

[白い熊]

---------- Original Message -----------
From: Tassilo Horn <tassilo@member.fsf.org>
Sent: Thu, 15 Oct 2009 08:52:38 +0200

> Strange, normally Gnus should terminate all its runnig processes when
> quitting...

Yeah, perhaps however there is the problem of some process conflict upon Gnus startup, as evidenced from the *Message* I
posted. This is ever the case with the stable 22 release, since it also shows the startup conflict, but it doesn't
hickup like the CVS version.

> Hm, what emacs version are you using?  Maybe some devel snapshot?  Then
> I'd try with a stable release...

Latest CVS checkout, as of like two days ago. I'm trying to decipher what's causing the conflict. Any thoughts?

白い熊

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Tassilo Horn]

"白い熊" <ding_gnus.org@sumou.com> writes:

>> Hm, what emacs version are you using?  Maybe some devel snapshot?
>> Then I'd try with a stable release...
>
> Latest CVS checkout, as of like two days ago. I'm trying to decipher
> what's causing the conflict. Any thoughts?

Not really.  I also use emacs 23 from CVS, but I don't suffer from this
problem.  Maybe it's not an emacs problem, but a problem with the
external tool that provides the TLS connection: starttls or gnutls-cli.
Try using `M-x proced' to check if there are processes.  Have a look if
there are some such processes walking around as zombies.  (Maybe you
have to swith to the medium format first with `F medium RET'.)

Bye,
Tassilo

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Bjørn Mork]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 1833 bytes --]

Tassilo Horn <tassilo@member.fsf.org> writes:
> "白い熊" <ding_gnus.org@sumou.com> writes:
>
>>> Hm, what emacs version are you using?  Maybe some devel snapshot?
>>> Then I'd try with a stable release...
>>
>> Latest CVS checkout, as of like two days ago. I'm trying to decipher
>> what's causing the conflict. Any thoughts?
>
> Not really.  I also use emacs 23 from CVS, but I don't suffer from this
> problem.  Maybe it's not an emacs problem, but a problem with the
> external tool that provides the TLS connection: starttls or gnutls-cli.
> Try using `M-x proced' to check if there are processes.  Have a look if
> there are some such processes walking around as zombies.  (Maybe you
> have to swith to the medium format first with `F medium RET'.)

I believe this may be related to a symptom I've been seeing lately (as
in "for a few months/years..."):

My imap connections are tunneled over ssh, and I do therefore also have
an external process for each imap server even though I don't use TLS.  I
often move my laptop between different networks, and sometimes (well,
quite often actually :-) I forget to disconnect before moving it.  This
of course results in dead/hanging ssh connections.

In the past, this was no problem.  I would just do Jj twice to toggle
the agent plugged state off and on.  Possibly with a ctrl+g inbetween
because Gnus would hang on the dead ssh connection? I don't remember.
But the point is that some combination of gnus-agent-toggle-plugged and
keyboard-quit used to kill the external ssh processes and enable Gnus to
reconnect.

Nowadays, this is what happens when I press JJ and then ctrl+g when Gnus
just hangs:

Debugger entered--Lisp error: (quit)
  accept-process-output(#<process imap> 0 100)
  imap-wait-for-tag(160 nil)
  imap-send-command-wait("NOOP" nil)
  byte-code("ÃÄ\b\"‰\x19Å=ƒ\x0f

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Bjørn Mork]

[resending, as ignoring the unprintable byte-code didn't work out too
well - at least not on news.gmane.org]

Tassilo Horn <tassilo@member.fsf.org> writes:
> "白い熊" <ding_gnus.org@sumou.com> writes:
>
>>> Hm, what emacs version are you using?  Maybe some devel snapshot?
>>> Then I'd try with a stable release...
>>
>> Latest CVS checkout, as of like two days ago. I'm trying to decipher
>> what's causing the conflict. Any thoughts?
>
> Not really.  I also use emacs 23 from CVS, but I don't suffer from this
> problem.  Maybe it's not an emacs problem, but a problem with the
> external tool that provides the TLS connection: starttls or gnutls-cli.
> Try using `M-x proced' to check if there are processes.  Have a look if
> there are some such processes walking around as zombies.  (Maybe you
> have to swith to the medium format first with `F medium RET'.)

I believe this may be related to a symptom I've been seeing lately (as
in "for a few months/years..."):

My imap connections are tunneled over ssh, and I do therefore also have
an external process for each imap server even though I don't use TLS.  I
often move my laptop between different networks, and sometimes (well,
quite often actually :-) I forget to disconnect before moving it.  This
of course results in dead/hanging ssh connections.

In the past, this was no problem.  I would just do Jj twice to toggle
the agent plugged state off and on.  Possibly with a ctrl+g inbetween
because Gnus would hang on the dead ssh connection? I don't remember.
But the point is that some combination of gnus-agent-toggle-plugged and
keyboard-quit used to kill the external ssh processes and enable Gnus to
reconnect.

Nowadays, this is what happens when I press JJ and then ctrl+g when Gnus
just hangs:

Debugger entered--Lisp error: (quit)
  accept-process-output(#<process imap> 0 100)
  imap-wait-for-tag(160 nil)
  imap-send-command-wait("NOOP" nil)
  byte-code("...\"...=.......	..)." [buffer status imap-error imap-send-command-wait "NOOP" OK t nil] 4)
  imap-ping-server()
  imap-opened(" *nnimap* imap.telenor.net")
  nnimap-close-server("imap.telenor.net")
  gnus-close-server((nnimap "imap.telenor.net" (nnimap-stream shell) (imap-shell-program ("ssh canardo.mork.no nc %s 143")) (nnimap-need-unselect-to-notice-new-mail t) (nnimap-retrieve-groups-asynchronous nil)))
  gnus-agent-close-connections()
  gnus-agent-toggle-plugged(nil)
  call-interactively(gnus-agent-toggle-plugged nil nil)


And the hanging process persists.  Which makes Gnus unkillable, unable
to reconnect, and somewhat unusable.  

There are only two ways out of this state AFAIK:
 1) kill the external ssh process
 2) close the " *nnimap* server" buffer

Both will terminate the hanging ssh session and enable Gnus to
reconnect.

But both are rather cumbersome, just not enough so that I've bothered to
report this earlier.  Sorry about that.  It was a regression at some
point, but it happened quite a while ago.  Maybe years... I don't
remember and I made no note of it at the time.  Probably thought it was
just temporary and that "someone else" would notice and fix it...

The relevant parts of my ~/.gnus are:

(setq gnus-secondary-select-methods '(
    (nnimap "imap.telenor.net" 
	    (nnimap-stream shell)
	    (imap-shell-program ("ssh canardo.mork.no nc %s 143"))
	    (nnimap-need-unselect-to-notice-new-mail t)
	    (nnimap-retrieve-groups-asynchronous nil) ; workaround for client side splitting problem
    )
    (nnimap "mail.mork.no"
	    (nnimap-stream shell)
	    (imap-shell-program ("ssh canardo.mork.no nc %s 143"))
    ))


I'm currently using emacs 23.1 from Debian unstable.  But this problem
has been there for quite a while, as I said.  It is definitely there
with emacs 22.2 from Debian lenny as well.



Bjørn

Re: Gnus: Buffer has a running process; kill it? (y or n)

[Ted Zlatanov]

On Thu, 15 Oct 2009 17:29:25 +0200 Tassilo Horn <tassilo@member.fsf.org> wrote: 

TH> "白い熊" <ding_gnus.org@sumou.com> writes:
>>> Hm, what emacs version are you using?  Maybe some devel snapshot?
>>> Then I'd try with a stable release...
>> 
>> Latest CVS checkout, as of like two days ago. I'm trying to decipher
>> what's causing the conflict. Any thoughts?

TH> Not really.  I also use emacs 23 from CVS, but I don't suffer from this
TH> problem.  Maybe it's not an emacs problem, but a problem with the
TH> external tool that provides the TLS connection: starttls or gnutls-cli.
TH> Try using `M-x proced' to check if there are processes.  Have a look if
TH> there are some such processes walking around as zombies.  (Maybe you
TH> have to swith to the medium format first with `F medium RET'.)

There's a (perhaps) related bug report on the spam-report-gmane
function, which also keeps the process running.  That's a network ping
over HTTP, though.  I haven't investigated either that bug report or
this one yet (and won't be able to do so soon), but perhaps they are
related and triggered by a similar Emacs 23 issue.

Ted

Re: Gnus: Buffer has a running process; kill it? (y or n)

[白い熊]

---------- Original Message -----------
From: Tassilo Horn <tassilo@member.fsf.org>
Sent: Thu, 15 Oct 2009 08:52:38 +0200

> Hm, what emacs version are you using?  Maybe some devel snapshot?  Then
> I'd try with a stable release...

Oh, BTW, the stable 23 build exhibits the same problem, so it's not a latest CVS related problem.

白い熊