nohup dies

nohup dies

[Pau Amaro-Seoane]

Hi,

I have been using zsh for a while and I like it a lot. Yet, today I
ran into a snag...

I use a kind of beowulf cluster to run computer simulations and the
way I have launched them for years has always been

cat ./input | ~pau/executables/NB4_NoPN > diagnostic&

and then I usually shutdown the laptop or close the terminal.

Two years ago I moved to another institute and this has also a
cluster. Recently I restarted doing the same kind of simulations and
somehow I had problems with them all the time

I am in a very bad mood... you cannot imagine how wretched I am; I spent _many_
days trying to find "bugs" in my code and didn't understand at all why the runs
died after "some minutes" without any output message...

The problem? nohup dies in zsh 4.3.2

Today I closed the ssh session with exit and then exit again after the
message and got this:

zsh: you have running jobs.
tuffi1| exit
zsh: warning: 1 jobs SIGHUPed
[1]  + 18591 done       nohup cat ./input |
       18592 hangup     ~pau/executables/NB4_NoPN > diagnostic

When I used the cluster in my former institute everything went fine,
and there they have zsh 4.2.4

So it seems that "deadly" nohups are inherent to the most recent zsh version...
a "new feature" of the shell, maybe?

I changed the shell to ksh and the simulations (i.e. nohup) are not
killed when I quit the session...

What's wrong with zsh 4.2.4?

Cheers,

Pau Amaro Seoane

Re: nohup dies

[wgscott]

Hi Pau:

I've not run into this problem using any version of zsh, so I can't advise, but in general I find it is much more robust to start a screen session, issue the command, and detatch the screen.

You didn't say what your OS is, but OS X and linux come with screen by default, and I think you can install it on anything zsh would run on.

Bill



On Fri, 28 Sep 2007 22:38:48 +0200
"Pau Amaro-Seoane" <vim.unix@googlemail.com> wrote:

     Hi,
     
     I have been using zsh for a while and I like it a lot. Yet, today I
     ran into a snag...
     
     I use a kind of beowulf cluster to run computer simulations and the
     way I have launched them for years has always been
     
     cat ./input | ~pau/executables/NB4_NoPN > diagnostic&
     
     and then I usually shutdown the laptop or close the terminal.
     
     Two years ago I moved to another institute and this has also a
     cluster. Recently I restarted doing the same kind of simulations and
     somehow I had problems with them all the time
     
     I am in a very bad mood... you cannot imagine how wretched I am; I spent _many_
     days trying to find "bugs" in my code and didn't understand at all why the runs
     died after "some minutes" without any output message...
     
     The problem? nohup dies in zsh 4.3.2
     
     Today I closed the ssh session with exit and then exit again after the
     message and got this:
     
     zsh: you have running jobs.
     tuffi1| exit
     zsh: warning: 1 jobs SIGHUPed
     [1]  + 18591 done       nohup cat ./input |
            18592 hangup     ~pau/executables/NB4_NoPN > diagnostic
     
     When I used the cluster in my former institute everything went fine,
     and there they have zsh 4.2.4
     
     So it seems that "deadly" nohups are inherent to the most recent zsh version...
     a "new feature" of the shell, maybe?
     
     I changed the shell to ksh and the simulations (i.e. nohup) are not
     killed when I quit the session...
     
     What's wrong with zsh 4.2.4?
     
     Cheers,
     
     Pau Amaro Seoane
     

Re: nohup dies

[Pau Amaro-Seoane]

Hi Bill,

thanks for the answer.

By the way, I meant

>     nohup cat ./input | ~pau/executables/NB4_NoPN > diagnostic&

when I wrote

>      cat ./input | ~pau/executables/NB4_NoPN > diagnostic&

!

Yes, I am thinking of screen... I used it a lot in the past but I
stopped because it'd screw up my mutt colour scheme and I never could
fix that...

The OS of the cluster is Debian GNU/Linux etch.

If you knew how long I have needed to realise about this! I was
hunting fictitious bugs in my code...

sigh...

thanks,

Pau

2007/9/28, wgscott@chemistry.ucsc.edu <wgscott@chemistry.ucsc.edu>:
> Hi Pau:
>
> I've not run into this problem using any version of zsh, so I can't advise, but in general I find it is much more robust to start a screen session, issue the command, and detatch the screen.
>
> You didn't say what your OS is, but OS X and linux come with screen by default, and I think you can install it on anything zsh would run on.
>
> Bill
>
>
>
> On Fri, 28 Sep 2007 22:38:48 +0200
> "Pau Amaro-Seoane" <vim.unix@googlemail.com> wrote:
>
>      Hi,
>
>      I have been using zsh for a while and I like it a lot. Yet, today I
>      ran into a snag...
>
>      I use a kind of beowulf cluster to run computer simulations and the
>      way I have launched them for years has always been
>
>      cat ./input | ~pau/executables/NB4_NoPN > diagnostic&
>
>      and then I usually shutdown the laptop or close the terminal.
>
>      Two years ago I moved to another institute and this has also a
>      cluster. Recently I restarted doing the same kind of simulations and
>      somehow I had problems with them all the time
>
>      I am in a very bad mood... you cannot imagine how wretched I am; I spent _many_
>      days trying to find "bugs" in my code and didn't understand at all why the runs
>      died after "some minutes" without any output message...
>
>      The problem? nohup dies in zsh 4.3.2
>
>      Today I closed the ssh session with exit and then exit again after the
>      message and got this:
>
>      zsh: you have running jobs.
>      tuffi1| exit
>      zsh: warning: 1 jobs SIGHUPed
>      [1]  + 18591 done       nohup cat ./input |
>             18592 hangup     ~pau/executables/NB4_NoPN > diagnostic
>
>      When I used the cluster in my former institute everything went fine,
>      and there they have zsh 4.2.4
>
>      So it seems that "deadly" nohups are inherent to the most recent zsh version...
>      a "new feature" of the shell, maybe?
>
>      I changed the shell to ksh and the simulations (i.e. nohup) are not
>      killed when I quit the session...
>
>      What's wrong with zsh 4.2.4?
>
>      Cheers,
>
>      Pau Amaro Seoane
>
>

Re: nohup dies

[Peter Stephenson]

"Pau Amaro-Seoane" wrote:
> I have been using zsh for a while and I like it a lot. Yet, today I
> ran into a snag...

There are various issues here...

> The problem? nohup dies in zsh 4.3.2

nohup is not a shell builtin; it's simply an executable programme that's
run by the shell and that in turn runs the command line passed as
arguments.

> Today I closed the ssh session with exit and then exit again after the
> message and got this:
> 
> zsh: you have running jobs.
> tuffi1| exit
> zsh: warning: 1 jobs SIGHUPed
> [1]  + 18591 done       nohup cat ./input |
>        18592 hangup     ~pau/executables/NB4_NoPN > diagnostic

The pipeline is forked before the nohup is executed.  In more detail.

  zsh creates a pipeline
  zsh forks:
     In the subshell, "nohup cat ./input" is run
  zsh forks again
     In the new subshell, ~pau/executables/... is run

So the "nohup" doesn't cover the second executable.

> When I used the cluster in my former institute everything went fine,
> and there they have zsh 4.2.4

Is it possible the option "setopt nohup" was in effect there?  This has
the same effect as "nohup" for all background commands.  (It's not
*quite* the same; the command nohup protects the following command line
from the signal, while the option stops the shell sending the signal
when it exits.)

> I changed the shell to ksh and the simulations (i.e. nohup) are not
> killed when I quit the session...

A lot of other shells, I presume (but I haven't actually checked)
including ksh, don't send SIGHUP to processes when the shell exits.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: nohup dies

[Pau Amaro-Seoane]

Hi,

thanks for the answer

> nohup is not a shell builtin; it's simply an executable programme that's

man nohup

(...)
       NOTE:  your  shell may have its own version of nohup, which
usually supersedes the version described
       here.  Please refer to your shell's documentation for details
about the options it supports.
(...)

>zsh creates a pipeline
> zsh forks:
>    In the subshell, "nohup cat ./input" is run
> zsh forks again
>    In the new subshell, ~pau/executables/... is run

>So the "nohup" doesn't cover the second executable.

can you tell me of a more robust/elegant way of doing it?

>
> A lot of other shells, I presume (but I haven't actually checked)
> including ksh, don't send SIGHUP to processes when the shell exits.

But zsh is supposed to emulate ksh, right?

nohup whatever should work regardless of shell. And it _did_ in the
older version of zsh, now not anymore. Can you tell me why?

the "solution" for now is to launch the simulation with ksh... or
resort to screen... but it's a pity, I stick to zsh a lot

Cheers,

Pau

Re: nohup dies

[Tim Haynes]

"Pau Amaro-Seoane" <vim.unix@googlemail.com> writes:

[snip]
>>zsh creates a pipeline
>> zsh forks:
>>    In the subshell, "nohup cat ./input" is run
>> zsh forks again
>>    In the new subshell, ~pau/executables/... is run
>
>>So the "nohup" doesn't cover the second executable.
>
> can you tell me of a more robust/elegant way of doing it?

Well, let's take your original command:

(nohup) cat ./input | ~pau/executables/NB4_NoPN > diagnostic&

Why not add a nohup before the second executable in the pipe?  :

nohup cat ./input | nohup ~pau/executables/NB4_NoPN > diagnostic&

If you don't like that, then why not wrap the whole thing up in a new
invocation of a shell? :

nohup zsh -c 'cat ./input | nohup ~pau/executables/NB4_NoPN > diagnostic ' &

You could disown the job after it's been spawned, too.

While we're here, it should be observed that if you were disciplined or
pedantic enough not to be wasting a fork() running cat justonefile |, you
wouldn't have had this problem in the first place:

nohup ~pau/executables/NB4_NoPN < input > diagnostic &

> But zsh is supposed to emulate ksh, right?

And the polite bits of every other shell ;)

> the "solution" for now is to launch the simulation with ksh... or resort
> to screen... but it's a pity, I stick to zsh a lot

Screen has its place.

HTH,

~Tim
-- 
http://pig.sty.nu/
http://scot-pics.sty.nu/

Re: nohup dies

[Peter Stephenson]

"Pau Amaro-Seoane" wrote:
> > nohup is not a shell builtin; it's simply an executable programme that's
> 
> man nohup
> 
> (...)
>        NOTE:  your  shell may have its own version of nohup, which
> usually supersedes the version described
>        here.  Please refer to your shell's documentation for details
> about the options it supports.
> (...)

Yes, I'm well aware nohup *may* be a shell builtin, but in zsh it
isn't.  The easy way to check is to run "which nohup".

> > A lot of other shells, I presume (but I haven't actually checked)
> > including ksh, don't send SIGHUP to processes when the shell exits.
> 
> But zsh is supposed to emulate ksh, right?

Not unless you have "emulate ksh" in effect.  If you do, "setupt nohup"
is turned on and you have the same (global) effect as in ksh (you don't
need the "nohup" command at all).

> nohup whatever should work regardless of shell.

"nohup" does work; as I pointed out you're using it the wrong way.  The
difference in ksh is that you don't need it at all; it's harmless if you
do use it, but ksh doesn't send the HUP signal anyway.

> And it _did_ in the
> older version of zsh, now not anymore. Can you tell me why?

Nothing here has changed, but as I suggested before your old setup may
have had the "nohup" option turned on.

> the "solution" for now is to launch the simulation with ksh... or
> resort to screen... but it's a pity, I stick to zsh a lot

Just "setopt nohup" in your initialisation file, or "emulate ksh" if
you're set on ksh syntax generally.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: nohup dies

[Pau Amaro-Seoane]

| Just "setopt nohup" in your initialisation file

nope...

I tried that and the process died again... Same error

The only way is to start the simulation from ksh. Zsh stubbornly
refuses to nohup it...

I didn't try emulate ksh since I stick to zsh syntax and have billions
of zsh scripts.

Pau


2007/9/29, Peter Stephenson <p.w.stephenson@ntlworld.com>:
> "Pau Amaro-Seoane" wrote:
> > > nohup is not a shell builtin; it's simply an executable programme that's
> >
> > man nohup
> >
> > (...)
> >        NOTE:  your  shell may have its own version of nohup, which
> > usually supersedes the version described
> >        here.  Please refer to your shell's documentation for details
> > about the options it supports.
> > (...)
>
> Yes, I'm well aware nohup *may* be a shell builtin, but in zsh it
> isn't.  The easy way to check is to run "which nohup".
>
> > > A lot of other shells, I presume (but I haven't actually checked)
> > > including ksh, don't send SIGHUP to processes when the shell exits.
> >
> > But zsh is supposed to emulate ksh, right?
>
> Not unless you have "emulate ksh" in effect.  If you do, "setupt nohup"
> is turned on and you have the same (global) effect as in ksh (you don't
> need the "nohup" command at all).
>
> > nohup whatever should work regardless of shell.
>
> "nohup" does work; as I pointed out you're using it the wrong way.  The
> difference in ksh is that you don't need it at all; it's harmless if you
> do use it, but ksh doesn't send the HUP signal anyway.
>
> > And it _did_ in the
> > older version of zsh, now not anymore. Can you tell me why?
>
> Nothing here has changed, but as I suggested before your old setup may
> have had the "nohup" option turned on.
>
> > the "solution" for now is to launch the simulation with ksh... or
> > resort to screen... but it's a pity, I stick to zsh a lot
>
> Just "setopt nohup" in your initialisation file, or "emulate ksh" if
> you're set on ksh syntax generally.
>
> --
> Peter Stephenson <p.w.stephenson@ntlworld.com>
> Web page now at http://homepage.ntlworld.com/p.w.stephenson/
>

Re: nohup dies

[Peter Stephenson]

"Pau Amaro-Seoane" wrote:
> | Just "setopt nohup" in your initialisation file
> 
> nope...
> 
> I tried that and the process died again... Same error

Strange, since I just verified the option is working.  Let's check

- If you do this in an interactive shell and exit, zsh prints
  "zsh: warning: 1 jobs SIGHUPed"?
- This still happens if you do "setopt nohup" on the command line
  before exiting?

If both are the case, then something is very strange.

-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: nohup dies

[Pau Amaro-Seoane]

Hi,

first of all, thanks for your attention.

I am sure screen would work. As you say, I am trying to do it "my
way". I should indeed try without my long zshrc. By the way, I tried
both: with only nohup in front of the cat and with it before and after
the pipe. The result is the same. Zsh prints "you have jobs running"
or something similar; if I type "exit" again then the terminal
actually freezes and when I log it later the simulation is not there
anymore.


> - If you do this in an interactive shell and exit, zsh prints
>   "zsh: warning: 1 jobs SIGHUPed"?

 "zsh: you have jobs running"

> - This still happens if you do "setopt nohup" on the command line
>   before exiting?

didn't try that one... but I tried before launching the simulation.
Doesn't it suffice?

>
> If both are the case, then something is very strange.
>

Let me try to wipe out my zshrc file and see what happens...

Pau

Re: nohup dies

[Pau Amaro-Seoane]

Hi,

I have been playing around and found the problem. Let's go step by step...

I tried the vanilla zshrc by just moving my zshrc out of where zsh
looks for it when I start a session.

After having done this, I set nohup and try to launch the simulation
as I have done all the last years:

tuffi3% set nohup
tuffi3% nohup  cat ./inicia| ~pau/executables/NB4_NoPN > diagnostic &
[1] 19492 19493
tuffi3%
tuffi3% exit
zsh: you have running jobs.
tuffi3% exit
zsh: warning: 1 jobs SIGHUPed

This is new: "zsh: warning: 1 jobs SIGHUPed"

But when I log in again, the job is not there anymore

I try again now with nohup before and after the pipe AND set nohup before and
after quitting

tuffi3% set nohup
tuffi3% vim inicia
tuffi3% nohup  cat ./inicia| nohup ~pau/executables/NB4_NoPN > diagnostic &
[1] 19511 19512
tuffi3% set nohup
tuffi3% exit
zsh: you have running jobs.
tuffi3% exit
zsh: warning: 1 jobs SIGHUPed
Connection to tuffi3 closed.


Hurrah! NB4_NoPN is there when I log in again.

Now... what was the problem? My zshrc? nohup before and after the pipe
or set nohup?

let's see... Let's kill the job and restart zsh to make sure set nohup
is not loaded

tuffi3% pkill -9 NB4_NoPN
tuffi3% zsh

Let's try before the two nohups

tuffi3% nohup  cat ./inicia| nohup ~pau/executables/NB4_NoPN > diagnostic &
[1] 19542 19543
tuffi3% exit
zsh: you have running jobs.
tuffi3% exit
zsh: warning: 1 jobs SIGHUPed

I log in again and... the job is there!

Let's try now NO nohup at all and "set nohup"

tuffi3% pkill -9 NB4_NoPN
tuffi3% cat ./inicia| ~pau/executables/NB4_NoPN > diagnostic &
[1] 19553 19554
tuffi3% exit
zsh: you have running jobs.
tuffi3% exit
zsh: warning: 1 jobs SIGHUPed
Connection to tuffi3 closed.

I log in again: The job has died

Now ONE nohup with "set nohup" on

tuffi3% set nohup
tuffi3% nohup  cat ./inicia| ~pau/executables/NB4_NoPN > diagnostic &
[1] 19566 19567
tuffi3% exit
zsh: you have running jobs.
tuffi3% exit
zsh: warning: 1 jobs SIGHUPed
Connection to tuffi3 closed.

I log in again: The job has died

Now just two nohup and "set nohup" off

tuffi3% nohup  cat ./inicia| nohup ~pau/executables/NB4_NoPN > diagnostic &
zsh: you have running jobs.
tuffi3% exit
zsh: warning: 1 jobs SIGHUPed
Connection to tuffi3 closed.

I log in again: The job is there, it has survived

And now the last test: My zshrc + two "nohups"

tuffi3| nohup  cat ./inicia| nohup ~pau/executables/NB4_NoPN > diagnostic &
[1] 19626 19627
tuffi3| exit
zsh: you have running jobs.
tuffi3| exit
zsh: warning: 1 jobs SIGHUPed
fins després, plaerdemavida
Connection to tuffi3 closed.

I log in again: The job is there.


And just out of curiousity: my zshrc and the nohup AFTER the pipe

tuffi3| pkill -9 NB4_NoPN
tuffi3| cat ./inicia| nohup ~pau/executables/NB4_NoPN > diagnostic &
[1] 19652 19653
tuffi3| exit
zsh: you have running jobs.
tuffi3| exit
zsh: warning: 1 jobs SIGHUPed
fins després, plaerdemavida
Connection to tuffi3 closed.

I log in again: The job is there.


Moral of the story:

(1) My zshrc is not the problem
(2) Setting nohup after the pipe fixes it
(3) ksh behaves differently

But, let me insist: In the old cluster, with a former version of zsh, the shell
"understood what I meant" when I set nohup at the beginning of the
whole command, as ksh does

Why doesn't zsh understand it any more?

Thanks

Pau

Re: nohup dies

[Matthew Wozniski]

On Mon, Oct 01, 2007 at 09:29:24AM +0200, Pau Amaro-Seoane wrote:
> tuffi3% set nohup
That line is your problem.  Options in zsh are set using 'setopt', not
'set'.  You were setting $argv to 'nohup', which wasn't what you
wanted to do.  So, the following should do exactly what you want:

tuffi3% setopt nohup
tuffi3% cat ./inicia| ~pau/executables/NB4_NoPN > diagnostic &
[1] 19492 19493
tuffi3%
tuffi3% exit
zsh: you have running jobs.
tuffi3% exit

As noted, you no longer need to use the nohup command at all if you
use 'setopt nohup'.  Also as pointed out, the nohup command works if
you tell it "don't send a hup signal to cat AND don't send a hup
signal to NB4_NoPN" by putting a call to nohup before each of them.
And, also as pointed out, the best way for you to do this would have
been:
tuffi3% nohup ~pau/executables/NB4_NoPN < inicial > diagnostic &

~Matt Wozniski

Re: nohup dies

[Pau Amaro-Seoane]

"Options in zsh are set using 'setopt' "

aaaargh!!

I know it... I must still be under (5 hours) jetlag... plus the 20 hours trip

In any case thanks a lot. I'll give it a chance.

Cheers,

Pau