night of the living dead (processes)?

night of the living dead (processes)?

[TGAPE!]

Is there any way to handle the children produced in the <() construct
(and similar ones, as well) in zsh, instead of tossing them to init?
I've been running scripts which utilize this construct heavily, and I've
been monitoring them with top, and I've noticed they toss an awful lot
of zombies to init, based on the fact that init starts taking lots of
time (60%+ CPU at bad times), and if I renice init, top starts catching
zombies, a couple times more than twenty.

init is not good at handling zombies, since it needs to re-read
/etc/inittab each time (and it should, too), as well as checking all of
the tasks mentioned there to make sure they're still running.  I'm
running ZSH_VERSION 3.1.1.  I contemplated upgrading to 3.0.5, but it's
slower for all of my longest running tasks (**/glob stuff, and such) as
well as all other so I did not make install.  Bash also has this
problem.

Ed

Re: night of the living dead (processes)?

[Bart Schaefer]

On Oct 22, 12:14am, TGAPE! wrote:
} Subject: night of the living dead (processes)?
}
} Is there any way to handle the children produced in the <() construct
} (and similar ones, as well) in zsh, instead of tossing them to init?

The only way a process should ever get "tossed" is if its parent exits
without wait()ing for it.  The top-level zsh obviously isn't exiting (is
it?), so that must mean zsh fork()ed and then the subshell fork()ed again
before the orphaned process was finally exec()d.  That in turn means (at
least I think it must) that you have a pipeline inside the <(...).

The only way to avoid orphaning those jobs would be to have intermediate
subshells fork() an additional time, rather than exec()ing the last job
in the pipeline directly, and then hang around doing nothing but wait()
until all the piped jobs have exited.  Only init and the immediate parent
of a job may wait() for it.  So you either have to burn an extra slot in
the process table (and swap space for a duplicate of the shell) for the
full duration of every subshelled pipeline, or do what zsh (and bash, it
seems) does.

I suppose which is best depends on whether your performance is CPU-limited
or process-space-limited.

} I'm running ZSH_VERSION 3.1.1.  I contemplated upgrading to 3.0.5

Changing to 3.0.5 would not alter this process management behavior.

> but it's
} slower for all of my longest running tasks (**/glob stuff, and such) as
} well as all other so I did not make install.

What do you mean "as well as all other"?

Can you give some more specific examples, and some statistics?  Is there
an earlier 3.0.x that shows the performance you expect, or is there some
reason why all 3.1.y should be faster than 3.0.x?  I haven't noticed any
particular difference from earlier 3.0s with 3.0.5.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

workaround for Re: night of the living dead (processes)?

[TGAPE!]

I'm posting this under the delusion that some other people other than
myself might be interested in it.  Sorry to everyone who isn't...

Bart Schaefer wrote:
> On Oct 22, 12:14am, TGAPE! wrote:
>} Subject: night of the living dead (processes)?
>}
>} Is there any way to handle the children produced in the <() construct
>} (and similar ones, as well) in zsh, instead of tossing them to init?
>
> The only way a process should ever get "tossed" is if its parent exits
> without wait()ing for it.  The top-level zsh obviously isn't exiting (is
> it?), so that must mean zsh fork()ed and then the subshell fork()ed again
> before the orphaned process was finally exec()d.  That in turn means (at
> least I think it must) that you have a pipeline inside the <(...).
>
> The only way to avoid orphaning those jobs would be to have intermediate
> subshells fork() an additional time, rather than exec()ing the last job
> in the pipeline directly, and then hang around doing nothing but wait()
> until all the piped jobs have exited.  Only init and the immediate parent
> of a job may wait() for it.  So you either have to burn an extra slot in
> the process table (and swap space for a duplicate of the shell) for the
> full duration of every subshelled pipeline, or do what zsh (and bash, it
> seems) does.

Right.  *BUT*, zsh, bash, and ksh all have a fix for this - shell
functions.  Simply make a shell function to do your pipeline - that will
give you your extra shell process.

For example, change
    m=0; while read x y
    do
      ((t = $(ls -l $x | awk '{ print $5 }') - $(ls -l $y | awk '{print $5}')))
      if (($t > $m)); then
        m=$t
      fi
    done < u;echo $m
to
    function getsize() {
      ls -l $1 | awk '{ print $5 }'
    }
    m=0; while read x y
    do
      ((t = `getsize $x` - $(getsize $y)))
      if (($t > $m)); then
        m=$t
      fi
    done < u;echo $m


Now, I'm sure, just like with everything else I do, there's an even
better way to do this.  However, this works, and doesn't thrash init.

> I suppose which is best depends on whether your performance is CPU-limited
> or process-space-limited.

Extra swapping takes *far* less time than init's thrashing, unless your
machine is so busy that virtually every page of memory is active.  If
that's the case, you need help.  (At least, on my system, the thing's
been utterly more responsive using this method - even my other processes
run much better...  Your mileage might vary, but I don't think it'll vary
significantly.  Still, this is suffering the delusion you'd have done
something this insane, but we all have our delusions.)

Ed

night of the living dead (processes)?

[Ed Grimm]

Is there any way to handle the children produced in the <() construct
(and similar ones, as well) in zsh, instead of tossing them to init?
I've been running scripts which utilize this construct heavily, and I've
been monitoring them with top, and I've noticed they toss an awful lot
of zombies to init, based on the fact that init starts taking lots of
time (60%+ CPU at bad times), and if I renice init, top starts catching
zombies, a couple times more than twenty.

init is not good at handling zombies, since it needs to re-read
/etc/inittab each time (and it should, too), as well as checking all of
the tasks mentioned there to make sure they're still running.  I'm
running ZSH_VERSION 3.1.1.  I contemplated upgrading to 3.0.5, but it's
slower for all of my longest running tasks (**/glob stuff, and such) as
well as all other so I did not make install.  Bash also has this
problem.

Ed