RE: why does "jobs | wc" not work?

RE: why does "jobs | wc" not work?

[Borzenkov Andrey]

> 
> Yes, I noticed that in the mean time.  Actually, all I am trying
> to do is fetch the number of background jobs in sh without calling
> external commands

bor@itsrm2% sleep 1000000 &
[1] 20639
bor@itsrm2% echo $#jobstates 
1
bor@itsrm2% echo $ZSH_VERSION 
4.0.4

-andrey


 and - preferrably - not writing temporary data
> to files.  It's surprisingly difficult to do:
> 
>   I=0; jobs | while read FOO; do I=$[I+1]; done; echo $I
> 
> works neither in zsh (jobs produces no output) nor in sh (the I
> variable is local to the subshell running "while").  In sh
> (actually, bash in sh mode), I can assign the output of jobs to
> a variable:
> 
>   JOBS=`jobs`
> 
> But that doesn't help because I see no way to get that as input
> into the while loop without forking it into a subshell.  In zsh,
> this should work:
> 
>   I=0; echo "$JOBS" | while read X; do I=$[I+1]; done; echo $I
> 
> But then,
> 
>   JOBS=`jobs`
> 
> fails :-/
> 
> All I can think of is to write the output in a temporary file:
> 
>   jobs > x; I=0; while read X; do I=$[I+1]; done < x; echo $I
> 
> Can anyone think of a more efficient way (speed does matter here)?
> 
> Bye
> 
> Dominik ^_^  ^_^

why does "jobs | wc" not work?

[Dominik Vogt]

There seems to be a strange bug with the jobs command in
zsh-4.0.4.  It seems that the output of the jobs command refuses
to go into a pipe.

This works:

  $ sleep 1000 &
  [1] 6262
  $ jobs > foo
  $ wc foo
        1       5      29 foo   <--

But this does not:

  $ sleep 1000 &
  [1] 6262
  $ jobs | wc
        0       0       0       <-- ?

And also:

  $ echo `jobs`

  $

Is there a specific reason why jobs generates no output when
running in a pipe?

Note:  In pdksh and bash (as bash and sh), jobs works as I would
expect; csh and tcsh work like zsh.

Bye

Dominik ^_^  ^_^

P.S.:  Please CC me.

Re: why does "jobs | wc" not work?

[Vincent Lefevre]

In article <20021126104930.GE1937@gmx.de>,
   Dominik Vogt <dominik.vogt@gmx.de> wrote:
> There seems to be a strange bug with the jobs command in
> zsh-4.0.4.  It seems that the output of the jobs command refuses
> to go into a pipe.

> This works:

>   $ sleep 1000 &
>   [1] 6262
>   $ jobs > foo
>   $ wc foo
>         1       5      29 foo   <--

> But this does not:

>   $ sleep 1000 &
>   [1] 6262
>   $ jobs | wc
>         0       0       0       <-- ?

> And also:

>   $ echo `jobs`

>   $

> Is there a specific reason why jobs generates no output when
> running in a pipe?

Because jobs is executed in a subshell?

This is not the case with jobs > foo. "true | jobs" is also OK.

> Note:  In pdksh and bash (as bash and sh), jobs works as I would
> expect; csh and tcsh work like zsh.

AFAIK, shells do not implement the pipe and jobs in the same way.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: why does "jobs | wc" not work?

[Peter Stephenson]

Dominik Vogt wrote:
> There seems to be a strange bug with the jobs command in
> zsh-4.0.4.  It seems that the output of the jobs command refuses
> to go into a pipe.

This isn't strictly a bug, but there's a workaround in the 4.1 code.

In zsh

jobs | anything

causes the shell to fork for the left hand side of the pipeline.  This
is then no longer the shell with the job control, hence doesn't show any
jobs.  Other shells fork the `anything' and run `jobs' in the current
shell.  We deliberately don't do that because we like to be able to do

anything | read foo

to get the variable $foo set in the current shell (and all sorts of similar
examples).

In 4.1, we use the obvious workaround:  remember that we're in a subshell
of an interactive shell and keep a fossilized list of the jobs.  This can
mean that the task list isn't up to date at the point where the jobs
command is actually run --- though there are obvious races when
background jobs terminate anyway, so I don't think that's a major
concern.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: why does "jobs | wc" not work?

[Vincent Lefevre]

On Tue, Nov 26, 2002 at 11:09:15 +0000, Peter Stephenson wrote:
> This isn't strictly a bug, but there's a workaround in the 4.1 code.
> 
> In zsh
> 
> jobs | anything
> 
> causes the shell to fork for the left hand side of the pipeline.  This
> is then no longer the shell with the job control, hence doesn't show any
> jobs.  Other shells fork the `anything' and run `jobs' in the current
> shell.

But with bash for instance, both

  jobs | cat

and

  true | jobs

work. So, it seems that jobs still works even in a child process.
The same thing that 4.1 does?

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: why does "jobs | wc" not work?

[Peter Stephenson]

Vincent Lefevre wrote:
> But with bash for instance, both
> 
>   jobs | cat
> 
> and
> 
>   true | jobs
> 
> work. So, it seems that jobs still works even in a child process.
> The same thing that 4.1 does?

It certainly looks like a similar hack of some sorts.  None of

   true | { jobs; }
   true | ( jobs; )
   function j { jobs; }; true | j

produce any output in the version of bash we have here (which probably isn't
up to date).  It so happens that with the hack we use, the special
case survives for the equivalent code on the left hand side of the pipe,
but a hack is a hack...

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: why does "jobs | wc" not work?

[Dominik Vogt]

On Tue, Nov 26, 2002 at 11:09:15AM +0000, Peter Stephenson wrote:
> Dominik Vogt wrote:
> > There seems to be a strange bug with the jobs command in
> > zsh-4.0.4.  It seems that the output of the jobs command refuses
> > to go into a pipe.
> 
> This isn't strictly a bug, but there's a workaround in the 4.1 code.
> 
> In zsh
> 
> jobs | anything
> 
> causes the shell to fork for the left hand side of the pipeline.

Um, why must builtin commands run in a subshell?  I would have
naively thought commands like "echo something | ..." would just
run in the current shell.  Spawning a subshell sounds a bit
inefficient.

> This
> is then no longer the shell with the job control, hence doesn't show any
> jobs.  Other shells fork the `anything' and run `jobs' in the current
> shell.  We deliberately don't do that because we like to be able to do
> 
> anything | read foo
> 
> to get the variable $foo set in the current shell (and all sorts of similar
> examples).
> 
> In 4.1, we use the obvious workaround:  remember that we're in a subshell
> of an interactive shell and keep a fossilized list of the jobs.

Obvious, maybe, but it still sounds wrong in my ears.

> This can
> mean that the task list isn't up to date at the point where the jobs
> command is actually run --- though there are obvious races when
> background jobs terminate anyway, so I don't think that's a major
> concern.

Bye

Dominik ^_^  ^_^

Re: why does "jobs | wc" not work?

[Vincent Lefevre]

On Tue, Nov 26, 2002 at 12:42:22 +0100, Dominik Vogt wrote:
> On Tue, Nov 26, 2002 at 11:09:15AM +0000, Peter Stephenson wrote:
> > In zsh
> > 
> > jobs | anything
> > 
> > causes the shell to fork for the left hand side of the pipeline.
> 
> Um, why must builtin commands run in a subshell?  I would have
> naively thought commands like "echo something | ..." would just
> run in the current shell.  Spawning a subshell sounds a bit
> inefficient.

On the contrary, both commands can run asynchronously. Thus, this is
more efficient, well at least for commands that take time.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: why does "jobs | wc" not work?

[Peter Stephenson]

Dominik Vogt wrote:
> > In zsh
> > 
> > jobs | anything
> > 
> > causes the shell to fork for the left hand side of the pipeline.
> 
> Um, why must builtin commands run in a subshell?  I would have
> naively thought commands like "echo something | ..." would just
> run in the current shell.  Spawning a subshell sounds a bit
> inefficient.

That's how all pipes work in all shells and scripting languages in all
variants of UNIX:  one process feeds its output to another process, so
one side of the pipe has to be forked as a separate process.  The only
question is which bits are forked off.

The only obvious alternative is the horrible method which used to be
used in MSDOS: fake separate processes by switching between the two
tasks; but we are certainly never going to have a task manager in the
shell when we have a proper multiprocessor operating system to do the
job more efficiently.

> > In 4.1, we use the obvious workaround:  remember that we're in a subshell
> > of an interactive shell and keep a fossilized list of the jobs.
> 
> Obvious, maybe, but it still sounds wrong in my ears.

Just in case my explanation above hasn't made this remark obvious:
wrong in what way?  It's either that or no output at all.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: why does "jobs | wc" not work?

[Dominik Vogt]

On Tue, Nov 26, 2002 at 12:08:16PM +0000, Peter Stephenson wrote:
> Dominik Vogt wrote:
> > > In zsh
> > > 
> > > jobs | anything
> > > 
> > > causes the shell to fork for the left hand side of the pipeline.
> > 
> > Um, why must builtin commands run in a subshell?  I would have
> > naively thought commands like "echo something | ..." would just
> > run in the current shell.  Spawning a subshell sounds a bit
> > inefficient.
> 
> That's how all pipes work in all shells and scripting languages in all
> variants of UNIX:  one process feeds its output to another process, so
> one side of the pipe has to be forked as a separate process.  The only
> question is which bits are forked off.

Okay, understood.

>From a user's point of view I think it would be better to fork the
right hand side of the pipe.  This way, the left hand side always
generates the same output, regardless of the context in which it
is used.  Of course this is not better per se, but only because
more often the output is generated by the left side of the pipe.

Not that I want to talk you into changing anything.  In my eyes,
the real flaw lies in the design of the jobs command.  Commands
printing information about the current shell should not rely on
design choices inherent to the shell.

> The only obvious alternative is the horrible method which used to be
> used in MSDOS: fake separate processes by switching between the two
> tasks; but we are certainly never going to have a task manager in the
> shell when we have a proper multiprocessor operating system to do the
> job more efficiently.

Nobody wants to go back to MSDOS days :-)

> > > In 4.1, we use the obvious workaround:  remember that we're in a subshell
> > > of an interactive shell and keep a fossilized list of the jobs.
> > 
> > Obvious, maybe, but it still sounds wrong in my ears.
> 
> Just in case my explanation above hasn't made this remark obvious:
> wrong in what way?

In exporting private information of a shell to other shells.

> It's either that or no output at all.

Personally, I would just document that fact in the man page and
point the user to the right way to acquire this information.

  Note:  Due to the inherent design of zsh and some other shells,
  the jobs command does not work on the left hand side of a pipe
  (e.g. "jobs | grep zsh").  The same information can be acquired
  from the job... parameters.  See section <xyz> in the
  zsh<foobar> man page for details.

Anyway, documenting such pitfalls right in the command
descriptions in the man page is a good idea.

Bye

Dominik ^_^  ^_^

Re: why does "jobs | wc" not work?

[Peter Stephenson]

Dominik Vogt wrote:
> From a user's point of view I think it would be better to fork the
> right hand side of the pipe.  This way, the left hand side always
> generates the same output, regardless of the context in which it
> is used.

If you look in the completion system, you will see various places where
we do stuff on the lines of

  output_some_values | while read line; ...

in order to process values for the current shell, and very few where we
need to output transient data from shell builtins.  I think jobs is the
only common case.

> In my eyes,
> the real flaw lies in the design of the jobs command.  Commands
> printing information about the current shell should not rely on
> design choices inherent to the shell.

Surely that's the one thing they have got to rely on :-).

The basic problem is that job control relies on the process structure of
UNIX.  There is no way this can be completely hidden from the user, nor
should it be or it will be useless for its job of dealing with multiple
processes efficiently.

I agree it could have been better documented, it's not at all obvious
without some prior knowledge of the architecture.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: why does "jobs | wc" not work?

[Dominik Vogt]

On Tue, Nov 26, 2002 at 01:05:21PM +0000, Peter Stephenson wrote:
> Dominik Vogt wrote:
> > From a user's point of view I think it would be better to fork the
> > right hand side of the pipe.  This way, the left hand side always
> > generates the same output, regardless of the context in which it
> > is used.
> 
> If you look in the completion system, you will see various places where
> we do stuff on the lines of
> 
>   output_some_values | while read line; ...
> 
> in order to process values for the current shell, and very few where we
> need to output transient data from shell builtins.  I think jobs is the
> only common case.

Yes, I noticed that in the mean time.  Actually, all I am trying
to do is fetch the number of background jobs in sh without calling
external commands and - preferrably - not writing temporary data
to files.  It's surprisingly difficult to do:

  I=0; jobs | while read FOO; do I=$[I+1]; done; echo $I

works neither in zsh (jobs produces no output) nor in sh (the I
variable is local to the subshell running "while").  In sh
(actually, bash in sh mode), I can assign the output of jobs to
a variable:

  JOBS=`jobs`

But that doesn't help because I see no way to get that as input
into the while loop without forking it into a subshell.  In zsh,
this should work:

  I=0; echo "$JOBS" | while read X; do I=$[I+1]; done; echo $I

But then,

  JOBS=`jobs`

fails :-/

All I can think of is to write the output in a temporary file:

  jobs > x; I=0; while read X; do I=$[I+1]; done < x; echo $I

Can anyone think of a more efficient way (speed does matter here)?

Bye

Dominik ^_^  ^_^

Re: why does "jobs | wc" not work?

[Danek Duvall]

On Tue, Nov 26, 2002 at 02:33:33PM +0100, Dominik Vogt wrote:

>   jobs > x; I=0; while read X; do I=$[I+1]; done < x; echo $I
> 
> Can anyone think of a more efficient way (speed does matter here)?

Assuming you have a recent enough version of zsh (4.0.x has it, but I'm
not sure when it was introduced):

    echo $#jobstates

$jobdirs and $jobtexts are also available.  They're in the zsh/parameter
module if you don't already have it loaded.

Danek

Re: why does "jobs | wc" not work?

[Vincent Lefevre]

On Tue, Nov 26, 2002 at 13:48:14 +0100, Dominik Vogt wrote:
> From a user's point of view I think it would be better to fork the
> right hand side of the pipe.  This way, the left hand side always
> generates the same output, regardless of the context in which it
> is used.

With most programs, I suppose that this should be equivalent.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/> - 100%
validated (X)HTML - Acorn Risc PC, Yellow Pig 17, Championnat International
des Jeux Mathématiques et Logiques, TETRHEX, etc.
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: why does "jobs | wc" not work?

[Stephane Chazelas]

On Tue, Nov 26, 2002 at 11:49:30AM +0100, Dominik Vogt wrote:
> There seems to be a strange bug with the jobs command in
> zsh-4.0.4.  It seems that the output of the jobs command refuses
> to go into a pipe.
[...]

It's more a problem of subshell.

( jobs )
doesn't work.

Note that you can get the number of jobs with ${#jobtexts}

Here is my PS1:
PS1='%(?..%B(%?%)%b)%m:%50<..<%~${jobtexts:+"[${#jobtexts}]"}%(#.#.$) '

-- 
Stéphane