zsh-users
 help / color / mirror / code / Atom feed
* processing of pipelines
@ 1999-08-31 15:41 Sweth Chandramouli
  1999-08-31 22:34 ` Bart Schaefer
  0 siblings, 1 reply; 3+ messages in thread
From: Sweth Chandramouli @ 1999-08-31 15:41 UTC (permalink / raw)
  To: ZSH Users

	i've been part of a recent discussion in the comp.unix.shell
newsgroup about how different shells process pipelines, and thought i 
should ask this group about zsh's behaviour.  some quick tests first 
led me to believe that zsh runs each command in a pipeline
sequentially in the current process; some more research now makes me 
think that this only appeared to be the case because i was testing
using a no-op on one side of the pipe, and zsh somehow checks to see 
if the command on the right side of a pipe is actually reading from
the pipe; if not, it treats the pipe like a semicolon.
	my new hypothesis, then, is that zsh (like ksh) runs all commands
in a pipeline in sub-processes except for the last command, which is
run in the current process, but that when the pipe isn't actually
being used, it splits the pipeline up into smaller lists to be
processed individually.  yes?  no?  something else entirely?

	-- sweth.

-- 
Sweth Chandramouli ; <sweth@gwu.edu>
<a href="http://astaroth.nit.gwu.edu/resume/">Will Work For Food.</a>
<a href="http://astaroth.nit.gwu.edu/~sweth/disc.html">*</a>


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: processing of pipelines
  1999-08-31 15:41 processing of pipelines Sweth Chandramouli
@ 1999-08-31 22:34 ` Bart Schaefer
  0 siblings, 0 replies; 3+ messages in thread
From: Bart Schaefer @ 1999-08-31 22:34 UTC (permalink / raw)
  To: Sweth Chandramouli; +Cc: zsh-users

On Aug 31, 11:41am, Sweth Chandramouli wrote:
> Subject: processing of pipelines
> 	i've been part of a recent discussion in the comp.unix.shell
> newsgroup about how different shells process pipelines, and thought i 
> should ask this group about zsh's behaviour.

Sven is probably the best person to give details of the internals, but:

> some quick tests first 
> led me to believe that zsh runs each command in a pipeline
> sequentially in the current process;

If you think about this for any length of time, you'll see that it is
impossible.  If the command is not a shell builtin, the shell MUST fork()
in order to exec() the external program.  Even if the command is a builtin,
having the same process both write and read from the opposite ends of the
same pipe is an invitation to deadlock [which is why `cmd` and $(cmd) also
fork, and I wish Sven luck inventing a way around it that doesn't involve
temporary files and just as much overhead as forking].

> some more research now makes me 
> think that this only appeared to be the case because i was testing
> using a no-op on one side of the pipe, and zsh somehow checks to see 
> if the command on the right side of a pipe is actually reading from
> the pipe; if not, it treats the pipe like a semicolon.

No, that's also impossible.  What zsh does in some circumstances is use a
second pipe between the parent and the child as a semaphore, to delay the
exec() until the parent has successfully entered the child in its process
table.  This avoids a potential race condition where the child may run to
completion and exit before the parent has even returned from the fork()
call.  It does mean that extremely tiny jobs start a bit slower in zsh.

> 	my new hypothesis, then, is that zsh (like ksh) runs all commands
> in a pipeline in sub-processes except for the last command, which is
> run in the current process

Once again, "run in the current process" is not possible except for shell
builtins.  Zsh *does* run the last command in a pipeline in the current
shell when the command *is* a builtin, even if that builtin is a loop,
which is AFAIK different from any other shell; it means that you can do
things like

    some external command | while read line; do export $line; done

and the current shell's environment will actually be modified.  Ksh would
have to use

    some external command > somefile
    . somefile

to get a similar effect.

> but that when the pipe isn't actually
> being used, it splits the pipeline up into smaller lists to be
> processed individually.  yes?  no?  something else entirely?

Something else entirely.

Here's a way to peek at the process tree:

alias -g child='perl -e '\''print @ARGV, ": ", getppid(), "-->", $$, "\n";
			while (<STDIN>) { print; }'\'

echo $$ | child A | child B | child C | child D
D: 5812-->5901
C: 5812-->5900
B: 5812-->5899
A: 5812-->5898
5812

Now try sticking an "exec" in the middle somewhere:

echo $$ | child A | exec child B | child C | child D
D: 5812-->5910
C: 5812-->5909
B: 5812-->5908
A: 5812-->5907
5812

Note that it made no difference; child B was already being exec()d.  Now
put the exec at the end (be sure you start a new shell to try this, or
you'll never see the output):

echo $$ | child A | child B | child C | exec child D
D: 5518-->5812
C: 5812-->5914
B: 5812-->5913
A: 5812-->5912
5812

Now 5812 has exited; it exec'd the last perl in the pipeline, replacing
the shell with it's child.

Note the slight difference when you wrap the whole thing in parens:

(echo $$ | child A | child B | child C | child D) 
D: 5942-->5963
C: 5963-->5967
B: 5963-->5966
A: 5963-->5965
5942

(echo $$ | child A | child B | child C | exec child D) 
D: 5942-->5968
C: 5968-->5972
B: 5968-->5971
A: 5968-->5970
5942

Zsh knows that it's safe to exec the last child when in a subshell, so it
does so even if you don't explicitly say "exec".

Now try it with "child" as a shell function:

unalias \child
child() { perl -e 'print @ARGV, ": ", getppid(), "-->", $$, "\n";
                        while (<STDIN>) { print; }' $* }

echo $$ | child A | child B | child C | child D
D: 5942-->6021
C: 6019-->6020
B: 6017-->6018
A: 6015-->6016
5942

echo $$ | child A | child B | child C | exec child D
D: 5942-->6029
C: 6027-->6028
B: 6025-->6026
A: 6023-->6024
5942

Note that each shell function got its own process, even when the last one
was to be exec'd (though zsh still exits after the last function finishes,
as if it really had done an "exec").  This is so that complex process
management within the shell function is handled by the function process,
while the parent zsh manages the surrounding pipeline.

Make sense?  BTW, there's a proposed patch to 3.1.6 that would change this
slightly in some circumstances, but the basic ideas are the same.


^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: processing of pipelines
@ 1999-09-01  8:29 Sven Wischnowsky
  0 siblings, 0 replies; 3+ messages in thread
From: Sven Wischnowsky @ 1999-09-01  8:29 UTC (permalink / raw)
  To: zsh-users


Bart Schaefer wrote:

> ...
>
> Zsh *does* run the last command in a pipeline in the current
> shell when the command *is* a builtin, even if that builtin is a loop,
> which is AFAIK different from any other shell; it means that you can do
> things like
> 
>     some external command | while read line; do export $line; done
> 
> and the current shell's environment will actually be modified.
>
> ...

Only one more comment: once such a pipe is suspended, the shell
builtin/construct at the end is put in a sub-shell (and the
environment of the parent won't change any more, of course).

This has caused some trouble lately... (but you *can* suspend such
pipes and if you don't do that you can use parameters changed there -- 
its an attempt to get the best of both worlds).

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de


^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~1999-09-01  8:29 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
1999-08-31 15:41 processing of pipelines Sweth Chandramouli
1999-08-31 22:34 ` Bart Schaefer
1999-09-01  8:29 Sven Wischnowsky

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).