Bi-directional pipe

Bi-directional pipe

[Guillaume Chazarain]

Hi,

Is there a way to make a bi-directional pipe in zsh?
That is, something like: foo | bar doing implicitly bar | foo, so
foo stdin <- bar stdout and
foo stdout <- bar stdin

Here is an example of what I want to do:

#!/bin/zsh

mkfifo fifo
exec 4>&1
(
exec 0<fifo
rm fifo
RES=0
while [ "$RES" -lt 10 ]; do
    echo "$RES+1"
    read RES
    echo "$RES" >&4
done
) | bc -q > fifo


The script enclosed between parenthesis writes and read to/from bc.
This approach uses a FIFO, so I would have hoped for a solution to
avoid needing a writable part of the filesystem ;-)

Maybe there is a magic incantation I can substitute to the | in order to
have a bi-directional pipe, or some function to create a pipe that I would
redirect to.

Thanks in advance for any help.

-- 
Guillaume

Re: Bi-directional pipe

[Peter Stephenson]

Guillaume Chazarain wrote:
> Is there a way to make a bi-directional pipe in zsh?
> That is, something like: foo | bar doing implicitly bar | foo, so
> foo stdin <- bar stdout and
> foo stdout <- bar stdin

With that sort of syntactically supported pipe, this depends on the OS.
Some, like Solaris, have bidirectional pipes by default.  On Solaris I
can do:

 ( exec 0<&1; echo I said foo; read line; print $line >/dev/tty) |
 ( exec 1>&0; read line; print $line >/dev/tty; echo You said bar; )
I said foo
You said bar

On Fedora Core 5, however, this gives me

I said foo
zsh: write error: bad file descriptor

If you are happy with something that looks a little bit like a
biderectional pipe, but one of the processes is actually asynchronous,
coprocesses are the simplest way of doing this (as mentioned in my
reponse zsh-users/10354 yesterday).  They contain a bidirectional pipe
as standard.  What's more, there's nothing to stop you from redirecting
stdin and stdout in the main shell from and to the coprocess.  The
following seems to work fine:

  (
  coproc {
    read line
    print Coproc got $line >&2
    print Reply
  }
  exec 1>&p 0<&p

  print Request
  read line

  print Main shell got $line >&2
  )

producing:
Coproc got Request
Main shell got Reply

The only point of the business with fd 2 is to confirm that it's
working.  In summary, you can get what you want with:

  (
    coproc {
       <foo code here>
    }
    exec 1>&p 0<&p

    <bar code here>
  )

You don't even need the subshell, necessary: that's just there to
protect the interactive environment from the redirection.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


To access the latest news from CSR copy this link into a web browser:  http://www.csr.com/email_sig.php

Re: Bi-directional pipe

[Guillaume Chazarain]

Peter Stephenson wrote :
> On Fedora Core 5, however, this gives me
>
> I said foo
> zsh: write error: bad file descriptor
>   
What a pity :-(
> In summary, you can get what you want with:
>
>   (
>     coproc {
>        <foo code here>
>     }
>     exec 1>&p 0<&p
>
>     <bar code here>
>   )
>   
Thank you very much, that's exactly what I needed :-)

-- 
Guillaume

Re: Bi-directional pipe

[Bart Schaefer]

On Jun 7,  2:50pm, Guillaume Chazarain wrote:
}
} Is there a way to make a bi-directional pipe in zsh?

Peter has already given an answer to this, but I just have to add that
having two processes connected in a loop this way is a sure road to
deadlock unless both processes are fully prepared for it (which means
for example that they're polling their input and output with select()
or poll() and using non-blocking reads and writes).

It might *seem* to work in simple tests, but that's just because the
operating system and/or the I/O library are doing buffering of the
input and output streams.  As soon as one of the processes produces
more output than will fit in the buffer, you're in deep trouble.

Re: Bi-directional pipe

[Guillaume Chazarain]

Bart Schaefer wrote :
> As soon as one of the processes produces
> more output than will fit in the buffer, you're in deep trouble.
>   
My solution is to do something like this:
echo "$LARGE_REQUEST" >&p &
read REPLY <&p

-- 
Guillaume

Re: Bi-directional pipe

[Peter Stephenson]

Bart Schaefer wrote:
> On Jun 7,  2:50pm, Guillaume Chazarain wrote:
> }
> } Is there a way to make a bi-directional pipe in zsh?
> 
> Peter has already given an answer to this, but I just have to add that
> having two processes connected in a loop this way is a sure road to
> deadlock unless both processes are fully prepared for it (which means
> for example that they're polling their input and output with select()
> or poll() and using non-blocking reads and writes).

Right, that's one of the reasons for using the TCP system as described
in my previous response to Johann; it drains any data that arrives
during an interactive session.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


To access the latest news from CSR copy this link into a web browser:  http://www.csr.com/email_sig.php

Re: Bi-directional pipe

[Bart Schaefer]

On Jun 9, 11:06am, Guillaume Chazarain wrote:
}
} My solution is to do something like this:
} echo "$LARGE_REQUEST" >&p &
} read REPLY <&p

In that case you don't have only two processes.  If one process "knows"
to fork for large writes, that process is prepared for the situation.
Most read-stdin/write-stdout unix commands that one might execute are
not so prepared, is my point.

Re: Bi-directional pipe

[Vincent Lefevre]

On 2006-06-09 02:51:43 -0700, Bart Schaefer wrote:
> On Jun 9, 11:06am, Guillaume Chazarain wrote:
> } My solution is to do something like this:
> } echo "$LARGE_REQUEST" >&p &
> } read REPLY <&p
> 
> In that case you don't have only two processes.  If one process "knows"
> to fork for large writes, that process is prepared for the situation.
> Most read-stdin/write-stdout unix commands that one might execute are
> not so prepared, is my point.

Could you give an example? I'd say that's a bug in the command that
starts the bidirectional pipe. I don't see how commands like grep
could do anything.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / SPACES project at LORIA

Re: Bi-directional pipe

[Bart Schaefer]

On Jun 11,  5:06pm, Vincent Lefevre wrote:
} Subject: Re: Bi-directional pipe
}
} Could you give an example?

A correct one?  No.

} I'd say that's a bug in the command that starts the bidirectional
} pipe.

Exactly.

Note that we're not really talking about "a bidirectional pipe" here.
We're talking about a loop in a pipeline -- there are two pipes, each
from a stdout to a stdin.  It's only "bi"-directional because there
are only two processes in the loop.

} I don't see how commands like grep could do anything.

You mean, why would anyone even think of connecting two programs this
way?  Side effects.  Consider tee, the "w" command in sed, or anything
that produces stderr output that might be interesting.

More likely, though, is that someone writes a new program expecting it
to work in such a loop, but without thinking through the consequences.