From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (qmail 21233 invoked by alias); 10 Mar 2012 17:31:17 -0000 Mailing-List: contact zsh-workers-help@zsh.org; run by ezmlm Precedence: bulk X-No-Archive: yes List-Id: Zsh Workers List List-Post: List-Help: X-Seq: 30338 Received: (qmail 28489 invoked from network); 10 Mar 2012 17:31:06 -0000 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on f.primenet.com.au X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.2 Received-SPF: none (ns1.primenet.com.au: domain at closedmail.com does not designate permitted sender hosts) From: Bart Schaefer Message-id: <120310093040.ZM17762@torch.brasslantern.com> Date: Sat, 10 Mar 2012 09:30:40 -0800 X-Mailer: OpenZMail Classic (0.9.2 24April2005) To: zsh-workers@zsh.org Subject: Puzzling coproc descriptor problem MIME-version: 1.0 Content-type: text/plain; charset=us-ascii This got kind of long as I made notes on my experiments. For the real potentially fixable problem, skip down to after "END FIRST EXAMPLE", but I'm also puzzled about #3 in that first example. Fiddling with Nikolai Weibull's question about capturing both stdout and stderr in separate variables ... the following ought to work but does not for the reasons noted: ## BEGIN FIRST EXAMPLE SCRIPT # First set up a coproc to capture stderr coproc { err="$(cat)"; print -r ${(qq)err} } # Problem #1 is the subshell spawned for { ... } is holding open the # stdin/stdout descriptors, so $(cat) never sees EOF on stdin. It # works for a single line of data with "read -E" in place of cat. # Hmm, pdksh 5.2 and ksh93 also seem to have this same problem, so it # may be endemic to coprocesses, but let's finish the example. # Copy and close descriptors in the parent exec {e0}<&p {e1}>&p coproc : # Now set up a coproc to capture stdout. Note we have to close the # descriptors for the first coproc inside the scope of the second; this # is expected and not a problem with the internals. coproc { exec {e0}<&- {e1}>&-; out="$(cat)"; print -r ${(qq)out} } # Problem #2 same as #1. # Copy descriptors in the parent exec {o0}<&p {o1}>&p # Problem #3 is that at this point, according to the "lsof" command on # linux, the descriptor identified by $o0 is open for *writing* when it # should be open for *reading*. $e0, which was created by exactly the # same form of exec, is correctly open for reading. $o1 and $e1 are # both correctly open for writing. # ## torch% print $o0 $e0 ## 14 12 ## torch% lsof -p $$ | egrep '(12|14)[rwu]' ## zsh 13449 schaefer 12r FIFO 0,7 9045146 pipe ## zsh 13449 schaefer 14w FIFO 0,7 9045462 pipe # # Close descriptors in the parent coproc : # Run the command with output to the appropriate coprocs ... { print THIS IS STDOUT; print -u2 THIS IS STDERR } >&$o1 2>&$e1 # ... and close those descriptors to close the input of $(cat), # except see problem #1 so this is actually futile. exec {o1}>&- {e1}>&- # Finally, read from the two coproc outputs (this hangs forever) out="$(cat <&$o0)" err="$(cat <&$e0)" # And close the read descriptors at the end, not strictly necessary exec {o0}<&- {e0}<&- ## END FIRST EXAMPLE SCRIPT OK, so using a process substitution to read from the coproc input is not going to work, even in other shells. I should be able to get around that by using a loop on "read" that stays all in the current shell. Unfortunately, that doesn't work either: ## BEGIN SECOND EXAMPLE SCRIPT coproc { while { IFS= read -r in } { err+="$in" } print -r ${(qq)err} } exec {e0}<&p {e1}>&p coproc : coproc { exec {e0}<&- {e1}>&-; while { IFS= read -r in } { err+="$in" } print -r ${(qq)err} } exec {o0}<&p {o1}>&p coproc : { print THIS IS STDOUT; print -u2 THIS IS STDERR } >&$o1 2>&$e1 exec {o1}>&- {e1}>&- # Sadly, read from the two coproc outputs STILL hangs forever out="$(cat <&$o0)" err="$(cat <&$e0)" exec {o0}<&- {e0}<&- ## END SECOND EXAMPLE SCRIPT So what's going on? Why doesn't the read loop see end-of-file? torch% coproc { IFS= read -r in } [1] 17673 torch% lsof -p $! | grep '[0-9][rwu]' zsh 17673 schaefer 0r FIFO 0,7 9175667 pipe zsh 17673 schaefer 1w FIFO 0,7 9175666 pipe zsh 17673 schaefer 2u CHR 136,6 8 /dev/pts/6 zsh 17673 schaefer 10u CHR 136,6 8 /dev/pts/6 zsh 17673 schaefer 11r FIFO 0,7 9175666 pipe zsh 17673 schaefer 14w FIFO 0,7 9175667 pipe What? Why is the coproc shell holding open the opposite ends of both ends of the pipe (fd 0r == fd 14w, fd 1w == fd 11r)? This means this shell is talking to itself, in effect; it's impossible for it to ever get EOF on its standard input. If I make a similar examination of pdksh: ksh 17650 schaefer 0r FIFO 0,7 9175095 pipe ksh 17650 schaefer 1w FIFO 0,7 9175096 pipe ksh 17650 schaefer 2u CHR 136,6 8 /dev/pts/6 ksh 17650 schaefer 12r FIFO 0,7 9172900 pipe ksh 17650 schaefer 14r FIFO 0,7 9172926 pipe I'm not sure what 12r and 14r represent here, but they're not the same as 0r, so my second example would work in pdksh with appropriate syntax adjustments. If I do this instead: torch% coproc { exec zsh -fc 'IFS= read -r in' } [1] 17718 torch% lsof -p $! | grep '[0-9][rwu]' zsh 17718 schaefer 0r FIFO 0,7 9176345 pipe zsh 17718 schaefer 1w FIFO 0,7 9176344 pipe zsh 17718 schaefer 2u CHR 136,6 8 /dev/pts/6 zsh 17718 schaefer 10u CHR 5,0 2285 /dev/tty zsh 17718 schaefer 11r CHR 1,3 2286 /dev/null Ah, much better. So those extra copies of the "wrong end" of the pipe are closed on exec ... but for current shell constructs they ought to be getting explicitly closed in the subshell run for coproc. This probably relates back to the discussion from last February that includes zsh-workers/28762.