Is it possible to capture stdout and stderr to separate variables in Zsh?

Is it possible to capture stdout and stderr to separate variables in Zsh?

[Nikolai Weibull]

Hi!

Is it possible to capture stdout and stderr to separate variables in Zsh?

I understand that it’s not possible in sh, but I was wondering if any
of Zsh’s additions in the redirection area would allow for such a
separation.

Thanks!

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Philippe Troin]

On Tue, 2012-03-06 at 09:09 +0100, Nikolai Weibull wrote:
> Is it possible to capture stdout and stderr to separate variables in Zsh?
> 
> I understand that it’s not possible in sh, but I was wondering if any
> of Zsh’s additions in the redirection area would allow for such a
> separation.

All I can think of is:

        coproc cat &
        pid=$!
        stdout="$( ( print "printed on stdout"; print -u 2 "printer on stderr" ) 2>&p )"
        sleep 1 
        kill "$pid"
        stderr="$(cat <&p)"
        print "stdout=\"$stdout\""
        print "stderr=\"$stderr\""

You'll notice the very ugly sleep+kill hack I had to use as I could not
find how you can close a coprocess's standard input cleanly.  Removing
the sleep+kill makes the cat <&p hang forever.

Of course this solution will hang if more than a buffer-full is printed
on stderr.  You can play tricks with dd by providing a bigger buffer, as
in:

        coproc dd obs=1M &

But in the end it's a losing game.  You might as well use a temporary
file for one of the output streams.

A completely different solution could involve the tcp zsh module which
can multiplex many streams with tcp_expect.  But that's probably too
involved for this problem.

Phil.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Bart Schaefer]

On Mar 6,  9:16am, Philippe Troin wrote:
}
} On Tue, 2012-03-06 at 09:09 +0100, Nikolai Weibull wrote:
} > Is it possible to capture stdout and stderr to separate variables in Zsh?
} 
} All I can think of is:
} 
}         coproc cat &
}         pid=$!
}         stdout="$( ( print "printed on stdout"; print -u 2 "printer on stderr" ) 2>&p )"
}         sleep 1 
}         kill "$pid"
}         stderr="$(cat <&p)"
} 
} You'll notice the very ugly sleep+kill hack I had to use as I could not
} find how you can close a coprocess's standard input cleanly.  Removing
} the sleep+kill makes the cat <&p hang forever.

You need this:  http://www.zsh.org/mla/users/2011/msg00095.html  :-)

} A completely different solution could involve the tcp zsh module which
} can multiplex many streams with tcp_expect.  But that's probably too
} involved for this problem.

You might also be able to do something with the zsh/zselect module, but
just use a temp file.  That solution works in bash, too.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Nikolai Weibull]

On Wed, Mar 7, 2012 at 08:01, Bart Schaefer <schaefer@brasslantern.com> wrote:

> } On Tue, 2012-03-06 at 09:09 +0100, Nikolai Weibull wrote:
> } > Is it possible to capture stdout and stderr to separate variables in Zsh?

> You might also be able to do something with the zsh/zselect module, but
> just use a temp file.  That solution works in bash, too.

Invoking two additional commands (mktemp and rm) is prohibitively
expensive on Cygwin for my use case.  I used the following solution
instead:

outs=("${(@0):-"$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)"}")

The indexes into outs are reversed in regard to the file descriptors,
but that’s not a big deal.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Bart Schaefer]

On Mar 7, 10:26am, Nikolai Weibull wrote:
}
} outs=("${(@0):-"$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)"}")

That's nice.  You can solve the problem that Philippe mentioned with
loss of trailing newlines, by embedding one NUL in $out like so:

outs=("${(@0):-"$({ out="$(x; print -n $'\0')" } 2>&1;
                    print $'\0'$out$status)"}")

Then I sort of like the idea of making it into an associative array:

typeset -A outs
outs=(STDERR "${(@0):-"$({ out="$(x; print -n $'\0')" } 2>&1;
                    print $'\0'STDOUT$'\0'${out}STATUS$'\0'$status)"}")
print -r $outs[STDOUT]

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Nikolai Weibull]

On Thu, Mar 8, 2012 at 15:53, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Mar 7, 10:26am, Nikolai Weibull wrote:
> }
> } outs=("${(@0):-"$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)"}")
>
> That's nice.  You can solve the problem that Philippe mentioned with
> loss of trailing newlines, by embedding one NUL in $out like so:
>
> outs=("${(@0):-"$({ out="$(x; print -n $'\0')" } 2>&1;
>                    print $'\0'$out$status)"}")

That will, however, override the value of $status.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Bart Schaefer]

On Mar 30,  9:15am, Nikolai Weibull wrote:
} Subject: Re: Is it possible to capture stdout and stderr to separate varia
}
} On Thu, Mar 8, 2012 at 15:53, Bart Schaefer <schaefer@brasslantern.com> wrote:
} > On Mar 7, 10:26am, Nikolai Weibull wrote:
} > }
} > } outs=("${(@0):-"$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)"}")
} >
} > That's nice.  You can solve the problem that Philippe mentioned with
} > loss of trailing newlines, by embedding one NUL in $out like so:
} >
} > outs=("${(@0):-"$({ out="$(x; print -n $'\0')" } 2>&1;
} >                    print $'\0'$out$status)"}")
} 
} That will, however, override the value of $status.

Indeed, good catch.  You need to pull $status inside the $(...) too
(in which case the trailing newline doesn't matter any more):

outs=("${(@0):-"$({ out="$(x; print $'\0'$status)" } 2>&1;
                   print $'\0'$out)"}")

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Han Pingtian]

On Wed, Mar 07, 2012 at 10:26:57AM +0100, Nikolai Weibull wrote:
> Invoking two additional commands (mktemp and rm) is prohibitively
> expensive on Cygwin for my use case.  I used the following solution
> instead:
> 
> outs=("${(@0):-"$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)"}")

Looks like it can be written as:

outs=("${(0)$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)}")

e.g.

% outs=("${(0)$({ out=$(fuser /usr/local/bin/zsh) } 2>&1; print $'\0'$out$'\0'$status)}")
/usr/local/bin/zsh: ee

  8013 14594
0

So the $'\n' and $'\0' are both reserved in the nested command
substitution since the whole substitution is in double quote. Not sure
if this is right here.

Thanks.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Philippe Troin]

On Tue, 2012-03-06 at 23:01 -0800, Bart Schaefer wrote:
> On Mar 6,  9:16am, Philippe Troin wrote:
> }
> } On Tue, 2012-03-06 at 09:09 +0100, Nikolai Weibull wrote:
> } > Is it possible to capture stdout and stderr to separate variables in Zsh?
> } 
> } All I can think of is:
> } 
> }         coproc cat &
> }         pid=$!
> }         stdout="$( ( print "printed on stdout"; print -u 2 "printer on stderr" ) 2>&p )"
> }         sleep 1 
> }         kill "$pid"
> }         stderr="$(cat <&p)"
> } 
> } You'll notice the very ugly sleep+kill hack I had to use as I could not
> } find how you can close a coprocess's standard input cleanly.  Removing
> } the sleep+kill makes the cat <&p hang forever.
> 
> You need this:  http://www.zsh.org/mla/users/2011/msg00095.html  :-)

I didn't see anything in there that could suppress the sleep+kill.

> } A completely different solution could involve the tcp zsh module which
> } can multiplex many streams with tcp_expect.  But that's probably too
> } involved for this problem.
> 
> You might also be able to do something with the zsh/zselect module, but
> just use a temp file.  That solution works in bash, too.

I like Nikolai's solution best, except that it's somewhat cryptic and it
munges NULs and trailing newlines.

outs=("${(@0):-"$({ out=$(x) } 2>&1; print $'\0'$out$'\0'$status)"}")

Phil.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Bart Schaefer]

On Mar 7, 11:37am, Philippe Troin wrote:
}
} > }         coproc cat &
} > }         pid=$!
} > }         stdout="$( ( print "printed on stdout"; print -u 2 "printer on stderr" ) 2>&p )"
} > }         sleep 1 
} > }         kill "$pid"
} > }         stderr="$(cat <&p)"
} > 
} > You need this:  http://www.zsh.org/mla/users/2011/msg00095.html  :-)
} 
} I didn't see anything in there that could suppress the sleep+kill.

Look at the part subtitled "Where might I go wrong with coproc?" ... it
doesn't answer the whole problem, but it tells you how to cleanly close
the descriptors.  The full solution is more like:

	coproc cat &
	exec {p}<&p		# Copy the <&p descriptor
	stdout="$( ( print "printed on stdout";
	             print -u 2 "printed on stderr") 2>&p )"
	coproc exit		# Close both <&p and >&p descriptors
	stderr="$(cat <&$p)"	# Read from copy descriptor
	exec {p}<&-		# Close the copy (optional)

I also dug this up:  http://www.zsh.org/mla/workers/2000/msg03684.html
but that predates the fancy {p}<&p syntax.

} I like Nikolai's solution best, except that it's somewhat cryptic

Yeah, I may have some thoughts on that one, too.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Han Pingtian]

On Tue, Mar 06, 2012 at 11:01:11PM -0800, Bart Schaefer wrote:
> On Mar 6,  9:16am, Philippe Troin wrote:
> }
> } On Tue, 2012-03-06 at 09:09 +0100, Nikolai Weibull wrote:
> } > Is it possible to capture stdout and stderr to separate variables in Zsh?
> } 
> } All I can think of is:
> } 
> }         coproc cat &
> }         pid=$!
> }         stdout="$( ( print "printed on stdout"; print -u 2 "printer on stderr" ) 2>&p )"
> }         sleep 1 
> }         kill "$pid"
> }         stderr="$(cat <&p)"
> } 
> } You'll notice the very ugly sleep+kill hack I had to use as I could not
> } find how you can close a coprocess's standard input cleanly.  Removing
> } the sleep+kill makes the cat <&p hang forever.
> 
> You need this:  http://www.zsh.org/mla/users/2011/msg00095.html  :-)
> 
Hi,

I'm using zsh 5.0, but looks like still cannot close the prior coproc
with "coproc exit":

% coproc while read line;do print -r -- "$line:u";done
[1] 31518
% coproc exit
[2] 31519
%
[2]  + done       exit
% jobs
[1]  + running    while read line; do; print -r -- "$line:u"; done
%

or I'm missing something here?

Thanks.

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Bart Schaefer]

On Sep 1,  7:59pm, Han Pingtian wrote:
}
} I'm using zsh 5.0, but looks like still cannot close the prior coproc
} with "coproc exit":
} 
} % coproc while read line;do print -r -- "$line:u";done

This is a bug/misfeature that I think I've mentioned before (possibly
only on zsh-workers) which affects built-in commands including loop
constructs.

The trouble is that zsh opens the coproc descriptors in the parent
shell (as it must) and marks them close-on-exec.  It then forks for
the actual coprocess job and dups the coproc descriptors to the job's
stdin and stdout.

Then the bug happens:  Because the coprocess is a built-in, there is
no exec, so the original coproc descriptors are never closed.  This
leaves the coprocess "talking to itself":

zsh     17133 schaefer    0r  FIFO    0,7          2256774 pipe
zsh     17133 schaefer    1w  FIFO    0,7          2256773 pipe
zsh     17133 schaefer    2u   CHR  136,2                4 /dev/pts/2
zsh     17133 schaefer   10u   CHR  136,2                4 /dev/pts/2
zsh     17133 schaefer   11r  FIFO    0,7          2256773 pipe
zsh     17133 schaefer   14w  FIFO    0,7          2256774 pipe

Note that 14w is connected to 0r and 11r is connected to 1w, all in
the same process.  IIRC exactly the same thing happens in at least
some revisions of ksh if you use a built-in construct as a coproc.

The workaround for now is to close the coproc inside the coproc:

% coproc { coproc exit; while read line;do print -r -- "$line:u";done }

Re: Is it possible to capture stdout and stderr to separate variables in Zsh?

[Paul Maisano]

Nikolai Weibull <now <at> bitwi.se> writes:

> 
> Hi!
> 
> Is it possible to capture stdout and stderr to separate variables in Zsh?
> 
> I understand that it’s not possible in sh, but I was wondering if any
> of Zsh’s additions in the redirection area would allow for such a
> separation.

Try this:

var1=`cmd` 2>&1 | read var2

Now var1 contains stdout from cmd and var2 contains stderr!

It might work with other shells too.

Regards,

Paul Maisano