Redirection bug for stderr?

Redirection bug for stderr?

[Pierre Neidhardt]

Hi!

I recently noticed a somewhat unexpected behaviour with a configuration-less zsh 5.0.7:

	 $ ls /root >/dev/null 2>&1
   $ ls /root 2>/dev/null | cat
   $ ls /root >/dev/null 2>&1 | cat
	 ls: cannot open directory /root: Permission denied

I believe last error line should not appear. The problem is about stream
redirection when piped. `ls /root` is just an example of command printing to
stderr.

On Arch Linux, zsh is built with:

	./configure --prefix=/usr \
		--docdir=/usr/share/doc/zsh \
		--htmldir=/usr/share/doc/zsh/html \
		--enable-etcdir=/etc/zsh \
		--enable-zshenv=/etc/zsh/zshenv \
		--enable-zlogin=/etc/zsh/zlogin \
		--enable-zlogout=/etc/zsh/zlogout \
		--enable-zprofile=/etc/zsh/zprofile \
		--enable-zshrc=/etc/zsh/zshrc \
		--enable-maildir-support \
		--with-term-lib='ncursesw' \
		--enable-multibyte \
		--enable-function-subdirs \
		--enable-fndir=/usr/share/zsh/functions \
		--enable-scriptdir=/usr/share/zsh/scripts \
		--with-tcsetpgrp \
		--enable-pcre \
		--enable-cap \
		--enable-zsh-secure-free

Any clue?

-- 
Pierre Neidhardt

Coming together is a beginning;
	keeping together is progress;
		working together is success.

Re: Redirection bug for stderr?

[Peter Stephenson]

On Thu, 20 Nov 2014 11:44:40 +0100
Pierre Neidhardt <ambrevar@gmail.com> wrote:
> I recently noticed a somewhat unexpected behaviour with a configuration-less zsh 5.0.7:
> 
> 	 $ ls /root >/dev/null 2>&1
>    $ ls /root 2>/dev/null | cat
>    $ ls /root >/dev/null 2>&1 | cat
> 	 ls: cannot open directory /root: Permission denied
> 
> I believe last error line should not appear.

I suspect you're hitting this.

3.26: Why is my output duplicated with `foo 2>&1 >foo.out | bar'?

  This is a slightly unexpected effect of the option MULTIOS, which is
  set by default.  Let's look more closely:

    foo 2>&1 >foo.out | bar

  What you're probably expecting is that the command `foo' sends its
  standard output to the pipe and so to the input of the command `bar',
  while it sends its standard error to the file `foo.out'.  What you
  actually see is that the output is going both to the pipe and into the
  file.  To be more explicit, here's the same example with real commands:

    % { print output; print error >&2 } 2>&1 >foo.out | sed 's/error/erratic'
    erratic
    output
    % cat foo.out
    output

  and you can see `output' appears twice.

  It becomes clearer what's going on if we write:

    % print output >foo1.out >foo2.out
    % cat foo1.out
    output
    % cat foo2.out
    output

  You might recognise this as a standard feature of zsh, called `multios'
  and controlled by the option of the same name, whereby output is copied
  to both files when the redirector appears twice.  What's going on in the
  first example is exactly the same, however the second redirector is
  disguised as a pipe.  So if you want to turn this effect off, you need
  to unset the option `MULTIOS'.

Re: Redirection bug for stderr?

[Stephane Chazelas]

2014-11-20 16:08:16 +0000, Peter Stephenson:
[...]
>     % { print output; print error >&2 } 2>&1 >foo.out | sed 's/error/erratic'
[...]
>   So if you want to turn this effect off, you need
>   to unset the option `MULTIOS'.
> 

Or:


% { print output; print error >&2 } 2>&1 >&- >foo.out | sed 's/error/erratic/'


(note the typo above (missing / in s/foo/bar).

-- 
Stephane

Re: Redirection bug for stderr?

[Pierre Neidhardt]

OK, thanks for the explanation! :)

On 14-11-20 16:58:59, Stephane Chazelas wrote:
> 2014-11-20 16:08:16 +0000, Peter Stephenson:
> [...]
> >     % { print output; print error >&2 } 2>&1 >foo.out | sed 's/error/erratic'
> [...]
> >   So if you want to turn this effect off, you need
> >   to unset the option `MULTIOS'.
> > 
> 
> Or:
> 
> 
> % { print output; print error >&2 } 2>&1 >&- >foo.out | sed 's/error/erratic/'
> 
> 
> (note the typo above (missing / in s/foo/bar).
> 
> -- 
> Stephane
> 

-- 
Pierre Neidhardt

Ma Bell is a mean mother!

[PATCH][DOC] Re: Redirection bug for stderr?

[Stephane Chazelas]

2014-11-20 16:58:59 +0000, Stephane Chazelas:
[...]
> Or:
>
> % { print output; print error >&2 } 2>&1 >&- >foo.out | sed 's/error/erratic/'
[...]

Here as a patch. Also adding a missing "hideval" in ${(t)var}
description (unrelated).


diff --git a/Doc/Zsh/expn.yo b/Doc/Zsh/expn.yo
index a0478e7..f1334e3 100644
--- a/Doc/Zsh/expn.yo
+++ b/Doc/Zsh/expn.yo
@@ -1061,6 +1061,9 @@ for arrays which keep only the first occurrence of duplicated values
 item(tt(hide))(
 for parameters with the `hide' flag
 )
+item(tt(hideval))(
+for parameters with the `hideval' flag
+)
 item(tt(special))(
 for special parameters defined by the shell
 )
diff --git a/Etc/FAQ.yo b/Etc/FAQ.yo
index 7d4b86e..eaffe71 100644
--- a/Etc/FAQ.yo
+++ b/Etc/FAQ.yo
@@ -1784,7 +1784,14 @@ sect(Why is my output duplicated with `tt(foo 2>&1 >foo.out | bar)'?)
   to both files when the redirector appears twice.  What's going on in the
   first example is exactly the same, however the second redirector is
   disguised as a pipe.  So if you want to turn this effect off, you need
-  to unset the option mytt(MULTIOS).
+  to unset the option mytt(MULTIOS) or write it:
+  verb(
+    % { print output; print error >&2 } 2>&1 >&- >foo.out | sed 's/error/erratic/'
+    erratic
+  )
+  By closing stdout with tt(>&-), we're cancelling the previous redirections
+  (to the pipe) and start anew with tt(>foo.out) instead of adding it as a
+  redirection target to stdout.
 
 
 sect(What are these `^' and `~' pattern characters, anyway?)