* Two bug reports: alias/function disagreements; incorrect redirection @ 2002-03-22 12:24 David Hughes 2002-03-22 12:41 ` Peter Stephenson 0 siblings, 1 reply; 11+ messages in thread From: David Hughes @ 2002-03-22 12:24 UTC (permalink / raw) To: zsh-workers These appear to be long-standing problems (since v2.6 at least), and they're still around as of v4.0.4, and they're not in the known-bugs FAQ, and they seem to be platform-independent. Bug I ===== Bad Things happen if I assign a name to an alias and then to a function. $ which echo echo: shell built-in command $ alias foo='echo bar' $ foo () { echo baz } $ which echo echo () { echo baz } Of course Very Bad Things now happen if I try to use `echo' or `foo'. Bug II ====== $ /bin/sh -c 'echo out; echo err >&2' 2>&1 >/dev/null err $ /bin/sh -c 'echo out; echo err >&2' 2>&1 >/dev/null | cat err out I have reproduced both of these bugs on these platforms: RedHat 7.2 GNU/Linux (heavily CERN-customised), zsh 4.0.4 RedHat 7.1 GNU/Linux, zsh 3.1.6-dev-22 SunOS, zsh 2.6-beta4 HP-UX, zsh 2.6-beta21 // David -- David Hughes UNIX sysadmin, Serco SA -+- Tel.: +41 22 767 4047 Computing Centre, CERN -+- David.W.Hughes@cern.ch +++ A good bug report is an expression of love. This message expresses my own opinions and should not be construed as the opinions of Serco (who employ me) or of CERN (where I work). ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Two bug reports: alias/function disagreements; incorrect redirection 2002-03-22 12:24 Two bug reports: alias/function disagreements; incorrect redirection David Hughes @ 2002-03-22 12:41 ` Peter Stephenson 2002-03-23 21:18 ` Piping stderr (was Re: Two bug reports) Wayne Davison 0 siblings, 1 reply; 11+ messages in thread From: Peter Stephenson @ 2002-03-22 12:41 UTC (permalink / raw) To: David Hughes, Zsh hackers list David Hughes wrote: > Bug I > ===== > > Bad Things happen if I assign a name to an alias and then to a function. > > $ which echo > echo: shell built-in command > $ alias foo='echo bar' > $ foo () { echo baz } > $ which echo > echo () { > echo baz > } > > Of course Very Bad Things now happen if I try to use `echo' or `foo'. This is inconvenient but is already mentioned in the FAQ (http://zsh.sunsite.dk/FAQ/), section 2.3. There is one other serious problem with aliases: consider alias l='/bin/ls -F' l() { /bin/ls -la "$@" | more } l in the function definition is in command position and is expanded as an alias, defining /bin/ls and -F as functions which call /bin/ls, which gets a bit recursive. This can be avoided if you use function to define a function, which doesn't expand aliases. It is possible to argue for extra warnings somewhere in this mess. Luckily, it is not possible to define function as an alias. (The last sentence is a lie, unfortunately, and I must get around to changing it.) > Bug II > ====== > > $ /bin/sh -c 'echo out; echo err >&2' 2>&1 >/dev/null > err > $ /bin/sh -c 'echo out; echo err >&2' 2>&1 >/dev/null | cat > err > out This is not a bug, and if you are in sh-compatibility mode it doesn't happen. From the FAQ again, 3.26: Why is my output duplicated with `foo 2>&1 >foo.out | bar'? This is a slightly unexpected effect of the option MULTIOS, which is set by default. Let's look more closely: foo 2>&1 >foo.out | bar What you're probably expecting is that the command foo sends its standard output to the pipe and so to the input of the command bar, while it sends its standard error to the file foo.out. What you actually see is that the output is going both to the pipe and into the file. To be more explicit, here's the same example with real commands: % { print output; print error >&2 } 2>&1 >foo.out | sed 's/error/erratic' erratic output % cat foo.out output and you can see `output' appears twice. It becomes clearer what's going on if we write: % print output >foo1.out >foo2.out % cat foo1.out output % cat foo2.out output You might recognise this as a standard feature of zsh, called `multios' and controlled by the option of the same name, whereby output is copied to both files when the redirector appears twice. What's going on in the first example is exactly the same, however the second redirector is disguised as a pipe. So if you want to turn this effect off, you need to unset the option MULTIOS. -- Peter Stephenson <pws@csr.com> Software Engineer CSR Ltd., Science Park, Milton Road, Cambridge, CB4 0WH, UK Tel: +44 (0)1223 392070 ********************************************************************** The information transmitted is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer. ********************************************************************** ^ permalink raw reply [flat|nested] 11+ messages in thread
* Piping stderr (was Re: Two bug reports) 2002-03-22 12:41 ` Peter Stephenson @ 2002-03-23 21:18 ` Wayne Davison 2002-03-23 22:45 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Wayne Davison @ 2002-03-23 21:18 UTC (permalink / raw) To: Zsh hackers list I'm wondering if it would be nice to create a syntax for piping just stderr? We already have "|&" for piping both stdout and stderr, so perhaps we could extend this somehow? What do you think about using "2|&"? Or maybe "2>|&" (which is probably safer). Do any other shells have a syntax for this? ..wayne.. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Piping stderr (was Re: Two bug reports) 2002-03-23 21:18 ` Piping stderr (was Re: Two bug reports) Wayne Davison @ 2002-03-23 22:45 ` Bart Schaefer 2002-03-23 23:47 ` Multios again (Re: Piping stderr (was Re: Two bug reports)) Bart Schaefer 2002-03-24 9:01 ` Piping stderr (was Re: Two bug reports) Wayne Davison 0 siblings, 2 replies; 11+ messages in thread From: Bart Schaefer @ 2002-03-23 22:45 UTC (permalink / raw) To: Wayne Davison, Zsh hackers list On Mar 23, 1:18pm, Wayne Davison wrote: } } I'm wondering if it would be nice to create a syntax for piping just } stderr? You mean, other than `2>>(...)'? } We already have "|&" for piping both stdout and stderr Hmm. `|&' is short for `2>&1 |'. What would the new syntax be short for? The problem being that in order to pipe "just stderr" you have to dispose of stdout somehow. The proposed new syntax could be short for `2>&1 >&- |' but then you may get "bad file descriptor" because of the closed stdout. Or it could be short for `2>&1 >/dev/null |' except that in the case of MULTIOS that doesn't redirect stdout, it duplicates it to both /dev/null and the pipe; we'd at the least have to document as we do for `>&' that the effect of MULTIOS is subverted. Plus, some platforms don't have a /dev/null (zsh does compile for DOS, sort of). The same problem with MULTIOS occurs with `3>&2 2>&1 1>&3 |'. } What do you think about using "2|&"? Or maybe "2>|&" Of those, it'd have to be the latter, which is currently a parse error. The issue is really one of syntactic cleanliness; `|', `&', and `|&' are all command separators, whereas redirections such as `2>&1' are part of the command itself. That's why `2>&1 |' works differently than `2>&1 >'. The grammar gets a bit ugly if you make something that starts with a digit into a command separator. } Do any other shells have a syntax for this? Not that I know of. -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 11+ messages in thread
* Multios again (Re: Piping stderr (was Re: Two bug reports)) 2002-03-23 22:45 ` Bart Schaefer @ 2002-03-23 23:47 ` Bart Schaefer 2002-03-24 9:01 ` Piping stderr (was Re: Two bug reports) Wayne Davison 1 sibling, 0 replies; 11+ messages in thread From: Bart Schaefer @ 2002-03-23 23:47 UTC (permalink / raw) To: Zsh hackers list On Mar 23, 10:45pm, Bart Schaefer wrote: } } The same problem with MULTIOS occurs with `3>&2 2>&1 1>&3 |'. Given that we have the >(...) syntax, I'm wondering there should be a way to allow multios for redirections but not for pipes. That would mean that in `command >file1 >file2 | prog', file1 and file2 would get output but the pipe would get nothing. That way, `command 2>&1 >file | prog' would work the way it does in other shells, and if you really mean for `prog' to get a copy of the input you can write `command 2>&1 >file >>(prog)' instead. It further would mean that `command 2>>(prog1) >file | prog2' would send ONLY the output of `prog1' through `prog2', which seems to be partly what Wayne was getting at with his question. Various weird tidbits while I'm on the subject: Using `1>&1' creates some pretty strange effects; it duplicates stdout 2**N times (where N is the number of uses of `1>&1'). Let's look at a few examples; the first two just set up the context. zsh% (print -u2 foo; print bar) >>(tr a-z A-Z) foo zsh% BAR We know that the >(...) runs asynchronously, so that's fine. zsh% (print -u2 foo; print bar) >>(tr a-z A-Z) | wc foo 2 2 8 Shows that the stdout of the >(...) expression also goes to the pipe. That implies a simple way to make >(...) act semi-synchronously: pipe the whole command to `cat'. Now for the strange stuff: zsh% (print -u2 foo; print bar) 1>&1 >>(tr a-z A-Z) | cat foo bar bar BAR So `cat' got two copies of the original stdout and >(...) got one. Yet: zsh% (print -u2 foo; print bar) >>(tr a-z A-Z) 1>&1 | cat foo bar bar BAR BAR Here, 1>&1 duplicated the input to >(...) as well as to the pipe. At first I thought it had somehow duplicated the output of >(...), but: zsh% (print -u2 foo; print bar) 2>>(tr a-z A-Z) 1>&1 | cat bar bar FOO Now for the real silliness: zsh% (print -u2 foo; print bar) 1>&1 >>(tr a-z A-Z) 1>&1 | cat foo bar bar bar bar BAR BAR Can this really be the way it's supposed to work? -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Piping stderr (was Re: Two bug reports) 2002-03-23 22:45 ` Bart Schaefer 2002-03-23 23:47 ` Multios again (Re: Piping stderr (was Re: Two bug reports)) Bart Schaefer @ 2002-03-24 9:01 ` Wayne Davison 2002-03-24 16:47 ` Borsenkow Andrej 1 sibling, 1 reply; 11+ messages in thread From: Wayne Davison @ 2002-03-24 9:01 UTC (permalink / raw) To: Bart Schaefer; +Cc: Zsh hackers list On Sat, 23 Mar 2002, Bart Schaefer wrote: > On Mar 23, 1:18pm, Wayne Davison wrote: > } I'm wondering if it would be nice to create a syntax for piping just > } stderr? > > You mean, other than `2>>(...)'? Uh ... Yeah, Yeah. That's what I meant. _Other_ than that. :-) (I had apparently neglected to find that syntax when I was searching the zsh man page.) So, I suppose that Peter's reply to David Hughes could be ammended to say that in addition to turning off multios, he could change this command: /bin/sh -c 'echo out; echo err >&2' 2>&1 >/dev/null | cat into this one: /bin/sh -c 'echo out; echo err >&2' >/dev/null 2>>(cat) The bad thing about this idiom is that it doesn't wait for "cat" to finish before moving on. If we implemented my proposed stderr-pipe syntax, the following command wouldn't have that problem: /bin/sh -c 'echo out; echo err >&2' >/dev/null 2>|& cat > Hmm. `|&' is short for `2>&1 |'. What would the new syntax be short > for? It wouldn't be short for anything. Instead "|" would become short for "1>|&". The construct simply means "take the numbered file handle and pipe it to stdin of the following command." > The problem being that in order to pipe "just stderr" you have > to dispose of stdout somehow. No, stdout doesn't need to be affected at all. If the user wants stdout to go somewhere other than the default, it can be redirected explicitly (as I did above). ..wayne.. ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Piping stderr (was Re: Two bug reports) 2002-03-24 9:01 ` Piping stderr (was Re: Two bug reports) Wayne Davison @ 2002-03-24 16:47 ` Borsenkow Andrej 2002-03-24 19:02 ` Bart Schaefer 0 siblings, 1 reply; 11+ messages in thread From: Borsenkow Andrej @ 2002-03-24 16:47 UTC (permalink / raw) To: Zsh hackers list В Вск, 24.03.2002, в 12:01, Wayne Davison написал: > > > The problem being that in order to pipe "just stderr" you have > > to dispose of stdout somehow. > > No, stdout doesn't need to be affected at all. If the user wants stdout > to go somewhere other than the default, it can be redirected explicitly > (as I did above). > That makes sense. More than once I wished to pipe stderr while leaving stdout alone. -andrej ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Piping stderr (was Re: Two bug reports) 2002-03-24 16:47 ` Borsenkow Andrej @ 2002-03-24 19:02 ` Bart Schaefer 2002-03-24 19:09 ` Bart Schaefer ` (2 more replies) 0 siblings, 3 replies; 11+ messages in thread From: Bart Schaefer @ 2002-03-24 19:02 UTC (permalink / raw) To: Zsh hackers list On Mar 24, 1:01am, Wayne Davison wrote: } Subject: Re: Piping stderr (was Re: Two bug reports) } } On Sat, 23 Mar 2002, Bart Schaefer wrote: } > } > You mean, other than `2>>(...)'? } } (I had apparently neglected to find that syntax when I was searching } the zsh man page.) The man page could stand improvement in this area. Even though pipes are not redirections, people tend to think of them that way, and it's very confusing that >(...) and |& are not at least cross-referenced in the same section as all the redirection operators. Even `|&' is buried in the middle of a boring-looking paragraph in the Shell Grammar section, and never metioned again. } So, I suppose that Peter's reply to David Hughes could be ammended to } } /bin/sh -c 'echo out; echo err >&2' >/dev/null 2>>(cat) } } The bad thing about this idiom is that it doesn't wait for "cat" to } finish before moving on. That's a separate bug that I've been complaining about for a while. I really think zsh should be tracking >(...) as a job and waiting for it rather than treating it as if disowned. That would be a much cleaner solution from the viewpoint of the shell language (though possibly a bit messier internally) than would adding a new pipe syntax. As I pointed out in a follow-up you can work around it by doing an extra pipe: /bin/sh -c 'echo out; echo err >&2' >/dev/null 2>>(cat) | cat In fact (hand smacks forehead), just add braces: { /bin/sh -c 'echo out; echo err >&2' >/dev/null 2>>(cat) } } > The problem being that in order to pipe "just stderr" you have } > to dispose of stdout somehow. } } No, stdout doesn't need to be affected at all. If the user wants stdout } to go somewhere other than the default, it can be redirected explicitly } (as I did above). Leaving stdout connected to the parent's stdout is still "disposing of" stdout; it's equivalent to: zsh% 3>&1 { { /bin/sh -c 'echo out; echo err >&2' >&3 } |& tr a-z A-Z } out ERR (The placement of `3>&1' there is illustrative, it could just as easily go after the last close brace.) You can always defeat multios by adding braces. -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Piping stderr (was Re: Two bug reports) 2002-03-24 19:02 ` Bart Schaefer @ 2002-03-24 19:09 ` Bart Schaefer 2002-03-24 19:50 ` PATCH: (Doc) " Bart Schaefer 2002-03-25 0:17 ` Wayne Davison 2 siblings, 0 replies; 11+ messages in thread From: Bart Schaefer @ 2002-03-24 19:09 UTC (permalink / raw) To: Zsh hackers list On Mar 24, 7:02pm, Bart Schaefer wrote: } } In fact (hand smacks forehead), just add braces: } } { /bin/sh -c 'echo out; echo err >&2' >/dev/null 2>>(cat) } Oops, sorry, that doesn't work unless you also redirect the whole brace expression somewhere. I confused myself. -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 11+ messages in thread
* PATCH: (Doc) Re: Piping stderr (was Re: Two bug reports) 2002-03-24 19:02 ` Bart Schaefer 2002-03-24 19:09 ` Bart Schaefer @ 2002-03-24 19:50 ` Bart Schaefer 2002-03-25 0:17 ` Wayne Davison 2 siblings, 0 replies; 11+ messages in thread From: Bart Schaefer @ 2002-03-24 19:50 UTC (permalink / raw) To: Zsh hackers list On Mar 24, 7:02pm, Bart Schaefer wrote: } } The man page could stand improvement in this area. Even though pipes are } not redirections, people tend to think of them that way, and it's very } confusing that >(...) and |& are not at least cross-referenced in the } same section as all the redirection operators. Even `|&' is buried in } the middle of a boring-looking paragraph in the Shell Grammar section, } and never metioned again. Anybody object to this? Index: Doc/Zsh/redirect.yo =================================================================== diff -c -r1.2 redirect.yo --- Doc/Zsh/redirect.yo 2001/07/10 09:05:18 1.2 +++ Doc/Zsh/redirect.yo 2002/03/24 19:45:25 @@ -138,6 +138,19 @@ file descriptor 2 would be associated with the terminal (assuming file descriptor 1 had been) and then file descriptor 1 would be associated with file var(fname). + +The `tt(|&)' command separator described in +ifzman(em(Simple Commands & Pipelines) in zmanref(zshmisc))\ +ifnzman(noderef(Simple Commands & Pipelines)) +is a shorthand for `tt(2>&1 |)'. + +For output redirections only, if var(word) is of the form +`tt(>LPAR())var(list)tt(RPAR())' then the output is piped to the command +represented by var(list). See +ifzman(\ +em(Process Substitution) in zmanref(zshexpn))\ +ifnzman(\ +noderef(Process Substitution)). sect(Multios) cindex(multios) pindex(MULTIOS, use of) -- Bart Schaefer Brass Lantern Enterprises http://www.well.com/user/barts http://www.brasslantern.com Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Piping stderr (was Re: Two bug reports) 2002-03-24 19:02 ` Bart Schaefer 2002-03-24 19:09 ` Bart Schaefer 2002-03-24 19:50 ` PATCH: (Doc) " Bart Schaefer @ 2002-03-25 0:17 ` Wayne Davison 2 siblings, 0 replies; 11+ messages in thread From: Wayne Davison @ 2002-03-25 0:17 UTC (permalink / raw) To: Bart Schaefer; +Cc: Zsh hackers list On Sun, 24 Mar 2002, Bart Schaefer wrote: > I really think zsh should be tracking >(...) as a job and waiting for > it rather than treating it as if disowned. Yes, that would be a good change, IMO. > That would be a much cleaner solution from the viewpoint of the shell > language (though possibly a bit messier internally) than would adding > a new pipe syntax. I agree. After looking at my proposed stderr-pipe syntax, I don't like it very much. So, using the existing 2>>(command) syntax is a good way to go, especially if zsh could be made to wait for the "command" to complete. ..wayne.. ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2002-03-25 0:18 UTC | newest] Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2002-03-22 12:24 Two bug reports: alias/function disagreements; incorrect redirection David Hughes 2002-03-22 12:41 ` Peter Stephenson 2002-03-23 21:18 ` Piping stderr (was Re: Two bug reports) Wayne Davison 2002-03-23 22:45 ` Bart Schaefer 2002-03-23 23:47 ` Multios again (Re: Piping stderr (was Re: Two bug reports)) Bart Schaefer 2002-03-24 9:01 ` Piping stderr (was Re: Two bug reports) Wayne Davison 2002-03-24 16:47 ` Borsenkow Andrej 2002-03-24 19:02 ` Bart Schaefer 2002-03-24 19:09 ` Bart Schaefer 2002-03-24 19:50 ` PATCH: (Doc) " Bart Schaefer 2002-03-25 0:17 ` Wayne Davison
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).