* subprocess @ 2020-06-05 2:03 Pier Paolo Grassi 2020-06-05 10:25 ` subprocess Mikael Magnusson 0 siblings, 1 reply; 8+ messages in thread From: Pier Paolo Grassi @ 2020-06-05 2:03 UTC (permalink / raw) To: Zsh-Users List [-- Attachment #1: Type: text/plain, Size: 258 bytes --] hello, can someone explain why (with zsh 5.1.1): efe() cat $1 efe <(seq 1 10) | wc -l cat: /proc/self/fd/13: No such file or directory 0 gives error, instead these all works fine: efe =(seq 1 10)| wc -l cat <(seq 1 10)| wc -l thanks -- Pier Paolo Grassi ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-05 2:03 subprocess Pier Paolo Grassi @ 2020-06-05 10:25 ` Mikael Magnusson 2020-06-06 1:00 ` subprocess Daniel Shahaf 0 siblings, 1 reply; 8+ messages in thread From: Mikael Magnusson @ 2020-06-05 10:25 UTC (permalink / raw) To: Pier Paolo Grassi; +Cc: Zsh-Users List On 6/5/20, Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > hello, can someone explain why (with zsh 5.1.1): Same in 5.8.0.2 > efe() cat $1 > efe <(seq 1 10) | wc -l > cat: /proc/self/fd/13: No such file or directory > 0 > > gives error, instead these all works fine: > efe =(seq 1 10)| wc -l > cat <(seq 1 10)| wc -l This also works, % efe <(seq 1 10) > >(wc -l) as does this, % wc -l < <(efe <(seq 1 10)) -- Mikael Magnusson ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-05 10:25 ` subprocess Mikael Magnusson @ 2020-06-06 1:00 ` Daniel Shahaf 2020-06-06 18:19 ` subprocess Bart Schaefer 0 siblings, 1 reply; 8+ messages in thread From: Daniel Shahaf @ 2020-06-06 1:00 UTC (permalink / raw) To: Mikael Magnusson; +Cc: Pier Paolo Grassi, Zsh-Users List Mikael Magnusson wrote on Fri, 05 Jun 2020 12:25 +0200: > On 6/5/20, Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > > hello, can someone explain why (with zsh 5.1.1): > > Same in 5.8.0.2 > > > efe() cat $1 > > efe <(seq 1 10) | wc -l > > cat: /proc/self/fd/13: No such file or directory > > 0 > > > > gives error, instead these all works fine: > > efe =(seq 1 10)| wc -l > > cat <(seq 1 10)| wc -l > > This also works, > % efe <(seq 1 10) > >(wc -l) > as does this, > % wc -l < <(efe <(seq 1 10)) And this, % () { cat $1 } <(seq 1 10) | wc -l ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-06 1:00 ` subprocess Daniel Shahaf @ 2020-06-06 18:19 ` Bart Schaefer 2020-06-06 19:53 ` subprocess Peter Stephenson 2020-06-08 14:22 ` subprocess Pier Paolo Grassi 0 siblings, 2 replies; 8+ messages in thread From: Bart Schaefer @ 2020-06-06 18:19 UTC (permalink / raw) To: Daniel Shahaf; +Cc: Mikael Magnusson, Pier Paolo Grassi, Zsh-Users List On Fri, Jun 5, 2020 at 6:01 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > On 6/5/20, Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > > > efe() cat $1 > > > efe <(seq 1 10) | wc -l > > > cat: /proc/self/fd/13: No such file or directory > > > 0 > > > > > > gives error, instead these all works fine: > > > efe =(seq 1 10)| wc -l > > > cat <(seq 1 10)| wc -l > > And this, > > % () { cat $1 } <(seq 1 10) | wc -l You can see what's happening if you do it this way: % efe() ls -l ${${1:+$1:h}:-/proc/self/fd} % efe <(seq 1 10) | cat total 0 lrwx------ 1 schaefer schaefer 64 Jun 6 10:59 0 -> /dev/pts/0 l-wx------ 1 schaefer schaefer 64 Jun 6 10:59 1 -> pipe:[1945962] lrwx------ 1 schaefer schaefer 64 Jun 6 10:59 2 -> /dev/pts/0 lr-x------ 1 schaefer schaefer 64 Jun 6 10:59 3 -> /proc/5533/fd % () { ls -l $1:h } <(seq 1 10) | cat total 0 lrwx------ 1 schaefer schaefer 64 Jun 6 11:00 0 -> /dev/pts/0 l-wx------ 1 schaefer schaefer 64 Jun 6 11:00 1 -> pipe:[1945988] lr-x------ 1 schaefer schaefer 64 Jun 6 11:00 11 -> pipe:[1944505] lrwx------ 1 schaefer schaefer 64 Jun 6 11:00 2 -> /dev/pts/0 lr-x------ 1 schaefer schaefer 64 Jun 6 11:00 3 -> /proc/5537/fd A named function on the left side of a pipeline becomes its own "job", where all descriptors above 10 are closed. An anonymous function stays in the parent shell (which is still a forked subshell, because of the pipeline) so those descriptors are not closed. % efe =(seq 1 10) | cat total 4 -rw------- 1 schaefer schaefer 21 Jun 6 11:09 zshkDpnRj In that case a real file has been created and is referenced by name, so the descriptors closing does not matter. % efe <<(seq 1 10) | cat total 0 lr-x------ 1 schaefer schaefer 64 Jun 6 11:13 0 -> pipe:[1946804] l-wx------ 1 schaefer schaefer 64 Jun 6 11:13 1 -> pipe:[1946579] lrwx------ 1 schaefer schaefer 64 Jun 6 11:13 2 -> /dev/pts/0 lr-x------ 1 schaefer schaefer 64 Jun 6 11:13 3 -> /proc/5599/fd For <<(...) no file is created at all, instead the standard input of the function is changed. > Mikael Magnusson wrote on Fri, 05 Jun 2020 12:25 +0200: > > > > This also works, > > % efe <(seq 1 10) > >(wc -l) > > as does this, > > % wc -l < <(efe <(seq 1 10)) In both of these cases, there is no pipeline, so efe stays in the parent shell and all the descriptors remain open. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-06 18:19 ` subprocess Bart Schaefer @ 2020-06-06 19:53 ` Peter Stephenson 2020-06-08 19:55 ` subprocess Peter Stephenson 2020-06-08 14:22 ` subprocess Pier Paolo Grassi 1 sibling, 1 reply; 8+ messages in thread From: Peter Stephenson @ 2020-06-06 19:53 UTC (permalink / raw) To: zsh-users On Sat, 2020-06-06 at 11:19 -0700, Bart Schaefer wrote: > On Fri, Jun 5, 2020 at 6:01 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > > On 6/5/20, Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > > > > efe() cat $1 > > > > efe <(seq 1 10) | wc -l > > > > cat: /proc/self/fd/13: No such file or directory > > A named function on the left side of a pipeline becomes its own "job", > where all descriptors above 10 are closed. Itʼs this call here. How ahout simply reordering so we tidy up after the function is called? That fixes the problem, hard to see how that can cause a leak...? pws diff --git a/Src/exec.c b/Src/exec.c index 2b8e2167f..53d3aa048 100644 --- a/Src/exec.c +++ b/Src/exec.c @@ -3969,8 +3969,8 @@ execcmd_exec(Estate state, Execcmd_params eparams, if (is_shfunc) { /* It's a shell function */ - pipecleanfilelist(filelist, 0); execshfunc((Shfunc) hn, args); + pipecleanfilelist(filelist, 0); } else { /* It's a builtin */ LinkList assigns = (LinkList)0; ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-06 19:53 ` subprocess Peter Stephenson @ 2020-06-08 19:55 ` Peter Stephenson 0 siblings, 0 replies; 8+ messages in thread From: Peter Stephenson @ 2020-06-08 19:55 UTC (permalink / raw) To: zsh-users On Sat, 2020-06-06 at 20:53 +0100, Peter Stephenson wrote: > On Sat, 2020-06-06 at 11:19 -0700, Bart Schaefer wrote: > > On Fri, Jun 5, 2020 at 6:01 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > > > On 6/5/20, Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > > > > > efe() cat $1 > > > > > efe <(seq 1 10) | wc -l > > > > > cat: /proc/self/fd/13: No such file or directory > > > > A named function on the left side of a pipeline becomes its own "job", > > where all descriptors above 10 are closed. > > Itʼs this call here. > > How ahout simply reordering so we tidy up after the function is called? > That fixes the problem, hard to see how that can cause a leak...? I've taken the silence for lack of controversy rather than stupefaction and submitted this. (For workers: I will assume the same for the CLOBBER_EMPTY change tomorrow. I already have a reply from Daniel.) pws ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-06 18:19 ` subprocess Bart Schaefer 2020-06-06 19:53 ` subprocess Peter Stephenson @ 2020-06-08 14:22 ` Pier Paolo Grassi 2020-06-08 21:32 ` subprocess Bart Schaefer 1 sibling, 1 reply; 8+ messages in thread From: Pier Paolo Grassi @ 2020-06-08 14:22 UTC (permalink / raw) To: Bart Schaefer; +Cc: Daniel Shahaf, Mikael Magnusson, Zsh-Users List [-- Attachment #1: Type: text/plain, Size: 2784 bytes --] thanks for the debugging technique and the explanation, very interesting. What would be the best way to learn more about these mechanisms without digging in the source code of the shell or the kernel? I am not very comfortable around c regards Il giorno sab 6 giu 2020 alle ore 20:19 Bart Schaefer < schaefer@brasslantern.com> ha scritto: > On Fri, Jun 5, 2020 at 6:01 PM Daniel Shahaf <d.s@daniel.shahaf.name> > wrote: > > > On 6/5/20, Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > > > > efe() cat $1 > > > > efe <(seq 1 10) | wc -l > > > > cat: /proc/self/fd/13: No such file or directory > > > > 0 > > > > > > > > gives error, instead these all works fine: > > > > efe =(seq 1 10)| wc -l > > > > cat <(seq 1 10)| wc -l > > > > And this, > > > > % () { cat $1 } <(seq 1 10) | wc -l > > You can see what's happening if you do it this way: > > % efe() ls -l ${${1:+$1:h}:-/proc/self/fd} > % efe <(seq 1 10) | cat > total 0 > lrwx------ 1 schaefer schaefer 64 Jun 6 10:59 0 -> /dev/pts/0 > l-wx------ 1 schaefer schaefer 64 Jun 6 10:59 1 -> pipe:[1945962] > lrwx------ 1 schaefer schaefer 64 Jun 6 10:59 2 -> /dev/pts/0 > lr-x------ 1 schaefer schaefer 64 Jun 6 10:59 3 -> /proc/5533/fd > % () { ls -l $1:h } <(seq 1 10) | cat > total 0 > lrwx------ 1 schaefer schaefer 64 Jun 6 11:00 0 -> /dev/pts/0 > l-wx------ 1 schaefer schaefer 64 Jun 6 11:00 1 -> pipe:[1945988] > lr-x------ 1 schaefer schaefer 64 Jun 6 11:00 11 -> pipe:[1944505] > lrwx------ 1 schaefer schaefer 64 Jun 6 11:00 2 -> /dev/pts/0 > lr-x------ 1 schaefer schaefer 64 Jun 6 11:00 3 -> /proc/5537/fd > > A named function on the left side of a pipeline becomes its own "job", > where all descriptors above 10 are closed. An anonymous function > stays in the parent shell (which is still a forked subshell, because > of the pipeline) so those descriptors are not closed. > > % efe =(seq 1 10) | cat > total 4 > -rw------- 1 schaefer schaefer 21 Jun 6 11:09 zshkDpnRj > > In that case a real file has been created and is referenced by name, > so the descriptors closing does not matter. > > % efe <<(seq 1 10) | cat > total 0 > lr-x------ 1 schaefer schaefer 64 Jun 6 11:13 0 -> pipe:[1946804] > l-wx------ 1 schaefer schaefer 64 Jun 6 11:13 1 -> pipe:[1946579] > lrwx------ 1 schaefer schaefer 64 Jun 6 11:13 2 -> /dev/pts/0 > lr-x------ 1 schaefer schaefer 64 Jun 6 11:13 3 -> /proc/5599/fd > > For <<(...) no file is created at all, instead the standard input of > the function is changed. > > > Mikael Magnusson wrote on Fri, 05 Jun 2020 12:25 +0200: > > > > > > This also works, > > > % efe <(seq 1 10) > >(wc -l) > > > as does this, > > > % wc -l < <(efe <(seq 1 10)) > > In both of these cases, there is no pipeline, so efe stays in the > parent shell and all the descriptors remain open. > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: subprocess 2020-06-08 14:22 ` subprocess Pier Paolo Grassi @ 2020-06-08 21:32 ` Bart Schaefer 0 siblings, 0 replies; 8+ messages in thread From: Bart Schaefer @ 2020-06-08 21:32 UTC (permalink / raw) To: Pier Paolo Grassi; +Cc: Daniel Shahaf, Mikael Magnusson, Zsh-Users List On Mon, Jun 8, 2020 at 7:22 AM Pier Paolo Grassi <pierpaolog@gmail.com> wrote: > > What would be the best way to learn more about these mechanisms without > digging in the source code of the shell or the kernel? Hmm. Digging in the source code probably isn't the best way even if you were interested. It's a tough question. This is all documented, but not in any single coherent place, mostly because there are just too many possible ways to combine things to explain all the variations. For example the closing of all descriptors >= 10 when starting a new job is long-standing, standardized shell behavior, so the zsh manual doesn't mention it, you're just expected to have learned (elsewhere) that is how shells work. I'm not sure the "named functions get their own job" part is explained very clearly anywhere, but it is explained that anonymous functions do not behave that way. (E.g., "Redirections may be applied to the anonymous function in the same manner as to a current-shell structure enclosed in braces.") It's explained that <(...) uses a special file representing a file descriptor ("If the system supports the /dev/fd mechanism, the command argument is the name of the device file corresponding to a file descriptor" -- that should probably mention /proc/self/fd as well) but does not warn that it might conflict with the "close all descriptors above 10" behavior. The upshot is that the best way to learn is probably to ask somebody. I don't think all these sorts of nuances are even covered by most books on shell programming. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-06-08 21:33 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-06-05 2:03 subprocess Pier Paolo Grassi 2020-06-05 10:25 ` subprocess Mikael Magnusson 2020-06-06 1:00 ` subprocess Daniel Shahaf 2020-06-06 18:19 ` subprocess Bart Schaefer 2020-06-06 19:53 ` subprocess Peter Stephenson 2020-06-08 19:55 ` subprocess Peter Stephenson 2020-06-08 14:22 ` subprocess Pier Paolo Grassi 2020-06-08 21:32 ` subprocess Bart Schaefer
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/zsh/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).