Re: piping question

Re: piping question

[Christian Neukirchen]

Bart Schaefer <schaefer@brasslantern.com> writes:

> On Oct 3,  4:42pm, Mikael Magnusson wrote:
> }
> } echo hi there | { xterm -e 'most <& 7' 7< <(cat) }
>
> I avoided that because I was concerned that some terminal emulators
> would close all the descriptors above 2 when launching the command.

Unfortunately, all these hacks don't work with urxvt.  If anyone has an
idea how to do it, I'd like to see.

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: piping question

[Bart Schaefer]

On Oct 4,  5:50pm, Christian Neukirchen wrote:
} Subject: Re: piping question
}
} Bart Schaefer <schaefer@brasslantern.com> writes:
} 
} > On Oct 3,  4:42pm, Mikael Magnusson wrote:
} > }
} > } echo hi there | { xterm -e 'most <& 7' 7< <(cat) }
} >
} > I avoided that because I was concerned that some terminal emulators
} > would close all the descriptors above 2 when launching the command.
} 
} Unfortunately, all these hacks don't work with urxvt.  If anyone has an
} idea how to do it, I'd like to see.

This appears to be a combination of two problems:

(1) The name used by <(cat) is platform-specific.  E.g., on Debian-
    based systems, it matches /proc/self/fd/<-> which means it is
    interpreted relative to the current process, and "most" is two
    levels removed from zsh.  On some other systems it's a named
    pipe, which works independent of /proc/self/ and is why it was
    the first thing I suggested.

(2) As I feared, rxvt closes all the descriptors except 0,1,2 which
    it opened, making /proc/self/fd/<3-> all return EOF.

To avoid this, you have to hand-code /proc/$$/fd/ instead of allowing
zsh to create the /self/ name.  The exact path may vary by platform.

Also because rxvt executes the -e argument list directly rather than
going through a shell, you don't need/want to re-quote arguments, but
you also can't redirect input.

So this should do it (options to set geometry etc. may be added):

    xmost() {
      if [[ -t 0 ]]
      then
        rxvt -e most "$@"
      else
        local procself=/proc/self
        procself=${procself:A}  # Resolve symlink to actual PID
        local stdin
        exec {stdin}<&0
        rxvt -e most "$@" $procself/fd/$stdin
	exec {stdin}<&-
      fi
    }

I don't know about "most" so I didn't attempt to get the options
right, but for "less" you need to add the -f option to be able to
read the fifo.  Also note that if rxvt is put in the background,
you need to avoid closing $stdin until after it finishes, so the
best way to make this asynchronous is to add "&" after the "fi".

-- 
Barton E. Schaefer

Re: piping question

[Christian Neukirchen]

Bart Schaefer <schaefer@brasslantern.com> writes:

>     xmost() {
>       if [[ -t 0 ]]
>       then
>         rxvt -e most "$@"
>       else
>         local procself=/proc/self
>         procself=${procself:A}  # Resolve symlink to actual PID
>         local stdin
>         exec {stdin}<&0
>         rxvt -e most "$@" $procself/fd/$stdin
> 	exec {stdin}<&-
>       fi
>     }
>
> I don't know about "most" so I didn't attempt to get the options
> right, but for "less" you need to add the -f option to be able to
> read the fifo.  Also note that if rxvt is put in the background,
> you need to avoid closing $stdin until after it finishes, so the
> best way to make this asynchronous is to add "&" after the "fi".

This works great with less -f, thanks!

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: piping question

[Bart Schaefer]

On Oct 4,  9:29pm, Christian Neukirchen wrote:
} Subject: Re: piping question
}
} Bart Schaefer <schaefer@brasslantern.com> writes:
} 
} >         local procself=/proc/self
} >         procself=${procself:A}  # Resolve symlink to actual PID
} >         local stdin
} >         exec {stdin}<&0
} >         rxvt -e most "$@" $procself/fd/$stdin
} >         exec {stdin}<&-
} 
} This works great with less -f, thanks!

Beware that whatever is on the left side of the pipe may not see EOF
when "less" exits, so you may need to arrange to interrupt it some
other way.

For who may be interested, the parent zsh is holding open the file
descriptor that serves as the read end of the pipe.  This happens only
if you backgroud the entire pipeline, e.g.

   cat /dev/zero | xmost &

The pipe here stays open in the parent shell until the whole job is done,
and trying to close it with <&- gives e.g.

  file descriptor 11 used by shell, not closed

So I think we still have a few bugs along the lines of workers/32171 and
32176 and perhaps 31919.

PATCH Re: piping question

[Bart Schaefer]

On Oct 4,  2:01pm, Bart Schaefer wrote:
}
} For who may be interested, the parent zsh is holding open the file
} descriptor that serves as the read end of the pipe.  This happens only
} if you backgrou[n]d the entire pipeline, e.g.
} 
}    cat /dev/zero | xmost &
} 
} So I think we still have a few bugs along the lines of workers/32171 and
} 32176 and perhaps 31919.

This seems to take care of it.  PWS, should I push before 5.0.7?

diff --git a/Src/jobs.c b/Src/jobs.c
index 83a4d96..bd95afb 100644
--- a/Src/jobs.c
+++ b/Src/jobs.c
@@ -1627,8 +1627,10 @@ spawnjob(void)
     }
     if (!hasprocs(thisjob))
 	deletejob(jobtab + thisjob, 0);
-    else
+    else {
 	jobtab[thisjob].stat |= STAT_LOCKED;
+	pipecleanfilelist(jobtab[thisjob].filelist);
+    }
     thisjob = -1;
 }
 
diff --git a/Test/A05execution.ztst b/Test/A05execution.ztst
index 8d256ff..ca97f4f 100644
--- a/Test/A05execution.ztst
+++ b/Test/A05execution.ztst
@@ -217,6 +217,21 @@ F:This similar test was triggering a reproducible failure with pipestatus.
 F:This test checks for a file descriptor leak that could cause the left
 F:side of a pipe to block on write after the right side has exited
 
+  { setopt MONITOR } 2>/dev/null
+  if [[ -o MONITOR ]]
+  then
+   ( while :; do print "This is a line"; done ) | () : &
+   sleep 1
+   jobs -l
+  else
+   print -u $ZTST_fd "Skipping pipe leak test, requires MONITOR option"
+   print "[0] 0 0"
+  fi
+0:Bug regression: piping to anonymous function; piping to backround function
+*>\[<->\] <-> <->
+F:This test checks for two different bugs, a parser segfault piping to an
+F:anonymous function, and a descriptor leak when backgrounding a pipeline
+
   print "autoload_redir() { print Autoloaded ksh style; } >autoload.log" >autoload_redir
   autoload -Uk autoload_redir
   autoload_redir

Re: PATCH Re: piping question

[Peter Stephenson]

On Sat, 04 Oct 2014 21:03:20 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> This seems to take care of it.  PWS, should I push before 5.0.7?

That looks worth having.

pws

Re: piping question

[Mikael Magnusson]

On 4 October 2014 17:50, Christian Neukirchen <chneukirchen@gmail.com> wrote:
> Bart Schaefer <schaefer@brasslantern.com> writes:
>
>> On Oct 3,  4:42pm, Mikael Magnusson wrote:
>> }
>> } echo hi there | { xterm -e 'most <& 7' 7< <(cat) }
>>
>> I avoided that because I was concerned that some terminal emulators
>> would close all the descriptors above 2 when launching the command.
>
> Unfortunately, all these hacks don't work with urxvt.  If anyone has an
> idea how to do it, I'd like to see.

It works fine for me, the only difference is that it doesn't
automatically invoke the shell for you if you pass something that
isn't an array to pass to execv;
echo hi there | { urxvt -e sh -c 'most <& 7' 7< <(cat) }

-- 
Mikael Magnusson

Re: piping question

[Christian Neukirchen]

Mikael Magnusson <mikachu@gmail.com> writes:

> On 4 October 2014 17:50, Christian Neukirchen <chneukirchen@gmail.com> wrote:
>> Bart Schaefer <schaefer@brasslantern.com> writes:
>>
>>> On Oct 3,  4:42pm, Mikael Magnusson wrote:
>>> }
>>> } echo hi there | { xterm -e 'most <& 7' 7< <(cat) }
>>>
>>> I avoided that because I was concerned that some terminal emulators
>>> would close all the descriptors above 2 when launching the command.
>>
>> Unfortunately, all these hacks don't work with urxvt.  If anyone has an
>> idea how to do it, I'd like to see.
>
> It works fine for me, the only difference is that it doesn't
> automatically invoke the shell for you if you pass something that
> isn't an array to pass to execv;
> echo hi there | { urxvt -e sh -c 'most <& 7' 7< <(cat) }

You are right!  I'm down to

xless() {
  {
    exec {stdin}<&0
    exec urxvt -e sh -c "less ${(qq)@} <&$stdin"
  } &!
}

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: piping question

[Bart Schaefer]

On Oct 5,  2:17pm, Mikael Magnusson wrote:
} Subject: Re: piping question
}
} On 4 October 2014 17:50, Christian Neukirchen <chneukirchen@gmail.com> wrote:
} > Bart Schaefer <schaefer@brasslantern.com> writes:
} >
} >> On Oct 3,  4:42pm, Mikael Magnusson wrote:
} >> }
} >> } echo hi there | { xterm -e 'most <& 7' 7< <(cat) }
} >>
} >> I avoided that because I was concerned that some terminal emulators
} >> would close all the descriptors above 2 when launching the command.
} >
} > Unfortunately, all these hacks don't work with urxvt.  If anyone has an
} > idea how to do it, I'd like to see.
} 
} It works fine for me


Curious.  It does NOT work for me -- rxvt v2.6.4 on ubuntu 12.04.5 --
and "strace -ff" shows rxvt closing file descriptors from 3 through
something over 300 (I got tired of watching the output scroll by).

Re: piping question

[Mikael Magnusson]

On 5 October 2014 19:04, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Oct 5,  2:17pm, Mikael Magnusson wrote:
> } Subject: Re: piping question
> }
> } On 4 October 2014 17:50, Christian Neukirchen <chneukirchen@gmail.com> wrote:
> } > Bart Schaefer <schaefer@brasslantern.com> writes:
> } >
> } >> On Oct 3,  4:42pm, Mikael Magnusson wrote:
> } >> }
> } >> } echo hi there | { xterm -e 'most <& 7' 7< <(cat) }
> } >>
> } >> I avoided that because I was concerned that some terminal emulators
> } >> would close all the descriptors above 2 when launching the command.
> } >
> } > Unfortunately, all these hacks don't work with urxvt.  If anyone has an
> } > idea how to do it, I'd like to see.
> }
> } It works fine for me
>
>
> Curious.  It does NOT work for me -- rxvt v2.6.4 on ubuntu 12.04.5 --
> and "strace -ff" shows rxvt closing file descriptors from 3 through
> something over 300 (I got tired of watching the output scroll by).

These days urxvt hardly shares any code with rxvt, the fork happened
over ten years ago.

-- 
Mikael Magnusson