Why multios affects >/dev/tty redirection?

Why multios affects >/dev/tty redirection?

[Sebastian Gniazdowski]

Hello,
I have a command:

scrapy crawl 2nd "$@" "${optarray[@]}" 2> /tmp/reply  2> /dev/tty 1>
/tmp/reply 1> /dev/tty

When invoked this way, scrapy (a python project) spawns an interactive
console and it isn't fully usable  – e.g.: cursor keys emit escape
sequencess rather t than moving the cursor or browsing history.

However, when I invoke scrqpy in following way:

scrapy crawl 2nd "$@" "${optarray[@]}"  2> /dev/tty 1> /dev/tty

Then the python interactive console spawned by scrapy is working fully
correct – cursor keys are working (ii.e. they move the cursor and
browse the history), the same for Home/End keys, etc.

Is there a way around this? The actual command that I'm running uses
multios to redirect un-colorized messages to a log files:

scrapy crawl 2nd_Crawler "$@" "${optarray[@]}" 2> >(ansifilter \
    >>! "$logpth/2nd_scrapy.slog") 2> /dev/tty \
    > >(ansifilter >>! "$logpth/2nd_scrapy.slog") 1> /dev/tty

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin
Blog: http://zdharma.org

Re: Why multios affects >/dev/tty redirection?

[Philippe Troin]

On Tue, 2019-02-05 at 02:28 +0100, Sebastian Gniazdowski wrote:
> Hello,
> I have a command:
> 
> scrapy crawl 2nd "$@" "${optarray[@]}" 2> /tmp/reply  2> /dev/tty 1>
> /tmp/reply 1> /dev/tty
> 
> When invoked this way, scrapy (a python project) spawns an
> interactive
> console and it isn't fully usable  – e.g.: cursor keys emit escape
> sequencess rather t than moving the cursor or browsing history.

That's because when you use multios, the stdout (and other multios
descriptors) are not connected to a real terminal, but to a pipe
connected to the invoking zsh process, zsh then sends the output to
multiple files or devices.
Because scrapy is not connected to a real terminal, it behaves
differently.

> However, when I invoke scrqpy in following way:
> 
> scrapy crawl 2nd "$@" "${optarray[@]}"  2> /dev/tty 1> /dev/tty

Here scappy is connected directly to the terminal, and everything works
as expected.

> Is there a way around this?

Not without the help of another tool that can emulate a terminal.
There was a very old tool called "pty" that could do that.
I wrote a long time ago an enhanced version of that tool called ptyopen
( http://ftp.fifi.org/phil/ptyopen/ ) that does the same thing.

You'd invoke it like:

   ptyopen scrapy crawl 2nd "$@" "${optarray[@]}" 2> /tmp/reply  2> /dev/tty 1> /tmp/reply 1> /dev/tty

It then creates a pseudo-terminal, launches scrapy in it, and the pty
output can then be used transparently with multios.

Phil.

Re: Why multios affects >/dev/tty redirection?

[Sebastian Gniazdowski]

On Tue, 5 Feb 2019 at 17:43, Philippe Troin <phil@fifi.org> wrote:
> That's because when you use multios, the stdout (and other multios
> descriptors) are not connected to a real terminal, but to a pipe
> connected to the invoking zsh process, zsh then sends the output to
> multiple files or devices.
> Because scrapy is not connected to a real terminal, it behaves
> differently.

I wonder if it's not fully on scrappy's side, i.e. that in such
situation there are no actual obstacles in providing a working
terminal-grasping app, which is rather proved by vim | cat working
just normally. I.e. if it's scrapy that does a ttty-test (i.e. calls
isatty()) and decides to provide a limited application. This is again
fully proven by the fact that this command:

vim > >(ansifilter > ansi.txt) > /dev/tty

Works as expected – i.e. vim outputs a warning about output being not
to a terminal, and then works without a single flaw, i.e. cursor-keys,
home, end keys also working fine.

So could Zshell answer positively to the isatty() call, if at least
one of the multios-outputs is a terminal? I think there's a big
meaning, sense in this, however I don't want the email to be verbose.

> > Is there a way around this?
>
> I wrote a long time ago an enhanced version of that tool called ptyopen
> ( http://ftp.fifi.org/phil/ptyopen/ ) that does the same thing.
>
> You'd invoke it like:
>
>    ptyopen scrapy crawl 2nd "$@" "${optarray[@]}" 2> /tmp/reply  2> /dev/tty 1> /tmp/reply 1> /dev/tty
>
> It then creates a pseudo-terminal, launches scrapy in it, and the pty
> output can then be used transparently with multios.

Interesting tool, however I cannot use it, because my client doesn't
want advanced, custom things, even a shell-scripts are questioned (the
project is in python).

-- 
Sebastian Gniazdowski
News: https://twitter.com/ZdharmaI
IRC: https://kiwiirc.com/client/chat.freenode.net:+6697/#zplugin
Blog: http://zdharma.org

Re: Why multios affects >/dev/tty redirection?

[Philippe Troin]

On Thu, 2019-02-07 at 14:00 +0100, Sebastian Gniazdowski wrote:
> On Tue, 5 Feb 2019 at 17:43, Philippe Troin <phil@fifi.org> wrote:
> > That's because when you use multios, the stdout (and other multios
> > descriptors) are not connected to a real terminal, but to a pipe
> > connected to the invoking zsh process, zsh then sends the output to
> > multiple files or devices.
> > Because scrapy is not connected to a real terminal, it behaves
> > differently.
> 
> I wonder if it's not fully on scrappy's side, i.e. that in such
> situation there are no actual obstacles in providing a working
> terminal-grasping app, which is rather proved by vim | cat working
> just normally. I.e. if it's scrapy that does a ttty-test (i.e. calls
> isatty()) and decides to provide a limited application. This is again
> fully proven by the fact that this command:
> 
> vim > >(ansifilter > ansi.txt) > /dev/tty
> 
> Works as expected – i.e. vim outputs a warning about output being not
> to a terminal, and then works without a single flaw, i.e. cursor-
> keys,
> home, end keys also working fine.

Try:
  cat | vim > >(ansifilter > ansi.txt) > /dev/tty
Things don't work as well in this case.

> So could Zshell answer positively to the isatty() call, if at least
> one of the multios-outputs is a terminal? I think there's a big
> meaning, sense in this, however I don't want the email to be verbose.

No, because it's the command (and not the shell) that calls isatty() on
a random descriptor that should be connected to a terminal in the app's
opinion.
You could argue that in this case zsh should set-up a pseudo terminal
automatically, but:
 * There are lot of extra semantics whose behavior is unclear in this
   case.  For example, terminal related signal and window sizing ioctls
   have to be translated/forwarded between the top level terminal and
   the emulated pty.
 * Zsh has no way of knowing that a command will interact with the
   terminal.
In short, it's not a good idea.

Have you investigated the script command?  It may do what you need, and
it's standard.

Phil.