nosh: User-space virtual terminal test and questions

nosh: User-space virtual terminal test and questions

[Guillermo]

Hello,

So, I've gotten around to testing user-space VTs with nosh tools on
Gentoo, as described in the nosh Guide. Mostly console-fb-realizer,
but I also tested console-termio-realizer on a kernel VT for
comparison. This was in a VirtualBox VM, so there were two framebuffer
devices available: the one provided by the VirtualBox Guest Additions
'vboxvideo' kernel module (vboxdrmfb), and the kernel's VESA
framebuffer device (vesafb). The combination of vboxvideo with
console-fb-realizer was explosive, I got a (guest) kernel panic. But
console-fb-realizer with the VESA framebuffer device worked, as did
console-termio-realizer. "Worked" meaning that I could log in and get
an interactive shell using the user-space VT.

For ease of troubleshooting I wanted a minimal setup instead of the
full-blown one of nosh-bundles and the external formats import
subsystem. I left the Gentoo standard agetty processes on /dev/tty1 to
/dev/tty6 alone, and had console-fb-realizer or
console-termio-realizer take /dev/tty8, and communicate directly with
console-terminal-emulator via /run/dev/vc1, with neither a multiplexor
(at first) nor an input method front-end processor. I also set up the
customary TUI login service using vc-get-tty and open-controlling-tty,
and managed to deal with the BSDness of the requirements on font and
keyboard map files with the help of FreeBSD's SVN repository :)

So now the questions:

1) What is the proper way (if any) to switch between kernel VTs and
user-space VTs? What I found out:
  * console-multiplexor-control with a numeric command and the active
kernel VT, or just 'tty', as the vtname, could switch to a different
kernel VT, just like Alt + function key.
  * console-multiplexor-control with an '8' command and the active
kernel VT, or just 'tty', as the vtname, and Alt + F8, could both
switch to to the user-space VT, with both realizers.
  * With console-termio-realizer, Alt + F1 in the user-space VT could
switch back to the first kernel VT, but I expected that it would
likely work, given that this arrangement is a terminal realized on
another terminal.
  * With console-fb-realizer, Alt + F1 did nothing but print "WARNING:
Unknown input message" on /dev/console. The messages were probably
coming from console-terminal-emulator, as I supposed (and tested
later) that this is designed to switch from a user-space VT to
*another* user-space VT, if console-fb-realizer is communicating with
a console-multiplexor process.

So that left 'console-multiplexor-control 0@tty1', which did switch to
the first kernel VT... but only if run as root. Looking more closely,
it seems that when a kernel VT is specified as the vtname,
console-multiplexor-control is just a wrapper around an ioctl() call
with a VT_ACTIVATE command, which apparently works on Linux for an
unprivileged process if the VT is its controlling terminal. And for a
user-space VT it isn't, just like it isn't either for an X11 terminal
like rxvt-unicode. So, is there an alternative to
console-multiplexor-control as root?

2) Key combinations with Alt (e.g. Alt + f or Alt + b to move forward
or backward one word on Bash) did not work with neither
console-fb-realizer nor console-termio-realizer. Any idea why, or does
this just happen to me?

3) More strangely, Ctrl + x (e.g. to exit from GNU nano) did not work
with console-termio-realizer, but did with console-fb-realizer. Other
key combinations with Ctrl worked fine. Any idea why, or does this
just happen to me?

4) With console-termio-realizer, green is blue and blue is green :D
Not with console-fb-realizer, though. Any idea why, or does this just
happen to me?

Thanks,
G.

Re: nosh: User-space virtual terminal test and questions

[Jonathan de Boyne Pollard]

[-- Attachment #1: Type: text/plain, Size: 4570 bytes --]

Guillermo:

> The combination of vboxvideo with |console-fb-realizer| was explosive, 
> I got a (guest) kernel panic.
>
|console-fb-realizer| does not employ the arcana of framebuffer I/O, it 
just using little more than the mode setting API and memory mapped I/O, 
so I'm initially going to lay that one at the door of the driver.  I 
have had poor experiences with it myself.  Try the |--80-columns| option 
(which prevents |console-fb-realizer| trying to switch to some of the 
more exciting high-resolution modes) and ensure that your driver exactly 
matches your kernel version.

I currently test |console-fb-realizer| on real hardware.  (-:

Guillermo:

> I [...] managed to deal with the BSDness of the requirements on font 
> and keyboard map files with the help of FreeBSD's SVN repository :)
>
For those reading this, there is a whole chapter in the _Guide_ on the 
multiplicity of places whence one can obtain fonts, keyboard maps, and 
input methods.  The toolset does not come with them for two major 
reasons.  First, it is a deliberate design decision to use existing 
formats and not require new ones be created from scratch.  Second, they 
come under a variety of copyright licences, and it is simpler to just 
keep those separate.  Quite a few of them are already packaged up for 
operating systems.  I discovered /yet another/ collection of input 
method files just recently ... in a Debian package.

Guillermo:

> 1) What is the proper way (if any) to switch between kernel VTs and 
> user-space VTs?
>
|console-fb-realizer| does not have hot key chords for initiating KVT 
switching that get stripped out of input processing and enacted 
locally.  All such chords are sent down the input FIFO to a /user-space/ 
VT multiplexor.  It could have, as there is room for such a specialized 
type of actions in the keyboard map.  Since I need to do more work on 
having multiple keyboard maps in a single |console-fb-realizer| 
instance, I might look into it.  But at the moment 
|console-multiplexor-control| pointed at a KVT is the tool, and I am not 
going to hold off 1.39 specifically for this.  I was going to do that 
work in 1.40.

Guillermo:

> 2) Key combinations with Alt (e.g. Alt + f or Alt + b to move forward 
> or backward one word on Bash) did not work with neither 
> |console-fb-realizer| nor |console-termio-realizer|. Any idea why, or 
> does this just happen to me?
>
I have an idea why.  I have put improving it on the to-do list. It's not 
you.  (-:

Guillermo:

> 3) More strangely, Ctrl + x (e.g. to exit from GNU nano) did not work 
> with |console-termio-realizer|, but did with |console-fb-realizer|.
>
Control+X is ␘ (cancel, U+0018) which cancels an in-progress ECMA-48 
escape or control sequence.  This is documented in ECMA-48:1991 § 8.3.6 
and in § 4.3.5 of DEC's _VT520/VT525 Video Terminal Programmer Information_.

|console-termio-realizer| decodes its input with a fully-fledged ECMA-48 
decoder, that implements the DEC/ECMA-documented behaviours of both ␘ 
and ␛ on the decoding state machine; rather than with a perennially 
incomplete collection of ad-hoc pattern recognitions as 
|console-ncurses-realizer| does.  It's the same decoder that 
|console-terminal-emulator| uses for output.  You can see ␘ in action 
with the new |console-decode-ecma48| tool in 1.39, which also uses the 
same decoder.  Type in an incomplete CSI sequence by hand, and cancel it 
with ␘.  It will be discarded.  The same will happen if you print one 
and cancel it, as output to the terminal emulator.

Although it would be entirely legitimate for a terminal to send ␘ in the 
middle of (say) a DECFNK control sequence in order to cancel it, it 
would be unusual, so I'll add another exception to ECMA-48 decoding for 
this, that |console-termio-realizer| can select.

ECMA-48 is of course nothing to do with the input protocols that 
|console-fb-realizer| speaks with its input devices.

Guillermo:

> 4) With |console-termio-realizer|, green is blue and blue is green :D 
> Not with |console-fb-realizer|, though.
>
Part of my usual Z shell prompt is green, so I am confident that I would 
have noticed it being blue in error.  (-:

What terminal type is |console-termio-realizer| outputting to?  Is this 
a GUI terminal emulator with a colour palette that has been altered from 
the standard colour set?  Is it a 24-bit-colour-capable terminal?  Use 
|console-control-sequence --foreground blue|console-decode-ecma48| from 
version 1.39 on that terminal.  What do you see?  SGR 34?  SGR 38;5;4?  
Or SGR 38;2;... ?

Re: nosh: User-space virtual terminal test and questions

[Guillermo]

El dom., 6 ene. 2019 a las 8:27, Jonathan de Boyne Pollard escribió:
>
> > The combination of vboxvideo with |console-fb-realizer| was explosive,
> > I got a (guest) kernel panic.
> >
> [...]
> so I'm initially going to lay that one at the door of the driver.

Yeah, I'm suspicious about the kernel module too. I didn't spend too
much time with this, I just blacklisted the module for the
console-fb-realizer tests, which gave me the VESA framebuffer device.
But I thought I should mention that it happened in case anyone else
wanted to try that, or had not experienced it.

> Try the --80-columns option
> (which prevents console-fb-realizer trying to switch to some of the
> more exciting high-resolution modes)

The option is not present if console-fb-realizer is built on
[GNU/]Linux, right? 'console-fb-realizer --help' doesn't show it, and
the C++ source file seems to have the relevant code contained inside
an #if defined(__FreeBSD__) || defined(__DragonFly__) ||
defined(__OpenBSD__) - #endif block.

> > I [...] managed to deal with the BSDness of the requirements on font
> > and keyboard map files with the help of FreeBSD's SVN repository :)
> >
> For those reading this, there is a whole chapter in the Guide on the
> multiplicity of places whence one can obtain fonts, keyboard maps, and
> input methods.  The toolset does not come with them for two major
> reasons.

Oh, I don't expect nosh to ship font and keyboard map files. That was
a humorous remark about them either having to be in a BSD-specific
format, or in a format that a BSD program (vtfontcvt) outputs.

> First, it is a deliberate design decision to use existing
> formats and not require new ones be created from scratch.

That's fine.

> > 3) More strangely, Ctrl + x (e.g. to exit from GNU nano) did not work
> > with |console-termio-realizer|, but did with |console-fb-realizer|.
> >
> Control+X is ␘ (cancel, U+0018) which cancels an in-progress ECMA-48
> escape or control sequence.
> [...]
> console-termio-realizer decodes its input with a fully-fledged ECMA-48
> decoder, [...]

So, how does one get the control character delivered to the
application? Is it possible with console-termio-realizer?

> > 4) With console-termio-realizer, green is blue and blue is green :D
> >
> Part of my usual Z shell prompt is green, so I am confident that I would
> have noticed it being blue in error.  (-:

I noticed it with GNU ls's colorized output (directories showed up
green and executable files showed up blue), and Bash's prompt, which
also happens to be green and blue, and looked reversed. This is the
value of PS1:

\[\033[01;32m\]\u@\h\[\033[01;34m\] w $\[\033[00m\]

> What terminal type is console-termio-realizer outputting to?  Is this
> a GUI terminal emulator with a colour palette that has been altered from
> the standard colour set?

Straight to a kernel VT, and I don't think the color palette is
changed from the default. This is how I lauhched
console-termio-realizer:

/etc/inittab:
us2:3:respawn:/home/guillermo/bin/start-termio-realizer

/home/guillermo/bin/start-termio-realizer:
#!/bin/nosh
foreground sleep 1 ;
vc-get-tty tty8
open-controlling-tty
vc-reset-tty
/lib/nosh/setuidgid -s user-vt-realizer
console-termio-realizer /run/dev/vc1

(Yeah, that's van Smoorenburg init. Minimal setup for ease of
troubleshooting, remember :)  )

I didn't see a console-termio-realizer (or console-ncurses-realizer)
example in nosh-bundles, maybe simple file descriptor redirections
would have sufficed, but that worked, so...

>  Is it a 24-bit-colour-capable terminal? Use
> console-control-sequence --foreground blue|console-decode-ecma48 from
> version 1.39 on that terminal.  What do you see?  SGR 34?  SGR 38;5;4?
> Or SGR 38;2;... ?

I'd have to build in-development 1.39 for that. I'll post the result
when I do that.

Thanks,
G.

Re: nosh: User-space virtual terminal test and questions

[Guillermo]

El dom., 6 ene. 2019 a las 8:27, Jonathan de Boyne Pollard escribió:
>
>  Use
> |console-control-sequence --foreground blue|console-decode-ecma48| from
> version 1.39 on that terminal.  What do you see?  SGR 34?  SGR 38;5;4?
> Or SGR 38;2;... ?

Well, guess what. I built in-development 1.39 and it turns out that:

> > 2) Key combinations with Alt (e.g. Alt + f or Alt + b to move forward
> > or backward one word on Bash) did not work with neither
> > console-fb-realizer nor console-termio-realizer.

Not anymore. They work with the new console-termio-realizer and the
new console-fb-realizer.

> > 3) More strangely, Ctrl + x (e.g. to exit from GNU nano) did not work
> > with console-termio-realizer, but did with console-fb-realizer.

Not anymore. Ctrl + X gets delivered to nano with the new
console-termio-realizer.

> > 4) With console-termio-realizer, green is blue and blue is green :D
> > Not with console-fb-realizer, though.

Not anymore. Blue and green display normally with the new
console-termio-realizer, as do yellow and magenta, which I found out
in later tests that were also reversed with the one from version 1.38.

So it looks like that's going to be fixed in 1.39.

So thanks,
G.

Re: nosh: User-space virtual terminal test and questions

[Jonathan de Boyne Pollard]

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]

The blue/green problem only manifested itself with certain terminal 
types, which I was not regularly using.

Since you have 1.39 built, and as long as you built from today's 
snapshot, you can be the second person in the world to enjoy this:

    while true ; do printf '\f' ; top -b -n 1 | sed -e '1,6s/^/#/' ; sleep 1 ; done | console-flat-table-viewer --header-count 1

Substitute

    top -b all | sed -e '1,8s/^/#/'

on FreeBSD, or indeed

    procstat -a

Notice the |ascii| format, too.

So what the world needs now is for M. Bercot to give us a Linux+BSD 
version of |s6-ps| that uses US and RS, or |vis()| encoding.  (-:

The FreeBSD people definitely need to give us a |procstat| that uses 
|strvis()|, given that the output of |procstat| is explicitly documented 
as being a table and |vis()| encoding is actually documented for this 
type of table in FreeBSD.  (See the manual for |fstab|, for example.)