ZSH display raw keycodes for ¡ and ¿ spanish characters

ZSH display raw keycodes for ¡ and ¿ spanish characters

[José García Juanino]

Hi zsh users,

after doing an upgrade of my FreeBSD operating system, zsh no longer
display properly the followings spanish characters: ¡ and ¿. I do not
know if this annoying issue is FreeBSD specific.

See the following screenshot:

http://picpaste.com/exclamation-L21PZhNY.png

When you type the '¡', I would expect that symbol in the interactive
shell, instead of '<00a1>' output.

Any comment or advice will be welcome, and sorry my poor english. Best regards

Re: ZSH display raw keycodes for ¡ and ¿ spanish characters

[Peter Stephenson]

On Wed, 27 Jul 2016 23:36:39 +0200
José García Juanino <jjuanino@gmail.com> wrote:
> after doing an upgrade of my FreeBSD operating system, zsh no longer
> display properly the followings spanish characters: ¡ and ¿. I do not
> know if this annoying issue is FreeBSD specific.
> 
> See the following screenshot:
> 
> http://picpaste.com/exclamation-L21PZhNY.png
> 
> When you type the '¡', I would expect that symbol in the interactive
> shell, instead of '<00a1>' output.
> 
> Any comment or advice will be welcome, and sorry my poor english. Best regards

zsh uses the system library to determine what's a printable character.

The most likely thing is a mismatch between the terminal input and the
display.  The <00a1> indicates the character being input is probably
correct as that's the charater you want in ISO Latin and Unicode.
Possibly the LANG environment variable isn't set correctly.  I'm
guessing it should be es_??.UTF-8 e.g. es_ES.UTF-8 for some variant of
Spanish.  It's also possible the terminal isn't using the right
encoding, which you might be able to check in the terminal's
configuration, e.g. Terminal -> Set Character Encoding in
gnome-terminal.

pws

Re: ZSH display raw keycodes for ¡ and ¿ spanish characters

[José García Juanino]

On 28 July 2016 at 16:17, Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Wed, 27 Jul 2016 23:36:39 +0200
> José García Juanino <jjuanino@gmail.com> wrote:
>> after doing an upgrade of my FreeBSD operating system, zsh no longer
>> display properly the followings spanish characters: ¡ and ¿. I do not
>> know if this annoying issue is FreeBSD specific.
>>
>> See the following screenshot:
>>
>> http://picpaste.com/exclamation-L21PZhNY.png
>>
>> When you type the '¡', I would expect that symbol in the interactive
>> shell, instead of '<00a1>' output.
>>
>> Any comment or advice will be welcome, and sorry my poor english. Best regards
>
> zsh uses the system library to determine what's a printable character.
>
> The most likely thing is a mismatch between the terminal input and the
> display.  The <00a1> indicates the character being input is probably
> correct as that's the charater you want in ISO Latin and Unicode.
> Possibly the LANG environment variable isn't set correctly.  I'm
> guessing it should be es_??.UTF-8 e.g. es_ES.UTF-8 for some variant of
> Spanish.  It's also possible the terminal isn't using the right
> encoding, which you might be able to check in the terminal's
> configuration, e.g. Terminal -> Set Character Encoding in
> gnome-terminal.
>
> pws


Thanks for your reply, and sorry for the delay of my response.

The locales are aligned with my terminal. I use LANG=es_ES.ISO8859-15
and mrxrvt as terminal.

Doing a further research I have found a workaround: disable-multibye
in configure. But it is not a zsh fault, as multibyte was enabled in
the previous FreeBSD release (10.3) and worked fine. I think that the
upgrade of ncurses has triggered the issue, but I do not know exactly
how. And I have discovered that other tools are affected also, as mutt
pager: now display the "¡" character as raw code. Both tools (zsh and
mutt) are linked with the same library: ncursesw.so.8.

Best regards

Re: ZSH display raw keycodes for ¡ and ¿ spanish characters

[Chr. von Stuckrad]

On Thu, 25 Aug 2016, José García Juanino wrote:

> >> When you type the '¡', I would expect that symbol in the interactive
> >> shell, instead of '<00a1>' output.

> Doing a further research I have found a workaround: disable-multibyte
> in configure. But it is not a zsh fault, as multibyte was enabled ...
> ... (mentioning mutt) ...

Having 'the same problem in mutt and zsh' reminded me of a time,
when I *remotely* connected from a local terminal (rxvt-unicode).

EVERY place, where your typed chars 'come across' (i.e. terminalprog,
pty, connection, remote pty (may be multiple times with screen or tmux),
last shell or mutt), the connection needs to know, whether to use
single or multibyte characters.  And often the default in ptys is
single, so then you need 'stty iutf8' to get correct data to shell
or program (and you need this on all steps of the connection and
inside terminals and screen or tmux).

After making sure, I ALWAYS and ONLY use 'utf8'-on-the-wire
all my problems with multibyte shell input and most of
the problems with mutt or vim are gone now.

Alternatively, all steps might be single-byte, and with the
same 'older' locale.  SOME utf8 combinations still come
through then, but others are interpreted as something like
<control-whatever><strange-char> messing up the line.
So then you should make sure to have NO utf8 at all.

Stucki

-- 
Christoph von Stuckrad      * * |nickname |Mail <stucki@mi.fu-berlin.de> \
Freie Universitaet Berlin   |/_*|'stucki' |Tel(Mo.,Mi.):+49 30 838-75 459|
Mathematik & Informatik EDV |\ *|if online|  (Di,Do,Fr):+49 30 77 39 6600|
Takustr. 9 / 14195 Berlin   * * |on IRCnet|Fax(home):   +49 30 77 39 6601/