utf-8 and zsh

utf-8 and zsh

[Julien Palmas]

Hi list
this is my first message.
i haven't been using UNIX stuff for very long since i'm a mac user. But 
with Mac OS X, it was the time to start ;-)
Anyway, i've heard about the different shells and even if i don't 
really know the others, i believe zsh is a very good one (completion, 
loops, etc ...)

that's why i would like one more feature to make it even better. utf-8 
support. after reading the list archive, i've seen that utf-8 is not 
yet supported in zsh 4.1.1. but i've also seen that zsh 4.2 is in alpha 
4 stage.

so i've got 2 questions :
- will zsh 4.2 will have utf-8 support
- when do you expect it to be available ? ;-) (answer not compulsory 
...)

i'm doing translation stuff with vim and i'd like my shell to fully 
handle utf-8 for more consistency in my settings.

thank you guys
julien

utf-8 and zsh

[Maurí­cio]

Hi,

Is there some zsh-specific way to enter utf-8
characters? Or should I rely on my terminal to
do that (xterm, mlterm etc.)?

Thanks,
Maurício

Re: utf-8 and zsh

[Andrey Borzenkov]

[-- Attachment #1: Type: text/plain, Size: 535 bytes --]

On Воскресенье 07 декабря 2008 23:10:29 Maurí­cio wrote:
> Hi,
>
> Is there some zsh-specific way to enter utf-8
> characters? Or should I rely on my terminal to
> do that (xterm, mlterm etc.)?
>

You can use insert-composed-char ZLE function; see man zshcontrib. 
Otherwise there are good chances that it will just work when you enter 
multibyte characters using OS supplied means.

Keep in mind that zsh will use whatever encoding is set by current 
locale; it does not convert to UTF-8 internally.

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

Re: utf-8 and zsh

[Benjamin R. Haskell]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1585 bytes --]

On Sun, 7 Dec 2008, Maurício wrote:

> Hi,
>
> Is there some zsh-specific way to enter utf-8 characters? Or should I 
> rely on my terminal to do that (xterm, mlterm etc.)?
>

There are two ZLE widgets related to entering special characters: 
insert-composed-char and insert-unicode-char

To use them, put this somewhere in your Zsh startup files:

autoload insert-composed-char
zle -N insert-composed-char
# bind it to some key (Ctrl+x in this example)
bindkey ^X insert-composed-char

autoload insert-unicode-char
zle -N insert-unicode-char
bindkey ^Y insert-unicode-char

Then, the key sequence: ^X e ' creates é, for example. See man zshall for
the full list. (search for insert-composed-char)

NB. On my system, though, it doesn't appear to be fully working (Both ^X
i ` and ^X i ' produce ì (even though the latter should produce í))

For an arbitrary Unicode point, you do:
^Y e 9 ^Y (é)
^Y 2 0 1 9 ^Y (’)
(That is: whatever-you-bound-it-to, the code in hexadecimal, then the
bound key again.)

Really, though, I've only found these useful outside of X11. Recent
things I've read say that X11 Input Methods are preferred to X11
Compose. But, I've found that Compose is what works best for me (a
native English speaker -- so, I only rarely need accented/other chars).

With Compose, for example, I can type Compose-key (which I've mapped to
'Menu'), e ', and get é. The sequences mostly seem logical (compose a e
= æ), and there are a lot of them.

(I actually made a custom keyboard map, which also puts some of the ones
I use most frequently as AltGr keys.)

Best,
Ben

Re: utf-8 and zsh

[Benjamin R. Haskell]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 448 bytes --]

On Sun, 7 Dec 2008, Benjamin R. Haskell wrote:

> There are two ZLE widgets related to entering special characters: 
> insert-composed-char and insert-unicode-char
> 
> [...]
> 
> NB. On my system, though, it doesn't appear to be fully working (Both ^X 
> i ` and ^X i ' produce ì (even though the latter should produce í))

Attached is a patch against CVS. The code for ì was in both sets. 
Corrected to the code for í in the '-row.

Best,
Ben

[-- Attachment #2: Type: TEXT/PLAIN, Size: 662 bytes --]

Index: Functions/Zle/define-composed-chars
===================================================================
RCS file: /cvsroot/zsh/zsh/Functions/Zle/define-composed-chars,v
retrieving revision 1.5
diff -u -r1.5 define-composed-chars
--- Functions/Zle/define-composed-chars	11 Dec 2006 03:42:59 -0000	1.5
+++ Functions/Zle/define-composed-chars	7 Dec 2008 22:26:25 -0000
@@ -23,7 +23,7 @@
 # acute
 a=\'
 z[$a]="\
-A C1 E C9 I CD O D3 U DA Y DD a E1 e E9 i EC o F3 u FA y FD C 106 c 107 \
+A C1 E C9 I CD O D3 U DA Y DD a E1 e E9 i ED o F3 u FA y FD C 106 c 107 \
 L 139 l 13A N 143 n 144 R 154 r 155 S 15A s 15B Z 179 z 17A \
 "
 # circumflex

Re: utf-8 and zsh

[Phil Pennock]

On 2008-12-07 at 17:26 -0500, Benjamin R. Haskell wrote:
> Attached is a patch against CVS. The code for ì was in both sets. 
> Corrected to the code for í in the '-row.

You might also find http://www.zsh.org/mla/workers/2007/msg00306.html
useful; X-seq: zsh-workers 23385

add_composition_pair (which I never did give a better name to),
digraph_decimal and digraphs.

FWIW, I use ^Xk because the 'k' is close enough to vim's ^k for digraphs
that I can remember it.  Then any digraphs I actually use, I make sure
exist in both vim and zsh.  For me, this is more portable than X11
composition but I tend to do things like ssh in from MacOS or even
Windows (via PuTTY), so between zsh and vim I'm mostly covered.

-Phil