problem in prompt in utf-8

["Zvi Har'El" <rl@math.technion.ac.il>]

Hello,

I have started using zsh-4.3.0 from the CVS, in a uft-8 locale. I enjoy it
very much. However, I have a problem with the prompting. This is not new, but
since the completion now works nicely, I thought I'll mention it, since it is
not solved yet.

I have the setting

    PS1=%/$\ 

I expect that 

    print -P $PS1 


and

    pwd 

will give the same output, which will also be the zsh prompt (except the final
$ and space, of course). However, if the current directory name contains
hebrew letter, which are in the range U+05D0 to U+05EA, i.e., the utf-8
sequences have two bytes, with the first one is always 0xD7 (M-W) and the
second  in the range 0x90 (M-^P) to 0xAA (M-*). I mkdir'ed a directory which
has all the letters in this range:

/home/rl$ mkdir אבגדהוזחטיךכלםמןנסעףפץצקרשת

cd'ed to that directory:

/home/rl$ cd אבגדהוזחטיךכלםמןנסעףפץצקרשת 

I got as an echo a correct result:

~/אבגדהוזחטיךכלםמןנסעףפץצקרשת

The next prompt had invalid utf-8 sequences:


/home/rl/������������לםמןנסעףפץצקרשת$ 


To make it more specific, all the range U+05D0 to U+05DB,  (second byte 0x90
to 0x9ba) got invalid. I don't know exactly what is wrong. Notice that 'pwd'
produces

/home/rl/אבגדהוזחטיךכלםמןנסעףפץצקרשת

I.e, all the letters are correct, while 'print -P $PS1' produces

/home/rl/אבגדהוזחטיך�לםמןנסעףפץצקרשת$

With exactly one invalid utf-8 seqience, more specifically, U+05DB (second
byte 0x9ba) - the last one in the previous range, is bad.

print -P $PS1 | cat -v produces

/home/rl/M-WM-^PM-WM-^QM-WM-^RM-WM-^SM-WM-^TM-WM-^UM-WM-^VM-WM-^WM-WM-^XM-WM-^YM-WM-^ZM-WM-WM-^\M-WM-^]M-WM-^^M-WM-^_M-WM- M-WM-!M-WM-"M-WM-#M-WM-$M-WM-%M-WM-&M-WM-'M-WM-(M-WM-)M-WM-*$

while pwd | cat -v produces

/home/rl/M-WM-^PM-WM-^QM-WM-^RM-WM-^SM-WM-^TM-WM-^UM-WM-^VM-WM-^WM-WM-^XM-WM-^YM-WM-^ZM-WM-^[M-WM-^\M-WM-^]M-WM-^^M-WM-^_M-WM- M-WM-!M-WM-"M-WM-#M-WM-$M-WM-%M-WM-&M-WM-'M-WM-(M-WM-)M-WM-*

It is perhaps hard to see the difference, but a close inspection shows that
the first string contains a solitary M-W between the M-WM-^Z and the the
M-WM-^\ sequences, while the second one contains there the sequence M-WM-^[ ,
i.e., a M-^[, or Meta-Esacpe, was dropped from the string. 

Unfortunately, I didn't find an easy way to put the real prompt on a file, so
I can't tell what is the exact sequences in it.

I hope this make some sense. 

-- 
Dr. Zvi Har'El      mailto:rl@math.technion.ac.il    Department of Mathematics
tel:+972-54-4227607 icq:179294841    Technion - Israel Institute of Technology
fax:+972-4-8293388  http://www.math.technion.ac.il/~rl/    Haifa 32000, ISRAEL
"If you can't say somethin' nice, don't say nothin' at all." -- Thumper (1942)
                               Sunday, 7 Elul 5765, 11 September 2005,  1:54PM