problems with multibyte support in dev-4?

problems with multibyte support in dev-4?

[Wataru Kagawa]

Hi all,

It appears to me that the dev-4 version is not supporting multibyte  
characters (Japanese) as good as the previous developmental  
versions.  In dev-4, Japanese characters in tab completion lists are  
not displayed properly (I see a bunch of numbers).  I hope the  
situation gets better before the release of 4.3.1.

Wataru Kagawa

Re: problems with multibyte support in dev-4?

[Zvi Har'El]

I see similar problems in Hebrew. The tab completion doesn't work on non-ascii
characters. In dev-3 it was ok, atleast until 10 February. I am trying to put
my finger on the patch which made this happen.

On Sat, 18 Feb 2006 22:54:41 +0900, Wataru Kagawa wrote about "problems with multibyte support in dev-4?":
> Hi all,
> 
> It appears to me that the dev-4 version is not supporting multibyte  
> characters (Japanese) as good as the previous developmental  
> versions.  In dev-4, Japanese characters in tab completion lists are  
> not displayed properly (I see a bunch of numbers).  I hope the  
> situation gets better before the release of 4.3.1.
> 
> Wataru Kagawa

-- 
Dr. Zvi Har'El      mailto:rl@math.technion.ac.il    Department of Mathematics
tel:+972-54-4227607 icq:179294841    Technion - Israel Institute of Technology
fax:+972-4-8293388  http://www.math.technion.ac.il/~rl/    Haifa 32000, ISRAEL
"If you can't say somethin' nice, don't say nothin' at all." -- Thumper (1942)
                            Saturday, 20 Shevat 5766, 18 February 2006, 4:24PM

Re: problems with multibyte support in dev-4?

[Clint Adams]

On Sat, Feb 18, 2006 at 10:54:41PM +0900, Wataru Kagawa wrote:
> It appears to me that the dev-4 version is not supporting multibyte  
> characters (Japanese) as good as the previous developmental  

On Sat, Feb 18, 2006 at 04:26:09PM +0200, Zvi Har'El wrote:
> I see similar problems in Hebrew. The tab completion doesn't work on non-ascii
> characters. In dev-3 it was ok, atleast until 10 February. I am trying to put

Are you using UTF-8 locales?

Re: problems with multibyte support in dev-4?

[Wataru Kagawa]

Hi Clint,

Yes, I have,

LANG=ja_JP.UTF-8

Tab completions were working up to dev-3.

On 2006/02/19, at 0:18, Clint Adams wrote:

> On Sat, Feb 18, 2006 at 10:54:41PM +0900, Wataru Kagawa wrote:
>> It appears to me that the dev-4 version is not supporting multibyte
>> characters (Japanese) as good as the previous developmental
>
> On Sat, Feb 18, 2006 at 04:26:09PM +0200, Zvi Har'El wrote:
>> I see similar problems in Hebrew. The tab completion doesn't work  
>> on non-ascii
>> characters. In dev-3 it was ok, atleast until 10 February. I am  
>> trying to put
>
> Are you using UTF-8 locales?

Re: problems with multibyte support in dev-4?

[Wayne Davison]

On Sat, Feb 18, 2006 at 10:54:41PM +0900, Wataru Kagawa wrote:
> In dev-4, Japanese characters in tab completion lists are  
> not displayed properly (I see a bunch of numbers).

Can you show us some of the output?  That should reveal what byte values
zsh is now escaping.  There was some code added in dev-4 that escapes
invalid characters (using a $'\123' idiom) earlier in the handling of
completed values, but that shouldn't affect any extra characters that
dev-3 wasn't already escaping in the display of the completion items.
Both algorithms are using the same validity check on the strings, the
only significant difference is that one works on on unmetafied strings
(the new one) and the other works on metafied strings (the old one).

..wayne..

Re: problems with multibyte support in dev-4?

[Wayne Davison]

On Sat, Feb 18, 2006 at 08:49:24AM -0800, Wayne Davison wrote:
> the only significant difference is that one works on on unmetafied
> strings (the new one) and the other works on metafied strings (the old
> one).

Ack!  That's actually the problem.  The new algorithm is supposed to be
operating on metafied strings.  I'll fix this.

..wayne..

Re: problems with multibyte support in dev-4?

[Wayne Davison]

On Sat, Feb 18, 2006 at 09:01:17AM -0800, Wayne Davison wrote:
> The new algorithm is supposed to be operating on metafied strings.
> I'll fix this.

I've committed a fix to CVS.  Anyone who'd like to manually patch it
into their source to try it out can grab it from here:

    http://opencoder.net/metafied.patch

Thanks for reporting this problem!

..wayne..

Re: problems with multibyte support in dev-4?

[Zvi Har'El]

[-- Attachment #1: Type: text/plain, Size: 869 bytes --]

I didn't see the fix in the CVS, but I manually grabbed the patch and 
applied it. It works! Thanks a lot.

Wayne Davison wrote, On 18/02/06 20:46:

> On Sat, Feb 18, 2006 at 09:01:17AM -0800, Wayne Davison wrote:
>   
>> The new algorithm is supposed to be operating on metafied strings.
>> I'll fix this.
>>     
>
> I've committed a fix to CVS.  Anyone who'd like to manually patch it
> into their source to try it out can grab it from here:
>
>     http://opencoder.net/metafied.patch
>
> Thanks for reporting this problem!
>
> ..wayne..
>   

-- 
Dr. Zvi Har'El      mailto:rl@math.technion.ac.il    Department of Mathematics
tel:+972-54-4227607 icq:179294841    Technion - Israel Institute of Technology
fax:+972-4-8293388  http://www.math.technion.ac.il/~rl/    Haifa 32000, ISRAEL
"If you can't say somethin' nice, don't say nothin' at all." -- Thumper (1942)


[-- Attachment #2: Type: text/html, Size: 1611 bytes --]

Re: problems with multibyte support in dev-4?

[Wataru Kagawa]

Hi Wayne,

Thanks for the patch.  Tab completion is now working nicely.

Wataru Kagawa
On 2006/02/19, at 3:46, Wayne Davison wrote:

> On Sat, Feb 18, 2006 at 09:01:17AM -0800, Wayne Davison wrote:
>> The new algorithm is supposed to be operating on metafied strings.
>> I'll fix this.
>
> I've committed a fix to CVS.  Anyone who'd like to manually patch it
> into their source to try it out can grab it from here:
>
>     http://opencoder.net/metafied.patch
>
> Thanks for reporting this problem!
>
> ..wayne..