multibyte implementation (for filename completion with umlauts)

multibyte implementation (for filename completion with umlauts)

[Andy Spiegl]

Another question from the past:  :-)

Back in 2011 we found that a filename with umlauts (öäü) can't be completed
case insensitively because zsh can't handle multibyte characters correctly
because it's a pretty complex issue:
 http://www.zsh.org/mla/users/2011/msg00015.html

Ever since then I live with it more or less happily.  All I could find
today in the docs is a TODO entry:
 * support for multibyte characters, including UTF-8

So I suppose the multibyte problem in the completion code is still there, right?

Actually, all I'd like to achieve is case insensitivity for umlauts like that:
 zstyle ':completion:*' matcher-list 'm:{A-ZÄÖÜa-zäöü}={a-zäöüA-ZÄÖÜ}

Any ideas?

Thanks so much!
 Andy

-- 
 The pre-release of the alpha for the forthcoming beta of the
 pre-build of the next point release should be out tonight.  (Graeme Devine)

Re: multibyte implementation (for filename completion with umlauts)

[Daniel Shahaf]

Andy Spiegl wrote on Thu, Sep 20, 2018 at 12:14:03 +0200:
> Actually, all I'd like to achieve is case insensitivity for umlauts like that:
>  zstyle ':completion:*' matcher-list 'm:{A-ZÄÖÜa-zäöü}={a-zäöüA-ZÄÖÜ}

I don't have a fix for you, but
.
    zstyle \* matcher-list 'm:{A-Za-z}={a-zA-Z} m:ae=Ä'
.
works for me for completing «fooae<TAB>» to «FOOÄBAR».  It doesn't work
if I use «ä» instead of «ae».  Therefore, how about writing a widget
that does s/ẍ/xe/ (for x in a,o,u,A,O,U) before attempting completion.
Would that be a workaround?

Cheers,

Daniel

Re: multibyte implementation (for filename completion with umlauts)

[Andy Spiegl]

Thanks Daniel.

> Therefore, how about writing a widget
> that does s/ẍ/xe/ (for x in a,o,u,A,O,U) before attempting completion.
> Would that be a workaround?
Yes, that sounds like a (pretty wicked hehe) workaround.

What would a widget like that look like?
I am really bad at zsh regexp and search/replace strings. :-(

While playing around I found a completer behaviour I cannot explain:
% touch testüng
% ls testün<TAB>   --> "ls testüng"
% ls testÜn<TAB>   --> BINGs
% ls Testün<TAB>   --> also just BINGs

Shouldn't
 zstyle ':completion:*' matcher-list 'm:{A-ZÄÖÜa-zäöü}={a-zäöüA-ZÄÖÜ}
take care of the capital T?
Apparently this one multibyte character screws up the whole completion process?

However, if I add
 zstyle ':completion:*' completer _complete _match _correct
then "ls Testün" and even "ls testÜn" is corrected/completed fine.

I am a bit lost. :-)

Thanks,
 Andy

-- 
 If you believe everything you read, better not read.
   (Japanese Proverb)

Re: multibyte implementation (for filename completion with umlauts)

[Daniel Shahaf]

> > Therefore, how about writing a widget
> > that does s/ẍ/xe/ (for x in a,o,u,A,O,U) before attempting completion.
> > Would that be a workaround?
> Yes, that sounds like a (pretty wicked hehe) workaround.
> 

Unfortunately, I just now see that while «fooae<TAB>» does complete to
«FOOÄBAR», «fooaeb<TAB>» does not.

> What would a widget like that look like?
> I am really bad at zsh regexp and search/replace strings. :-(

The string manipulation is the easy part:

    % () { local s=$1; s=${s//ö/oe}; typeset -p s } fööbar
    foeoebar

The trickier part is how to change only the last word on the command line.
I can think of a few options here, but I'm not sure what to recommend; perhaps
manipulate the end of $LBUFFER, consulting $words[CURRENT] to find the
beginning of the word before the cursor?  As you say later, perhaps using
_correct, or something modeled after it, would be better.

> While playing around I found a completer behaviour I cannot explain:
> % touch testüng
> % ls testün<TAB>   --> "ls testüng"
> % ls testÜn<TAB>   --> BINGs
> % ls Testün<TAB>   --> also just BINGs
> 
> Shouldn't
>  zstyle ':completion:*' matcher-list 'm:{A-ZÄÖÜa-zäöü}={a-zäöüA-ZÄÖÜ}
> take care of the capital T?
> Apparently this one multibyte character screws up the whole completion process?
> 

The capital T example for me if I change the order to m:{A-Za-zÄÖÜäöü}={a-zA-ZäöüÄÖÜ}.

> However, if I add
>  zstyle ':completion:*' completer _complete _match _correct
> then "ls Testün" and even "ls testÜn" is corrected/completed fine.