Re: UTF-8 input [was Re: PATCH: zle_params.c]

[Peter Stephenson <pws@csr.com>]

Bart Schaefer wrote:
> } In addition to getkey() and friends, there is the related matter of the
> } variable lastchar.  Currently this is a single character; I'm not yet
> } 100% sure whether we can keep this, or promote it to a wchar_t, or
> } whether we might need both types.  I fear it may be the last.
> 
> Not just lastchar, but also the KEYS parameter.  If wide chars are dealt
> with as sequences at the widget binding level, but BUFFER contains the
> corresponding wchars instead, then various currently-working tricks that
> involve inserting all or part of KEYS into BUFFER will fail.  At least,
> it becomes harder to emulate self-insert(-multibyte) in widget funcs.

I've been looking at all this rather slowly...

I think $KEYS is OK.  The current intention is for the input to key
bindings to remain multibyte strings (metafied where necessary).
Modifying BUFFER and other parameters converts from multibyte strings to
wide characters automatically; I've already written that bit (because it
was easy).  So if you feed back KEYS into the system from a function it
will pass through the mbtowc stuff at the appropriate level.

It does, of course, become trickier to decide how many characters there
are in $KEYS --- or, more critically, $BUFFER.  However, that's all
bound up with how we do ${#KEYS} etc. in the main shell, which is a
separate question.  I suspect we will want a flag such as ${(m)#KEYS} to
treat the string as a multibyte character string instead of raw bytes
and maybe a ${(M)#KEYS} not to.  The default is tricky; I'm sure you can
argue for it to handle multibyte strings by default in a suitable
locale, but when Perl did that it broke everything in sight.  However,
I'm not planning on doing that bit until we've got ZLE out of the way.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************