Re: bug in completion/expansion of files with LANG=C

[Wayne Davison <wayned@users.sourceforge.net>]

On Fri, Jan 06, 2006 at 10:40:19PM +0000, Peter Stephenson wrote:
> If we can't convert a multibyte character to a wchar_t we therefore
> can't do anything with it.

One thing we could do would be to transform an invalid value into a
literal '?' -- that would at least match the filename, but it could be
dangerous.

Another thing we could do is to pick a legal wide-character value that
would be used to indicate that the real value for this character was
stored in a parallel (byte-sized) array.  For instance, choosing the
delete character would be one possibility (obviously making it necessary
to turn a real \177 char into a \177 value in the parallel array).  Yes,
this would complicate the code a good bit, but I think we need to do
something to make it possible for people to complete all filenames that
may be on their system.

> Second, and less difficult, it's quite a big change to have characters
> in the command line displayed differently from the way they naturally
> output.

No, there's already a precedence for this:  control characters.  To
handle this, zsh has a refresh string that is built up from the command-
line string, and adding one extra rule to the code in zrefresh() that
would make these invalid characters display as multiple physical
characters would actually be pretty easy.

> It would be quite helpful to have them with some terminal effect, too.

That would certainly be nice, and could be used to make the display of
literal control characters better.  The simplest way to add it would
probably be to extend the rparams structure to add some per-character
flags that zrefresh() would set and refreshline() would obey.

An alternate possibility would be to use the idiom "^123" to output the
octal value of the invalid byte.  (I chose '^' because it is already
special for displaying control characters, and ctrl-digit values aren't
already used to indicate anything.)

..wayne..