Re: UNICODE Private Use Area characters in BUFFER

[Roman Perepelitsa <roman.perepelitsa@gmail.com>]

On Mon, Oct 24, 2022 at 3:44 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Sun, Oct 23, 2022 at 6:27 PM Mikael Magnusson <mikachu@gmail.com> wrote:
> >
> > I think if we should do anything with wcwidth9.h, it's remove it.
>
> I must be missing something.  If glibc already supports unicode9, then
> isn't the right thing just for Roman to NOT --enable-unicode9?

The behavior w.r.t. Prive Use Area characters in BUFFER is the same
whether or not --enable-unicode9 is used. In other words, iswprint()
returns false.

On Mon, Oct 24, 2022 at 1:36 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Sun, Oct 23, 2022 at 4:22 PM Roman Perepelitsa
> <roman.perepelitsa@gmail.com> wrote:
> >
> > This code shows how zsh treats characters from Private Use Area. The
> > output doesn't depend on or require a terminal. I wanted to show that
> > zsh handles characters from Private Use Area just fine. The only place
> > I know of where zsh cannot handle them is BUFFER.
>
> That's sort of the point?  BUFFER does depend on and require a
> terminal.  Asserting that zsh "handles" those characters in other
> contexts isn't indicative of anything beyond demonstrating that
> terminal "handling" is a special case.

The fact that Private Use Area characters work in PS1 is informative
here, as is the expansion of ${(m)#foo}. In both contexts zsh assumes
that Private Use Area characters occupy one column.

I went ahead and checked how other shells behave if you paste U+E0B0
into the command line. They all output the character as-is. Shells
that need to keep track of the cursor position all assume that U+E0B0
is narrow. The shells I tried are: bash, csh, mksh, ash and dash.

There is one thing I didn't mention that is relevant. There is at
least one terminal (iTerm2) that has an option to treat Private Use
Area characters as wide. This option is off by default.

Roman.