Re: [9fans] UTF-8 criticism?

["Joel Salomon" <salomo3@cooper.edu>]

>> Would moving to 32 bit signed (and only 0 -- 2^21 allowed, plus -1 for
>> EOF) as in the more recent revisions of Unicode take care of the
>> surrogates problem?
>
> this has nothing to do with EOF.

Sorry if I was unclear - let me try again. Would moving to 32 bit signed
(and only 0 -- 2^21 allowed), thus including all surrogates in the
directly accessible character set solve the problem?

Yes, this does open a new can of worms, but how much more difficult would
it be to move from 16 bit Runes to 21/32 bit wide Runes then it was to
move from 7 bit ASCII to Unicode in the first place?

As an aside, the way I've understood the Unicode standard (4.0), 21 bit
characters can be encoded in 1, 2, 3, or 4 bytes in UTF-8 and if text is
internally represented by int32, some out-of-band information (like EOF,
or bad UTF (but preserving the original bytes)) can be carried along.

--Joel