[musl] Shift JIS vs CP932

[musl] Shift JIS vs CP932

[Maxim Kouznetsov]

[-- Attachment #1: Type: text/plain, Size: 643 bytes --]

Hello,

Looks like on 2024-02-29 CP932 was set as an alias of Shift_JIS, but there are a number of character differences, see https://dev.mysql.com/doc/refman/8.4/en/charset-cp932.html for example. Those codepages are very similar, but not actually the same.

Maxim
The information contained in this communication from the sender is confidential. It is intended solely for use by the recipient and others authorized to receive it. If you are not the recipient, you are hereby notified that any disclosure, copying, distribution or taking action in relation of the contents of this information is strictly prohibited and may be unlawful.

[-- Attachment #2: Type: text/html, Size: 2329 bytes --]

Re: [musl] Shift JIS vs CP932

[Rich Felker]

On Mon, Aug 26, 2024 at 10:30:17PM +0000, Maxim Kouznetsov wrote:
> Hello,
> 
> Looks like on 2024-02-29 CP932 was set as an alias of Shift_JIS, but
> there are a number of character differences, see
> https://dev.mysql.com/doc/refman/8.4/en/charset-cp932.html for
> example. Those codepages are very similar, but not actually the
> same.

Yes, I realized this a while back and was trying to collect
information to figure out what we actually need to do to support cp932
properly. All the information on the differences is confusing and I'm
not sure yet whether the above is clarifying.

The big question is probably whether to make Shift_JIS be an alias for
cp932 (at least in the decoding direction) and always support the
extensions, or have the extensions only available when cp932 is
requested.

There's also a question of how to encode the information. If we extend
the jis0208 table with them, the other encodings of jis0208 need to
explicitly disallow the extensions, but it's probably more efficient
to pack them in there. OTOH if they conflict with actual later jis
versions we might want to support, having them in the jis0208 table
seems like a bad idea.

Rich