Re: Strange `UTF-8'?

[Toby Speight <streapadair@gmx.net>]

0> In article <vaf4s4wt7mp.fsf@lucy.cs.uni-dortmund.de>,
0> Kai Großjohann <URL:mailto:Kai.Grossjohann@CS.Uni-Dortmund.DE> ("Kai") wrote:

Kai> I received a message which was labeled charset=UTF-8, but my name was
Kai> all wrong.  I then used Mule-UCS to create a UTF-8 file with my name
Kai> in it, ran `less' on it, and got this output:
Kai>
Kai>     Gro<C3><9F>johann

Decoding C39F gives 3<<6 + 1F == DF == ß

So that looks right to me.


Kai> Running `less' on the message file was different:
Kai>
Kai>     Gro<C3><83>Yjohann

C383 => u-00C3 == Ã

And the Y (u-0059) is completely spurious.  (Remember, in UTF-8, all
US-ASCII characters stand for themselves - C0-FF are initial bytes,
and 80-BF are continuation bytes).


Kai> What is happening here?  Do the two encodings mean the same thing (but
Kai> Mule-UCS does not know that), or is the second one bogus?  Why is it
Kai> bogus?

The second one is wrong (but not bogus in the sense of broken - it
just represents something different from the intent).  I'm not sure
how it got like that.  It looks like it's been double encoded: the
leading C3 has been re-encoded as C383, and 9F, not being in Unicode
has gone very random.  Just a guess, though.


Kai> The message was created with:
Kai>
Kai> X-Mailer: Mozilla 4.72 [en] (X11; U; Linux 2.2.14-15mdksecure i686)