Re: UTF-8 support has stopped working

[Reiner Steib <4.uce.03.r.s@nurfuerspam.de>]

On Tue, Apr 15 2003, Steinar Bang wrote:

> [Note! Gnus insists making this message 7 parts because of charset
>        issues.  Should it automatically have selected utf-8?  

Yes.  I'm neither an expert on XEmacs nor on Unicode.  But XEmacs'
decoding of my article seems buggy.  Part 2 and part 7 of your message
are charset=iso-2022-jp (Japanese), but mine didn't contain Japanese
characters.

>>>>>> Reiner Steib <4.uce.03.r.s@nurfuerspam.de>:
>
>> On Mon, Apr 14 2003, Steinar Bang wrote:
>
>>> With XEmacs 21, it displays both UTF-8 encoded headers and message
>>> parts correctly.
>
>> Really?  Can you see Markus Kuhn's utf-8 demo file in XEmacs 21.4 more
>> or less correctly?  See
>> <news://news.gmane.org/utf-8-demo.fsf@marauder.physik.uni-ulm.de>.
>
> I see a suprising amount of stuff seemingly correctly.  Ie. I wasn't
> aware that I had the fonts to see it correctly

> I see question marks before "deutsche" and "Anführungszeichen", I
> see a rectangle immediately following "deutsche".  I see a double
> quote character with the thickest end at the bottom and curling
> towards the right (is this a "german quote"?)  just before the
> comma,

XEmacs uses a Japanese character for the low (i.e. left) double
quotation mark (after "Anführungszeichen"), which looks similar to the
correct character (but much more fat; and it's a `wide' character).

> and I see the Euro symbol (ie. not the dollar symbol you see).

Strange: Somebody in dcsg (German Gnus newsgroup) reported to see a
dollar too with Oort Gnus v0.10 / XEmacs 21.4 (Common Lisp).  And I
see a dollar with XEmacs 21.4.12 and a fresh CVS Gnus (or with 5.8.8),
too.  <news:yxzvfxogjqj.fsf@d79qxd0j.bifab.de>

>> But this is not a Gnus issue (I get the same opening the demo file
>> directly in XEmacs).  
>
>> In Emacs 21.3, I see some rectangles (I guess this is due to missing
>> fonts)
>
> I would have guessed that missing fonts was the root of the entire
> problem.  You're saying it's not?
[...]
> In any case, we're not seeing the same thing with the same software.

That's only the case for the EUR<->$ issue, isn't it?  (XEmacs and
Emacs are _not_ the same software of course.)

> So the difference in what we're seeing either comes from different
> versions of the software, or a different selection of fonts installed.
>
> I'm guessing at fonts.  See the end of the message for the missing
> font warnings I got.

I use a font server here with lots of fonts.  Although I get no font
warnings at all with XEmacs, I don't see the correct quotation marks
and EUR sign.  An the conversion to Japanese characters is very
strange, IMHO.  My impression is, that this is due to a buggy (or
incomplete) UTF-8 implementation in XEmacs.

BTW: Someone using XEmacs 21.5.11 and Oort Gnus 0.15 reported to see a
big, fat equal signs instead of the low quotaion marks and other
characters.  He also had to split the response in many parts, see...
<v91y0chhra.fsf@marauder.physik.uni-ulm.de> (test with some UTF-8
chars) and it followups:
<87of3fmuh0.fsf@520075220525-0001.dialin.t-online.de> (split)
<v97ka28j2q.fsf@marauder.physik.uni-ulm.de> (`M-x describe-char-after
RET'-analysis of some of the CJK chars of the previous message).

> Why some missing characters showed up as rectangles, some as question
> mark, and some as tilde, I have no idea.

I guess you should ask this in some XEmacs list or group.  The same
problems also happen when you look at Markus Kuhn's utf-8 demo file
(without Gnus) in XEmacs.
<URL:http://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-demo.txt>

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/