text/plain; charset=UTF-8 doesn't work anymore

text/plain; charset=UTF-8 doesn't work anymore

[Steinar Bang]

Platform: Intel Pentium M, debian sarge (testing), 
	  emacs21 21.3+8,
	  No Gnus v0.3 (CVS update from today),
	  mule-ucs 0.84.999+0.20030620-8

UTF-8 characters in articles, show up as two ISO-8859-1 characters for
me.

I think this used to work for me, but I usually use XEmacs to read
email with Gnus.  I've just recently switched to GNU Emacs because of
the infloop-when-opening-group bug that has been introduced by an
XEmacs bugfix.

I've tried both without mule-ucs, and with mule-ucs, but that doesn't
seem to make any difference.

Does anyone know if this problem is caused by my choice of Emacs, my
config (eg. locale settings?  I recently did a reinstall from the
sarge netinst CD, and let it install for a Norwegian locale), or if
this a recent bug in Gnus?

I noticed that an article with this problem was text/plain using
base64 as its CTE.  Perhaps there's something there?

Thanx!


- Steinar

Re: text/plain; charset=UTF-8 doesn't work anymore

[Reiner Steib]

On Fri, Jan 14 2005, Steinar Bang wrote:

> Platform: Intel Pentium M, debian sarge (testing), 
> 	  emacs21 21.3+8,
> 	  No Gnus v0.3 (CVS update from today),
> 	  mule-ucs 0.84.999+0.20030620-8
>
> UTF-8 characters in articles, show up as two ISO-8859-1 characters for
> me.
[...]
> I've tried both without mule-ucs, and with mule-ucs, but that doesn't
> seem to make any difference.

I'd strongly recommend to get rid of mule-ucs in Emacs 21.

> Does anyone know if this problem is caused by my choice of Emacs, my
> config (eg. locale settings?  I recently did a reinstall from the
> sarge netinst CD, and let it install for a Norwegian locale), 

Debian... check for `standard-display-european' in your site files and
delete it.

> or if this a recent bug in Gnus?

I don't think there were recent changes in Gnus that might influence
UTF-8 display.

> I noticed that an article with this problem was text/plain using
> base64 as its CTE.  Perhaps there's something there?

Please give Message Ids of those articles.  Without MIDs it's just
guessing.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: text/plain; charset=UTF-8 doesn't work anymore

[Katsumi Yamaoka]

>>>>> In <v9acrcs7fk.fsf@marauder.physik.uni-ulm.de> Reiner Steib wrote:

> I'd strongly recommend to get rid of mule-ucs in Emacs 21.

But it is essential if a user want to encode Japanese (and
possibly Chinese, Korean, etc.) text with the released versions
of Emacs 21 (i.e., 21.3 and earlier).  Try the following without
Mule-UCS.

(let ((str (string (make-char 'japanese-jisx0208 36 34))))
  (list str
	(decode-coding-string
	 (encode-coding-string str 'utf-8)
	 'utf-8)))

Re: text/plain; charset=UTF-8 doesn't work anymore

[Reiner Steib]

On Fri, Jan 14 2005, Katsumi Yamaoka wrote:

>>>>>> In <v9acrcs7fk.fsf@marauder.physik.uni-ulm.de> Reiner Steib wrote:
>
>> I'd strongly recommend to get rid of mule-ucs in Emacs 21.
>
> But it is essential if a user want to encode Japanese (and
> possibly Chinese, Korean, etc.) text with the released versions
> of Emacs 21 (i.e., 21.3 and earlier).  

Sorry, I wasn't aware of this.  I know that CJK support in Emacs 21.3
is not very good.  As Steinar is from Norway, this is probably not
very important for him, though.

I seem to remember that Mule-UCS has some problem with Emacs 21.

> Try the following without Mule-UCS.
>
> (let ((str (string (make-char 'japanese-jisx0208 36 34))))
>   (list str
> 	(decode-coding-string
> 	 (encode-coding-string str 'utf-8)
> 	 'utf-8)))

,----
| *** Welcome to IELM ***  Type (describe-mode) for help.
| ELISP> (let ((str (string (make-char 'japanese-jisx0208 36 34))))
| 	 (list str
| 	(decode-coding-string
| 	 (encode-coding-string str 'utf-8)
| 	 'utf-8)))
| ("あ" "�")
| 
| ELISP> emacs-version
| "21.3.1"
| ELISP> 
`----

Okay, it should have been the same character, I guess, but it isn't:

,----
|   character: あ (0151042, 53794, 0xd222)
|     charset: japanese-jisx0208 (JISX0208.1983/1990 Japanese Kanji: ISO-IR-87)
|  code point: 36 34
|      syntax: word
|    category: H:Japanese Hiragana characters of 2-byte character sets  
| 	     j:Japanese  
| 	     |:While filling, we can break a line at this character.  
| buffer code: 0x92 0xA4 0xA2
|   file code: not encodable by coding system iso-latin-9
|        font: -Misc-Fixed-Medium-R-Normal--14-130-75-75-C-140-JISX0208.1983-0
`----

,----
|   character: � (01175275, 326333, 0x4fabd)
|     charset: mule-unicode-e000-ffff
| 	     (Unicode characters of the range U+E000..U+FFFF.)
|  code point: 117 61
|      syntax: word
|    category:
| buffer code: 0x9C 0xF3 0xF5 0xBD
|   file code: not encodable by coding system iso-latin-9
|        font: -Misc-Fixed-Medium-R-Normal--14-130-75-75-C-70-ISO10646-1
`----

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: text/plain; charset=UTF-8 doesn't work anymore

[Steinar Bang]

>>>>> Reiner Steib <reinersteib+gmane@imap.cc>:

> I'd strongly recommend to get rid of mule-ucs in Emacs 21.

OK.  I only pulled in as an experiment, and it conflicts with nxml
anyway. 

> Debian... check for `standard-display-european' in your site files and
> delete it.

Actually it was in my ~/.emacs (which has its origins back in 1988,
and has collected a lot of... er... stuff, since then).

I've disabled it for GNU Emacs 21.

Re: text/plain; charset=UTF-8 doesn't work anymore

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

>>>>> Reiner Steib <reinersteib+gmane@imap.cc>:

>> Debian... check for `standard-display-european' in your site files and
>> delete it.

> Actually it was in my ~/.emacs (which has its origins back in 1988,
> and has collected a lot of... er... stuff, since then).

> I've disabled it for GNU Emacs 21.

And I forgot to say: that fixed the UTF-8 article problem.

Thanx!