why Chinese just shown in byte codes?

why Chinese just shown in byte codes?

[jidanni]

[-- Attachment #1: Type: text/plain, Size: 479 bytes --]

Why is the following message's Chinese just displayed in octal codes,
and not real Chinese, like mutt(1) can see?

I have to pipe the region through cat(1) to see it.

I use http://jidanni.org/comp/configuration/ and
$ dlocate -l w3m|grep ii
ii  w3m                   0.5.3-2+b1            WWW browsable pager with excellent tables/fr
ii  w3m-el-snapsho        1.4.437+0.2011        simple Emacs interface of w3m (development v
and gnus in
ii  emacs-snapshot        1:20110614-1

[-- Attachment #2: mail message --]
[-- Type: text/plain, Size: 8582 bytes --]

From bounce@couchsurfing.org Sun Jun 19 08:59:18 2011
Return-Path: <bounce@couchsurfing.org>
Received: from m.jidanni.org (69.163.137.246:993) by jidanni1.jidanni.org
  with IMAP4-SSL; 19 Jun 2011 00:59:18 -0000
X-Spam-Checker-Version: SpamAssassin 3.4.0-r1102360-1859 (2011-05-12) on
	ps11007.dreamhostps.com
X-Spam-Level: 
X-Spam-Status: No, score=-102.3 required=1.9 tests=RCVD_IN_DNSWL_MED,
	T_DKIM_INVALID,USER_IN_WHITELIST
X-Spam-Languages: en
X-Original-To: jidanni1@ps11007.dreamhostps.com
Delivered-To: jidanni1@ps11007.dreamhostps.com
Received: from homiemail-mx2.g.dreamhost.com (mailbigip.dreamhost.com [208.97.132.5])
	by ps11007.dreamhostps.com (Postfix) with ESMTP id 7BA9A2C0C162
	for <jidanni1@ps11007.dreamhostps.com>; Sat, 18 Jun 2011 17:56:21 -0700 (PDT)
Received: from messaging.couchsurfing.com (messaging.couchsurfing.com [209.51.135.26])
	by homiemail-mx2.g.dreamhost.com (Postfix) with ESMTP id 1EA124480D0
	for <jidanni@jidanni.org>; Sat, 18 Jun 2011 17:56:21 -0700 (PDT)
Received: from localhost (smtp2.couchsurfing.com [192.168.0.34])
	by messaging.couchsurfing.com (Postfix) with ESMTP id A5BB314C48A
	for <jidanni@jidanni.org>; Sun, 19 Jun 2011 00:56:20 +0000 (UTC)
X-DKIM: Sendmail DKIM Filter v2.8.3 messaging.couchsurfing.com A5BB314C48A
DKIM-Signature: v=1; a=rsa-sha1; c=simple/simple; d=couchsurfing.org;
	s=default; t=1308444980; bh=KBVz22Y+BOBTBsk7WZyMzLoM5aI=;
	h=From:To:Subject:Content-Type:Message-Id:Date;
	b=jsYYJrCSNCrvAluxO02BGmUJlQ6TO77mxsSW4naZfUty9aK7cP/4J0kAmysrwTvJQ
	 Vn6lFRtOLj/mrpIWsDFptGwd1cUNPonEF9fl6N+bTs70PJKJ80ya8xYqihcTJz6Q28
	 fmF+waHJCZSZtrtxumBoJEACvyBvdlLfJ+4Xynj8=
From: CouchSurfing <noreply@couchsurfing.org>
To: jidanni@jidanni.org
Subject: [CS  digest] Chinese Speakers
Envelope-Sender: bnce=109606816=1465046=jidanni===jidanni.org@couchsurfing.com
Errors-To: bnce=109606816=1465046=jidanni===jidanni.org@couchsurfing.com
X-Errors-To: bnce=109606816=1465046=jidanni===jidanni.org@couchsurfing.com
Content-Type: text/plain; charset="UTF-8"
 Digest for Group:
   Chinese Speakers
   http://www.couchsurfing.org/group.html?gid=42773
   Last Group Digest Sent:  Jun 19 12:55 am
Message-Id: <20110619005620.A5BB314C48A@messaging.couchsurfing.com>
Date: Sun, 19 Jun 2011 00:56:20 +0000 (UTC)


-------------------------------------------------------------
Outing For Chinese Speaker!
   Thread created: Jun 11 7:10 am    Responses: 
   Preview:
     如果这个小组到8个人以上的话，我们应该组织一次活动哈~呵呵~
   http://www.couchsurfing.org/group_read.html?gid=42773&post=9158691
   ----------------------------------------------------------

     Subject: Re: Outing For Chinese Speaker!
     From:    Cathy CHUNYAN - China
     Date:    Jun 11 1:08 pm

          no question it must be in china:P

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9161011
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9161011


     Subject: Re: Outing For Chinese Speaker!
     From:    CheFung Wu - China
     Date:    Jun 11 4:10 pm

          May be we can go "Virtual" if physical distance is a
          concern. ;)
          
          如果距离是个问题, 也许我们可以 shi yong
          "虚拟"

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9162375
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9162375


     Subject: Re: Outing For Chinese Speaker!
     From:    Yaojie Tang - Singapore
     Date:    Jun 11 4:58 pm

          OK, that will be a problem...
          Cause the people overseas will not come back to or visit
          China that often. Take me as an example, I can only make it
          on a yearly basis. So, probably, we can gather in different
          countries? Or we offer to host for each other in the future
          then? I want to visit Dalian in the future, so you can host
          me. While I can host you when you come here. 
          Can we do something like that? 
          Also, lets get more people into this group. LOL!

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9162760
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9162760


     Subject: Re: Outing For Chinese Speaker!
     From:    CheFung Wu - China
     Date:    Jun 12 4:53 am

          Wow. We have 8 (ba ge) members... ! 
          
          I was going to name the original 7 members the "Magnificent
          7", but the numbers just jumped from 6 to 8.
          
          We should call ourselves the "Core 8" or the "Essential
          Eight" as we expand... hahahha.... :)

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9166370
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9166370


     Subject: Re: Outing For Chinese Speaker!
     From:    Tzu Chi Chen - Taiwan
     Date:    Jun 13 1:59 am

          大家好呀^^~ 嘿嘿 還沒繁體字 出現..

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9173596
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9173596


     Subject: Re: Outing For Chinese Speaker!
     From:    CheFung Wu - China
     Date:    Jun 14 12:25 pm

          你好。 Welcome to our newest members:
          
          Tzu Chi Chen
          YUKO YUKO	
          Cristina Morales	
          
          It's a small group. It's not the quantity, but rather, the
          Quality.

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9187866
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9187866


-------------------------------------------------------------
Welcome
   Thread created: Jun 13 3:24 am    Responses: 
    Detail: I would like to take this opportunity to formally
      welcome all nine new members of Chinese speakers Group and
      to Thank you for joining. Your Participation and
      Contribution is greatly appreciated.    Why 華語 (Chinese
      Language)    Why not: æ¼... 
   http://www.couchsurfing.org/group_read.html?gid=42773&post=9173941
   ----------------------------------------------------------

     Subject: Welcome
     From:    CheFung Wu - China
     Date:    Jun 13 3:24 am

          I would like to take this opportunity to formally welcome
          all nine new members of Chinese speakers Group and to Thank
          you for joining. Your Participation and Contribution is
          greatly appreciated.
          
          Why 華語 (Chinese Language)
          
          Why not: 漢语， 国语， 普通话 (官話, based on
          北方話 )， 台语 (闽南语)， 粤语，赣语,
          吴语, 晉語, 客家話, 徽州話.... etc. etc.
          
          Well, I guess it's too many and too confusing. I like the
          way 華語 sounds, and it's simple. Quite international.
          
          Here you can post Chinese related information, employment,
          events, language exchange, or any thing relevant to Chinese
          Culture, Language or Arts. (In Pinyin, English, Simplified
          or Traditional Chinese)
          
          And of course, as Mr. Yaojie Tang 	suggested, let's get
          together some day. :)
          
          Again, welcome to the group and looking forward to your
          participation and contribution.

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9173941
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9173941


     Subject: Re: Welcome
     From:    CheFung Wu - China
     Date:    Jun 14 12:28 pm

          Why 華語 (Chinese Language)
          
          The original post was supposed to have this link in it.
          Oppsss.... here it is.
          
          http://en.wikipedia.org/wiki/Chinese_language

          REPLY: http://www.couchsurfing.org/group_post.html?gid=42773&reply=9187897
          THREAD: http://www.couchsurfing.org/group_read.html?gid=42773&post=9187897
This is a digest from the group Chinese Speakers http://www.couchsurfing.org/group.html?gid=42773

To stop receiving these digests, please change your group settings http://www.couchsurfing.org/group.html?gid=42773&manage=Y.

Do not reply directly to this email; use the response links above.


[-- Attachment #3: Type: text/plain, Size: 290 bytes --]

Maybe the problem is in
"
Content-Type: text/plain; charset="UTF-8"
 Digest for Group:
   Chinese Speakers
   http://www.couchsurfing.org/group.html?gid=42773
   Last Group Digest Sent:  Jun 19 12:55 am
"
Well mutt(1) can deal with it. Maybe gnus/w3m could be less strict about
formatting?

Re: why Chinese just shown in byte codes?

[Steinar Bang]

>>>>> jidanni@jidanni.org:
> Content-Type: text/plain; charset="UTF-8"
...
> Well mutt(1) can deal with it. Maybe gnus/w3m could be less strict
> about formatting?

If the content-type is text/plain, why would w3m be involved?

Re: why Chinese just shown in byte codes?

[Katsumi Yamaoka]

jidanni@jidanni.org wrote:
> Why is the following message's Chinese just displayed in octal codes,
> and not real Chinese, like mutt(1) can see?
[...]
> Maybe the problem is in
> "
> Content-Type: text/plain; charset="UTF-8"
>  Digest for Group:
>    Chinese Speakers
>    http://www.couchsurfing.org/group.html?gid=42773
>    Last Group Digest Sent:  Jun 19 12:55 am
> "

That's indeed just the cause (w3m is not, as Steinar Bang said).

> Well mutt(1) can deal with it. Maybe gnus/w3m could be less strict about
> formatting?

I don't know mutt, but I don't think every MUA worth dealing with
such a broken thing.  If you often get such mails, this will help
(waste time a bit even with normal mails, though).

(defadvice rfc2231-parse-string (before separate-charset-token activate)
  "Separate charset token being not terminated."
  (if (string-match "\\(charset=[^\t\n ]+\\)[\t\n ]*[^\t\n ;]" (ad-get-arg 0))
      (ad-set-arg 0 (concat (substring (ad-get-arg 0) 0 (match-end 1))
                            ";"
                            (substring (ad-get-arg 0) (match-end 1))))))

Re: why Chinese just shown in byte codes?

[Katsumi Yamaoka]

Katsumi Yamaoka wrote:
[...]
> If you often get such mails, this will help (waste time a bit
> even with normal mails, though).

Sorry, please replace it with:

(defadvice rfc2231-parse-string (before separate-charset-token activate)
  "Separate charset token being not terminated."
  (and (string-match "charset=[^\t\n ]+\\(?:\\'\\|\\([\t\n ]*[^\t\n ;]\\)\\)"
                     (ad-get-arg 0))
       (match-beginning 1)
       (ad-set-arg 0 (concat (substring (ad-get-arg 0) 0 (match-beginning 1))
                             ";"
                             (substring (ad-get-arg 0) (match-beginning 1))))))