Re: Garbled display of UTF-8 encoded mails

[Sven Joachim <svenjoac@gmx.de>]

On 2010-11-23 21:11 +0100, Sven Joachim wrote:

> Hi,
>
> during the last weeks I encountered a problem viewing articles (mainly
> mails) with non-ASCII characters, e.g. a German Umlaut "Ü" is displayed
> as an octal digit, \334.  This happens only if the message is encoded
> like this:
>
> Content-Type: text/plain; charset=utf-8
> Content-Transfer-Encoding: 8bit
>
> It does not happen with charset=iso-8859-1 or Content-Transfer-Encoding:
> quoted-printable (thus you cannot reproduce it with articles from
> Gmane).  Any idea what might have caused this?

I bisected the problem and found out that commit bda3e8962a is the
culprit:

commit bda3e8962af0aee90144c3ae8c5360aa4c106d94
Author: Katsumi Yamaoka <yamaoka@jpl.org>
Date:   Fri May 7 06:34:41 2010 +0000

    * binhex.el (binhex-decode-region-internal)
    * dns.el (dns-read-string-name, dns-write, dns-read, dns-read-type)
    (dns-query)
    * nnweb.el (nnweb-gmane-search)
    * pgg-parse.el (pgg-parse-armor)
    * pgg.el (pgg-verify-region)
    * sha1.el (sha1-string-external)
    * uudecode.el (uudecode-decode-region-internal)
    * yenc.el (yenc-decode-region): Don't run set-buffer-multibyte for
    XEmacs.
    
    * gnus-art.el (gnus-article-browse-html-parts)
    * gnus-group.el (gnus-read-ephemeral-gmane-group)
    (gnus-read-ephemeral-bug-grou): Use mm-make-temp-file instead of
    make-temp-file.
    
    * gnus-dired.el (gnus-dired-mode): Bind gnus-dired-mode-hook,
    gnus-dired-mode-on-hook and gnus-dired-mode-off-hook for XEmacs when
    compiling.
    
    * gnus-ml.el (gnus-mailing-list-mode): Bind gnus-mailing-list-mode-hook,
    gnus-mailing-list-mode-on-hook and gnus-mailing-list-mode-off-hook for
    XEmacs when compiling.
    
    * gnus-salt.el (gnus-pick-mode): Bind gnus-pick-mode-on-hook and
    gnus-pick-mode-off-hook for XEmacs when compiling.
    (gnus-binary-mode): Bind gnus-binary-mode-on-hook and
    gnus-binary-mode-off-hook for XEmacs when compiling.
    
    * gnus-sum.el (gnus-summary-limit-strange-charsets-predicate): Return
    nil if char-charset is not available.
    
    * imap.el (imap-disable-multibyte)
    * sieve-manage.el (sieve-manage-disable-multibyte): Redefine them as a
    macro.
    
    * mm-url.el (mm-url-form-encode-xwfu): Use mm-encode-coding-string
    instead of encode-coding-string.
    
    * mm-util.el (mm-enable-multibyte, mm-disable-multibyte): Use (featurep
    'xemacs) instead of mm-emacs-mule to switch function definitions.
    (mm-with-unibyte-current-buffer): Make it a progn macro for XEmacs.
    
    * lpath.el: Fbind delete-overlay and overlay-lists for XEmacs;
    bind temporary-file-directory for XEmacs;
    fbind make-temp-file, set-buffer-multibyte, string-as-multibyte and
    timer-set-function for XEmacs 21.4 and SXEmacs;
    bind timer-list for XEmacs 21.4 and SXEmacs;
    fbind char-charset and find-charset-region for non-Mule XEmacs;
    fbind decode-coding-region, decode-coding-string, detect-coding-region,
    encode-coding-region and encode-coding-string for XEmacs having no
    file-coding feature.


I refrain from including the whole diff which is more than 500 lines and
touches 20 files. :-/  Going to find out what exactly broke things
tomorrow, unless somebody (Katsumi?) beats me to it.

Sven