Bad encoding...

Bad encoding...

[David Kastrup]

Take for example
Message-ID: <dbrlka$3f3$04$1@news.t-online.com>
from de.talk.romance

Here are some seminal points:

    Mime-Version: 1.0
    Content-Type: multipart/alternative;
            boundary="----=_NextPart_000_0070_01C58F0E.65ADA4C0"

    This is a multi-part message in MIME format.

    ------=_NextPart_000_0070_01C58F0E.65ADA4C0
    Content-Type: text/plain;
            charset="iso-8859-1"
    Content-Transfer-Encoding: quoted-printable

    Immer wieder trifft man Menschen, die gezielt handeln. In vielen =
    F=E4llen ist es meistens die Tendenz zum Negativen. Was hat es auf sich? =

Now what happens with this on my System?

Coding system for saving this buffer:
  = -- emacs-mule-unix

Default coding system (for new files):
  u -- mule-utf-8 (alias: utf-8)

Coding system for keyboard input:
  nil
Coding system for terminal output:
  u -- utf-8 (alias of mule-utf-8)

Defaults for subprocess I/O:
  decoding: u -- mule-utf-8 (alias: utf-8)

  encoding: u -- mule-utf-8 (alias: utf-8)


Priority order for recognizing coding systems when reading files:
  1. mule-utf-8 (alias: utf-8)
  2. iso-latin-1 (alias: iso-8859-1 latin-1)

[...]

My locale is
    LANG=en_US.UTF-8
    LC_CTYPE="en_US.UTF-8"
    LC_NUMERIC="en_US.UTF-8"
    LC_TIME="en_US.UTF-8"
    LC_COLLATE="en_US.UTF-8"
    LC_MONETARY="en_US.UTF-8"
    LC_MESSAGES="en_US.UTF-8"
    LC_PAPER="en_US.UTF-8"
    LC_NAME="en_US.UTF-8"
    LC_ADDRESS="en_US.UTF-8"
    LC_TELEPHONE="en_US.UTF-8"
    LC_MEASUREMENT="en_US.UTF-8"
    LC_IDENTIFICATION="en_US.UTF-8"
    LC_ALL=

The above now seemingly assembles the above quoted-printable
sequences, generates the right latin-1 characters from them, converts
them into utf-8, interprets the resulting bytes as latin-1 and
converts this reinterpretation then into Emacs MULE, showing a buffer
that looks like utf-8 ending up by accident in a latin-1 buffer (the
buffer encoding indeed is latin-1).

And indeed, recode-region from "buffer was originally encoded as utf-8"
to "latin-1" yields the correct result.

I get this kind of nonsense not too rarely, most often with multi-part
messages that specify an encoding per-part.

I think this has been happening in Emacs CVS for at least half a year
or so, but it is also possible that the behavior just was not noticed
before by me.

-- 
David Kastrup, Kriemhildstr. 15, 44793 Bochum