Trouble displaying 8bit postings properly

Trouble displaying 8bit postings properly

[Karl Eichwalder]

Can you still access this posting?

    From: Edmund Lauterbach <e.lauterbach@gmx.de>
    Newsgroups: de.etc.bahn.tarif+service
    Subject: Re: PEP - Grundsaetzliches  und Machbares
    Date: Sat, 15 Mar 2003 00:08:14 +0100
    Message-ID: <3E7260DE.77CC579B@gmx.de>

Emacs started in an UTF-8 locale (LANG=de_DE.UTF-8 emacs), failed to
display the umplauts properly; the umlauts are displayed correctly when
I type 'C-u g'

The same is to be observed for this article (where '1 g ISO-8859-1 RET'
fails):

    From: mathias.boelckow@gmx.net (Mathias Boelckow)
    Newsgroups: de.etc.bahn.tarif+service
    Subject: Mein einfaches Preissystem MEP, was: PEP - Grundsaetzliches  und Machbares
    Date: Sun, 16 Mar 2003 00:46:28 GMT
    Organization: none
    Lines: 149
    Message-ID: <3e73794b.14305425@news.cis.dfn.de>


-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.gnu.franken.de/ke/                            |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: Trouble displaying 8bit postings properly

[Jesper Harder]

Karl Eichwalder <keichwa@gmx.net> writes:

> Can you still access this posting?
>
>     Message-ID: <3E7260DE.77CC579B@gmx.de>
>
> Emacs started in an UTF-8 locale (LANG=de_DE.UTF-8 emacs), failed to
> display the umplauts properly; the umlauts are displayed correctly when
> I type 'C-u g'
>
> The same is to be observed for this article (where '1 g ISO-8859-1 RET'
> fails):

You can do `C-u W M c latin-1' instead.

>     Message-ID: <3e73794b.14305425@news.cis.dfn.de>

The problem in both cases is that the articles contain octets that are
invalid in Latin-1.  

1. The first seems to be slightly corrupted for some reason -- notice the
   line ending with "Preissystem begr\201át".

2. The second is actually encoded in the evil Windows-1252 charset --
   the hyphen in "Hamburg-Rostock" is an n-dash.

The real solution to 2. is to have Emacs know about Windows-1252, which
it doesn't.  I think it's supported in the Unicode branch, though.

The first case is more difficult.  It's a tradeoff -- it would have
displayed correctly with the old code which doesn't try guess the right
encoding.  On the other hand, all latin-1 articles without a MIME
charset would be displayed incorrectly with the old code.

I think valid latin-1 articles without a charset declaration are more
common than than articles with a strange corruption -- so it's probably
a reasonable tradeoff [1].  What do you think?

[1] But note that these problems only apply in an UTF-8 locale -- both
articles display correctly in a latin-1 locale (except for the invalid
octets, of course).

Re: Trouble displaying 8bit postings properly

[Reiner Steib]

On Sun, Mar 16 2003, Jesper Harder wrote:

> The real solution to 2. is to have Emacs know about Windows-1252, which
> it doesn't.  I think it's supported in the Unicode branch, though.

Windows-1252 is supported in CVS-HEAD of Emacs.
Just say (require 'code-pages).

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: Trouble displaying 8bit postings properly

[Jesper Harder]

Reiner Steib <4.uce.03.r.s@nurfuerspam.de> writes:

> On Sun, Mar 16 2003, Jesper Harder wrote:
>
>> The real solution to 2. is to have Emacs know about Windows-1252, which
>> it doesn't.  I think it's supported in the Unicode branch, though.
>
> Windows-1252 is supported in CVS-HEAD of Emacs.
> Just say (require 'code-pages).

Thanks, I hadn't noticed that there's both the old 'codepages.el' and
the new 'code-pages.el'.

This will work, but you have to add Windows-1252 to the coding system
priorities explicitly -- otherwise Emacs won't detect it.  

But be sure _not_ to add it to the front, else Gnus will send out
windows-1252 crap just like Outlook Express.  Hmm, maybe we should put
in a check for this to avoid that embarrasment.