Postings with UTF-8 characters are mangled

Postings with UTF-8 characters are mangled

[Kai Großjohann]

Whenever I include a snippet from my summary buffers in a posting,
the posting is garbled.  Emacs declares it as UTF-16 but it is
actually (something similar to) UTF-8.

Argh.

I will make a followup to this posting which shows the problem.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Postings with UTF-8 characters are mangled

[Jesper Harder]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

> Whenever I include a snippet from my summary buffers in a posting,
> the posting is garbled.  Emacs declares it as UTF-16 but it is
> actually (something similar to) UTF-8.

It works for me with Emacs 21.2, so it's probably related to that you're
using CVS Emacs.

Test:

R.  Danny Siu               Re: HEADS UP: Largely rewritten gnus-agent implementation.
R.    ╰─╮ Danny Siu               
R.  Kai Großjohann          Postings with UTF-8 characters are mangled

Re: Postings with UTF-8 characters are mangled

[Kai Großjohann]

Jesper Harder <harder@myrealbox.com> writes:

> It works for me with Emacs 21.2, so it's probably related to that you're
> using CVS Emacs.

Hm.  It seems to work fine to just read UTF-8 files.

kai
-- 
~/.signature is: umop ap!sdn    (Frank Nobis)

Re: Postings with UTF-8 characters are mangled

[Jesper Harder]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

> Whenever I include a snippet from my summary buffers in a posting, the
> posting is garbled.  Emacs declares it as UTF-16 but it is actually
> (something similar to) UTF-8.

I think there are two problems:

1. UTF-16 seems to be broken in CVS Emacs.  I get an error when I try to
   save an ordinary file as UTF-16 -- so it's probably not Gnus related.

2. UTF-16 has higher priority than UTF-8 in CVS Emacs.  I think we
   should find a fix.  Using UTF-16 in email is bad.

I'm not sure what's best:  

* Ask the Emacs maintainers if the priority of UTF-8 and UTF-16 could be
  swapped? or

* Initialize `mm-coding-system-priorities' to something like:

(when (fboundp 'sort-coding-systems)
  (delq 'mule-utf-16-le 
	(delq 'mule-utf-16-be 
	      (sort-coding-systems (coding-system-list t)))))

Ideas?

Re: Postings with UTF-8 characters are mangled

[Kai Großjohann]

Jesper Harder <harder@ifa.au.dk> writes:

> * Ask the Emacs maintainers if the priority of UTF-8 and UTF-16 could be
>   swapped?

I think that's a good idea anyway.
-- 
Ambibibentists unite!

Re: Postings with UTF-8 characters are mangled

[Lars Magne Ingebrigtsen]

kai.grossjohann@uni-duisburg.de (Kai Großjohann) writes:

>> * Ask the Emacs maintainers if the priority of UTF-8 and UTF-16 could be
>>   swapped?
>
> I think that's a good idea anyway.

Definitely.  UTF-8 is seen much more commonly than UTF-16, and will
probably be *the* charset in the time ahead, so it makes sense to
give precedence to UTF-8.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen