mm-coding-system-priorities

mm-coding-system-priorities

[Miles Bader]

Hi,

I'm using the gnus in the Emacs source tree, so maybe this problem has
already been fixed:

`mm-coding-system-priorities' seems to only work properly if the members
are `canonical' character-set names instead of aliases, for instance
this fails to do the right thing:

   (setq mm-coding-system-priorities '(euc-jp))

but this works:

   (setq mm-coding-system-priorities (mapcar 'coding-system-base '(euc-jp)))

It seems to me that gnus ought to do this mapping itself, as many coding
systems have aliases which are more widely used than their canonical emacs
names.

Perhaps something like (untested):

--- orig/lisp/gnus/mm-util.el
+++ mod/lisp/gnus/mm-util.el
@@ -507,8 +507,11 @@
 	     ;; system that has one.
 	     (let ((systems (find-coding-systems-region b e)))
 	       (when mm-coding-system-priorities
-		 (setq systems
-		       (sort systems 'mm-sort-coding-systems-predicate)))
+		 (let ((mm-coding-system-priorities 
+			(mapcar #'coding-system-base
+				mm-coding-system-priorities)))
+		   (setq systems
+			 (sort systems 'mm-sort-coding-systems-predicate))))
 	       ;; Fixme: The `mime-charset' (`x-ctext') of `compound-text'
 	       ;; is not in the IANA list.
 	       (setq systems (delq 'compound-text systems))

Thanks,

-Miles
-- 
"Though they may have different meanings, the cries of 'Yeeeee-haw!' and
 'Allahu akbar!' are, in spirit, not actually all that different."

Re: mm-coding-system-priorities

[Jesper Harder]

Miles Bader <miles@gnu.org> writes:

> I'm using the gnus in the Emacs source tree, so maybe this problem has
> already been fixed:
>
> `mm-coding-system-priorities' seems to only work properly if the
> members are `canonical' character-set names instead of aliases,

It was fixed with this change:

2003-04-13  Reiner Steib  <Reiner.Steib@gmx.de>

	* mm-util.el (mm-sort-coding-systems-predicate): Convert elements
	of `mm-coding-system-priorities' to base coding system.

Re: mm-coding-system-priorities

[Reiner Steib]

On Wed, Mar 24 2004, Miles Bader wrote:

> I'm using the gnus in the Emacs source tree, so maybe this problem has
> already been fixed:
>
> `mm-coding-system-priorities' seems to only work properly if the members
> are `canonical' character-set names instead of aliases, 

I have fixed this (before 5.10.1 was released):

,----
| 2003-04-13  Reiner Steib  <Reiner.Steib@gmx.de>
| 
| 	* mm-util.el (mm-sort-coding-systems-predicate): Convert elements
| 	of `mm-coding-system-priorities' to base coding system.
`----

> It seems to me that gnus ought to do this mapping itself, as many coding
> systems have aliases which are more widely used than their canonical emacs
> names.

I agree.

> Perhaps something like (untested):
>
> --- orig/lisp/gnus/mm-util.el
> +++ mod/lisp/gnus/mm-util.el
> @@ -507,8 +507,11 @@
>  	     ;; system that has one.
>  	     (let ((systems (find-coding-systems-region b e)))
>  	       (when mm-coding-system-priorities
> -		 (setq systems
> -		       (sort systems 'mm-sort-coding-systems-predicate)))
> +		 (let ((mm-coding-system-priorities 
> +			(mapcar #'coding-system-base
> +				mm-coding-system-priorities)))
> +		   (setq systems
> +			 (sort systems 'mm-sort-coding-systems-predicate))))
>  	       ;; Fixme: The `mime-charset' (`x-ctext') of `compound-text'
>  	       ;; is not in the IANA list.
>  	       (setq systems (delq 'compound-text systems))

I've done it in `mm-sort-coding-systems-predicate' (but I don't
remember exactly why I did it in this way).  If you think that your
solution is better, we may change it.

(defun mm-sort-coding-systems-predicate (a b)
  (let ((priorities
	 (mapcar (lambda (cs)
		   ;; Note: invalid entries are dropped silently
		   (and (coding-system-p cs)
			(coding-system-base cs)))
		 mm-coding-system-priorities)))
    (> (length (memq a priorities))
       (length (memq b priorities)))))

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: mm-coding-system-priorities

[Miles Bader]

Reiner Steib <4.uce.03.r.s@nurfuerspam.de> writes:
> I've done it in `mm-sort-coding-systems-predicate' (but I don't
> remember exactly why I did it in this way).  If you think that your
> solution is better, we may change it.

The difference would seem to be that doing it in
`mm-sort-coding-systems-predicate' ends up doing the mapcar around log-N
times, whereas temporarily rebinding mm-coding-system-priorities just
does it once.  I think N is often around ~30-40 if the message contains
any non-ascii characters, so it may be worth changing (though
mm-coding-system-priorities is probably usually fairly short, in which
case each mapcar is not all that expensive).

-Miles
-- 
80% of success is just showing up.  --Woody Allen

charset=macintosh

[Karl Eichwalder]

[-- Attachment #1: Type: text/plain, Size: 653 bytes --]

Oort Gnus v0.16
GNU Emacs 21.3.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
 of 2003-03-05 on tux
200 gnu.franken.de InterNetNews server INN 2.3.2 ready

I wrote to attached message using the TeX iput method (C-x RET c TeX RET).
Especially I added these quotes: \ldq \rdq and then \ldots and \euro .

Sending the message, Gnus choose this encoding:

MIME-Version: 1.0
Content-Type: text/plain; charset=macintosh
Content-Transfer-Encoding: quoted-printable

I would have expected UTF-8 since UTF-8 is more standard than
"macintosh"; I know you can set your preferences somehow -- but I'd
like to see Gnus doing the right thing out of the box.


[-- Attachment #2: Type: message/rfc822, Size: 601 bytes --]

From: Karl Eichwalder <keichwa@gmx.net>
To: ke@gnu.franken.de
Subject: encoding test
Date: Fri, 07 Mar 2003 18:28:35 +0100
Message-ID: <shzno73x30.fsf@tux.gnu.franken.de>

“quotes”
and more…
Grüß Gott.

€ 1.

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.gnu.franken.de/ke/                            |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)


[-- Attachment #3: Type: text/plain, Size: 271 bytes --]

-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.gnu.franken.de/ke/                            |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: charset=macintosh

[Simon Josefsson]

Karl Eichwalder <keichwa@gmx.net> writes:

> Oort Gnus v0.16
> GNU Emacs 21.3.50.1 (i686-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
>  of 2003-03-05 on tux
> 200 gnu.franken.de InterNetNews server INN 2.3.2 ready
>
> I wrote to attached message using the TeX iput method (C-x RET c TeX RET).
> Especially I added these quotes: \ldq \rdq and then \ldots and \euro .
>
> Sending the message, Gnus choose this encoding:
>
> MIME-Version: 1.0
> Content-Type: text/plain; charset=macintosh
> Content-Transfer-Encoding: quoted-printable
>
> I would have expected UTF-8 since UTF-8 is more standard than
> "macintosh"; I know you can set your preferences somehow -- but I'd
> like to see Gnus doing the right thing out of the box.

Me too, but I'm not sure if Gnus should override Emacs' coding system
ordering, and emacs prefers mac-roman over utf-8.  The reason Emacs
doesn't list UTF-8 higher than macintosh is, supposedly, that the
UTF-8 support is not complete.  Once it is complete, the emacs
maintainers will probably change the preference ordering.

Fortunately it is easy to override the priorities within Gnus:

 '(mm-coding-system-priorities (quote (iso-latin-1 iso-latin-9 utf-8)))

Of course, the die-hard solution is to run emacs in a UTF-8 locale,
then UTF-8 is preferred everywhere.  I use this without problems, but
I do use the above line to downgrade to 8859-1/8859-15 in mail,
whenever possible.

Hm.  The UTF-8 discussion in Emacs 21.2 PROBLEMS is not present in
Emacs 21.3 prerelease, so perhaps it has been fixed.  I'll ask on
emacs-devel whether utf-8 should be preferred over mac-roman.

mm-coding-system-priorities (was: charset=macintosh)

[Reiner Steib]

On Fri, Mar 07 2003, Simon Josefsson wrote:

> The reason Emacs doesn't list UTF-8 higher than macintosh is,
> supposedly, that the UTF-8 support is not complete.  Once it is
> complete, the emacs maintainers will probably change the preference
> ordering.

Apparently it has already been changed in Emacs 21.3, see below.

> Fortunately it is easy to override the priorities within Gnus:
>  '(mm-coding-system-priorities (quote (iso-latin-1 iso-latin-9 utf-8)))
>
> Hm.  The UTF-8 discussion in Emacs 21.2 PROBLEMS is not present in
> Emacs 21.3 prerelease, so perhaps it has been fixed.  I'll ask on
> emacs-devel whether utf-8 should be preferred over mac-roman.

Sample text: “quotes”

With Emacs 21.3.1 and current Oort [1] I get (in mml-preview):

,----
| (setq mm-coding-system-priorities nil)
| ==> charset=utf-8
| 
| (setq mm-coding-system-priorities '(macintosh utf-8))
| ==> charset=utf-8
| 
| (setq mm-coding-system-priorities '(mac-roman utf-8))
| ==> charset=macintosh
`----

Adding windows-1252 [2] gives:

,----
| (setq mm-coding-system-priorities nil)
| ==> charset=utf-8
| 
| (setq mm-coding-system-priorities '(windows-1252 macintosh utf-8))
| ==> charset=windows-1252
| 
| (setq mm-coding-system-priorities '(utf-8 macintosh windows-1252))
| ==> charset=windows-1252
| 
| (setq mm-coding-system-priorities '(mule-utf-8 windows-1252))
| ==> charset=utf-8
`----

Apparently, we need to write `mule-utf-8' and `mac-roman' instead of
`utf-8' and `macintosh', respectively.

,----[ C-h v mm-coding-system-priorities RET ]
| mm-coding-system-priorities's value is nil
| 
| Documentation:
| Preferred coding systems for encoding outgoing mails.
`----

So `mm-coding-system-priorities' expects an Emacs coding system and
not a MIME charset.  But OTOH...

,----[ M-x describe-coding-system utf-8 RET ]
| u -- utf-8 (alias of mule-utf-8)
`----

Hm, does it even expect the original name of the coding-system and
*not* an alias?  If so, I think this should be made more clear in the
documentation.

Bye, Reiner.

[1] Oort Gnus v0.18
In GNU Emacs 21.3.1 (i586-pc-linux-gnu, X toolkit, Xaw3d scroll bars)
configured using `configure  --prefix=/soft/local/upp/emacs'
Important settings:
  value of $LC_CTYPE: en_US.ISO_8859-1
  value of $LANG: C
  locale-coding-system: iso-latin-1
  default-enable-multibyte-characters: t

[2] http://theotp1.physik.uni-ulm.de/~ste/comp/emacs/gnus/rs-windows-1252.el
(require 'rs-windows-1252)
(rs-use-windows-1252)

-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: mm-coding-system-priorities

[Lars Magne Ingebrigtsen]

Reiner Steib <4.uce.03.r.s@nurfuerspam.de> writes:

> So `mm-coding-system-priorities' expects an Emacs coding system and
> not a MIME charset.  But OTOH...
>
> ,----[ M-x describe-coding-system utf-8 RET ]
> | u -- utf-8 (alias of mule-utf-8)
> `----
>
> Hm, does it even expect the original name of the coding-system and
> *not* an alias?  If so, I think this should be made more clear in the
> documentation.

utf-8 is both a coding system and a charset, I think.  (And I don't
think it should matter whether you use the name or an alias.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen

Re: mm-coding-system-priorities

[Reiner Steib]

On Sat, Apr 12 2003, Lars Magne Ingebrigtsen wrote:

> Reiner Steib <4.uce.03.r.s@nurfuerspam.de> writes:
>> ,----[ M-x describe-coding-system utf-8 RET ]
>> | u -- utf-8 (alias of mule-utf-8)
>> `----
>>
>> Hm, does it even expect the original name of the coding-system and
>> *not* an alias?  If so, I think this should be made more clear in the
>> documentation.
>
> utf-8 is both a coding system and a charset, I think.

AFAICS, we need to distinguish between "base coding-system",
"aliased/subsidiary coding-system" and mime-charset.  utf-8 is both,
an "aliased/subsidiary coding-system" and a mime-charset, but not a
"base coding-system".  The problem is, that using utf-8 in
`mm-coding-system-priorities' has no effect[1] at all: You need to use
a non-aliased coding-system, as returned by `coding-system-base':

,----[ C-h f coding-system-base RET ]
| coding-system-base is a compiled Lisp function in `international/mule'.
| (coding-system-base CODING-SYSTEM)
| 
| Return the base coding system of CODING-SYSTEM.
| A base coding system is what made by `make-coding-system'.
| Any alias nor subsidiary coding systems are not base coding system.
`----

The doc-string of `mm-coding-system-priorities' doesn't mention this.
Even the default value for users with "current-language-environment =
Japanese" contains the bogus entry utf-8.

> (And I don't think it should matter whether you use the name or an
> alias.)

Changing `mm-sort-coding-systems-predicate' should achieve it (I have
committed this):

(defun mm-sort-coding-systems-predicate (a b)
  (let ((priorities
	 (mapcar (lambda (cs)
		   ;; Note: invalid entries are dropped silently
		   (and (coding-system-p cs)
			(coding-system-base cs)))
		 mm-coding-system-priorities)))
    (> (length (memq a priorities))
       (length (memq b priorities)))))

Bye, Reiner.

[1] See my test in <news:v965pyw1gv.fsf@marauder.physik.uni-ulm.de>
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: mm-coding-system-priorities

[Lars Magne Ingebrigtsen]

Reiner Steib <4.uce.03.r.s@nurfuerspam.de> writes:

> | A base coding system is what made by `make-coding-system'.
> | Any alias nor subsidiary coding systems are not base coding system.

Ah, right; I didn't know that.

> Changing `mm-sort-coding-systems-predicate' should achieve it (I have
> committed this):

Cool.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen