XEmacs, Gnus and mm-coding-system priorities.

XEmacs, Gnus and mm-coding-system priorities.

[Aidan Kehoe]

[-- Attachment #1: message body and .signature --]
[-- Type: text/plain, Size: 2528 bytes --]


Hi!

I’ve written a patch for Gnus that handles MIME character set selection more
intelligently under XEmacs. With this change, and with Stephen Turnbull’s
latin-unity available—as is usual under XEmacs with Mule—you can set
mm-coding-system-priorities to an ordered list, and Gnus will choose the
first coding system that can encode the message, as the MIME character set
to send. This is the case even with multiple character sets in a message, a
case that’s badly handled at the moment.

Katsumi Yamaoka; In b9yeki8faw8.fsf@jpl.org on gnu.emacs.gnus, you say that
mm-coding-system-priorities should be honoured by the latest Gnus even in
XEmacs. I’m unclear as to how this is possible, given that
mm-use-find-coding-systems-region must always be nil under XEmacs
(find-coding-systems-region doesn’t exist in it) and the only place the
variable is referenced is after checking if it’s “t”. Cf. line 594 of
mm-util.el. 

Reiner Steib, in message v9oehcoozg.fsf%40marauder.physik.uni-ulm.de ,
suggests that I separate out the bulk of my changes into a separate
function, which I’ve done in the attached patch. I’ve tested the patch on 
GNU Emacs 21.3.1 with the CVS trunk version of Gnus, and no change was
observed, as expected. I’ve also tested the patch with the CVS trunk version
of Gnus, and XEmacs 21.4.13, but CVS Gnus is unable to connect to a NNTP
server for me with that combination. 21.5.18 crashes for me when I try to do
the same thing. 

With the stable Gnus that Debian installed with 21.4, 5.10.2, the patch
works fine for me. Ditto for the version a recent XEmacs sumo tarball had,
5.10.6.

As well as the patch, I’ve attached a file that tests what it does. Since it
only does anything on XEmacs, you’ll need a copy of XEmacs to run it. Both
files are available uncompressed over the web at
http://parhasard.net/mm-xemacs-coding-systems.diff and
http://parhasard.net/mm-xemacs-coding-test.el , respectively.  

Reiner also tells me that I need to send a copyright assignment to the FSF
to get this added to mainline Gnus; where do I get the forms for this? Can I
download PDFs and print them out?

Best regards,  

	- Aidan
-- 
“As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious day
the plain folks of the land will reach their heart’s desire at last and the
White House will be adorned by a downright moron.” – H.L. Mencken 


[-- Attachment #2: Diff to have XEmacs pay attention to mm-coding-system-priorities --]
[-- Type: application/octet-stream, Size: 1913 bytes --]

[-- Attachment #3: Test Emacs lisp to check the previous app. --]
[-- Type: application/octet-stream, Size: 1015 bytes --]

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

>>>>> In <16816.39575.977351.530618.dm0@vm.parhasard.net> Aidan Kehoe wrote:

> Katsumi Yamaoka; In b9yeki8faw8.fsf@jpl.org on gnu.emacs.gnus, you say that
> mm-coding-system-priorities should be honoured by the latest Gnus even in
> XEmacs. I’m unclear as to how this is possible, given that
> mm-use-find-coding-systems-region must always be nil under XEmacs
> (find-coding-systems-region doesn’t exist in it) and the only place the
> variable is referenced is after checking if it’s “t”. Cf. line 594 of
> mm-util.el.

It is natural that mm-use-find-coding-systems-region is nil in
XEmacs.  The way to determine the MIME charset in XEmacs is:

1. Find a list of MULE charsets using `find-charset-region' by
   way of `mm-find-charset-region'.

2. Convert each MULE charset into MIME charset using
   `mm-mime-charset'.  It calls `mm-mule-charset-to-mime-charset'
   which refers to `mm-coding-system-priorities' by way of
   `mm-sort-coding-systems-predicate'.

In XEmacs, the default order of preferred MIME charsets is
specified in order of `mm-mime-mule-charset-alist'.  And
`mm-coding-system-priorities' enables users to modify the order.

I hardly know what latin-unity does as I wrote in the
gnu.emacs.gnus newsgroup, but it seems to me that the latest
mm-util.el doesn't have an ability to convert Latin-9 to Latin-1.
For instance, the following form

(let ((mm-coding-system-priorities '(iso-8859-1 iso-8859-15)))
  (rfc2047-encode-string (string (make-char 'latin-iso8859-15 95))))

returns "=?iso-8859-1?Q?=DF?=" for Emacs, but
"=?iso-8859-15?Q?=DF?=" for XEmacs.

I tried your patch with the old mm-util.el and confirmed it
works.  However, it doesn't work with non-Latin characters:

(let ((mm-coding-system-priorities '(shift_jis)))
  (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))

It returns "=?shift_jis?Q?=83=C0?=" for Emacs, but
"=?iso-2022-jp?B?GyRCJkIbKEI=?=" for XEmacs.

Although the default order of `mm-mime-mule-charset-alist' is
just suitable for Japanese messages and we don't need to modify
`mm-coding-system-priorities' in Japan, we will need to much
improve it.  I will study your patch and look for the way to
incorporate the latin-unity feature in the latest mm-util.el.

Regards,

Re: XEmacs, Gnus and mm-coding-system priorities.

[Aidan Kehoe]

 Ar an séiú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

 > [...] I tried your patch with the old mm-util.el and confirmed it works.
 > However, it doesn't work with non-Latin characters:
 > 
 > (let ((mm-coding-system-priorities '(shift_jis)))
 >   (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))
 > 
 > It returns "=?shift_jis?Q?=83=C0?=" for Emacs, but
 > "=?iso-2022-jp?B?GyRCJkIbKEI=?=" for XEmacs.

Yes, that does make sense, because Latin Unity doesn’t yet work with
non-ISO, non-Unicode character sets. (If it ever will.)

Right, given that, the patch can’t solve the general case. You should
probably leave it out of the trunk until Latin Unity gets support for the
East Asian character sets, if it ever does. (And it might need a name change
at that point :-) Shame, because it did eliminate the most annoying thing I
find about Gnus on Mule; I _never_ want to send messages as iso-2022-jp, nor
as multipart messages split by character set.  

 > Although the default order of `mm-mime-mule-charset-alist' is just
 > suitable for Japanese messages and we don't need to modify
 > `mm-coding-system-priorities' in Japan, we will need to much improve it.
 > I will study your patch and look for the way to incorporate the
 > latin-unity feature in the latest mm-util.el.

Okay. Note that in Western Europe, the right thing to do is probably to send
a message in iso-8859-1 if its contents fit into that character set,
iso-8859-15 if it has the Euro sign, and UTF-8 if it has anything more
obscure than that. The current behaviour doesn’t really allow us to tell
Gnus that in XEmacs. 

-- 
“As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious day
the plain folks of the land will reach their heart’s desire at last and the
White House will be adorned by a downright moron.” – H.L. Mencken 

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

[-- Attachment #0: Type: message/rfc822, Size: 2483 bytes --]

From: Aidan Kehoe <kehoea@parhasard.net>
To: Katsumi Yamaoka <yamaoka@jpl.org>
Cc: ding@gnus.org
Subject: Re: XEmacs, Gnus and mm-coding-system priorities.
Date: Mon, 6 Dec 2004 13:35:46 +0000
Message-ID: <16820.24626.23607.206081.dm0@vm.parhasard.net>


 Ar an séiú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

 > [...] I tried your patch with the old mm-util.el and confirmed it works.
 > However, it doesn't work with non-Latin characters:
 > 
 > (let ((mm-coding-system-priorities '(shift_jis)))
 >   (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))
 > 
 > It returns "=?shift_jis?Q?=83=C0?=" for Emacs, but
 > "=?iso-2022-jp?B?GyRCJkIbKEI=?=" for XEmacs.

Yes, that does make sense, because Latin Unity doesn’t yet work with
non-ISO, non-Unicode character sets. (If it ever will.)

Right, given that, the patch can’t solve the general case. You should
probably leave it out of the trunk until Latin Unity gets support for the
East Asian character sets, if it ever does. (And it might need a name change
at that point :-) Shame, because it did eliminate the most annoying thing I
find about Gnus on Mule; I _never_ want to send messages as iso-2022-jp, nor
as multipart messages split by character set.  

 > Although the default order of `mm-mime-mule-charset-alist' is just
 > suitable for Japanese messages and we don't need to modify
 > `mm-coding-system-priorities' in Japan, we will need to much improve it.
 > I will study your patch and look for the way to incorporate the
 > latin-unity feature in the latest mm-util.el.

Okay. Note that in Western Europe, the right thing to do is probably to send
a message in iso-8859-1 if its contents fit into that character set,
iso-8859-15 if it has the Euro sign, and UTF-8 if it has anything more
obscure than that. The current behaviour doesn’t really allow us to tell
Gnus that in XEmacs. 

-- 
“As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious day
the plain folks of the land will reach their heart’s desire at last and the
White House will be adorned by a downright moron.” – H.L. Mencken 

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 121 bytes --]

It is strange that Aidan Kehoe's message replying to me and the
ding list didn't appear in the list again.  Here it is:


[-- Attachment #2: Type: message/rfc822, Size: 2591 bytes --]

From: Aidan Kehoe <kehoea@parhasard.net>
To: Katsumi Yamaoka <yamaoka@jpl.org>
Cc: ding@gnus.org
Subject: Re: XEmacs, Gnus and mm-coding-system priorities.
Date: Mon, 6 Dec 2004 13:35:46 +0000
Message-ID: <16820.24626.23607.206081.dm0@vm.parhasard.net>

 Ar an séiú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

 > [...] I tried your patch with the old mm-util.el and confirmed it works.
 > However, it doesn't work with non-Latin characters:
 > 
 > (let ((mm-coding-system-priorities '(shift_jis)))
 >   (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))
 > 
 > It returns "=?shift_jis?Q?=83=C0?=" for Emacs, but
 > "=?iso-2022-jp?B?GyRCJkIbKEI=?=" for XEmacs.

Yes, that does make sense, because Latin Unity doesn’t yet work with
non-ISO, non-Unicode character sets. (If it ever will.)

Right, given that, the patch can’t solve the general case. You should
probably leave it out of the trunk until Latin Unity gets support for the
East Asian character sets, if it ever does. (And it might need a name change
at that point :-) Shame, because it did eliminate the most annoying thing I
find about Gnus on Mule; I _never_ want to send messages as iso-2022-jp, nor
as multipart messages split by character set.  

 > Although the default order of `mm-mime-mule-charset-alist' is just
 > suitable for Japanese messages and we don't need to modify
 > `mm-coding-system-priorities' in Japan, we will need to much improve it.
 > I will study your patch and look for the way to incorporate the
 > latin-unity feature in the latest mm-util.el.

Okay. Note that in Western Europe, the right thing to do is probably to send
a message in iso-8859-1 if its contents fit into that character set,
iso-8859-15 if it has the Euro sign, and UTF-8 if it has anything more
obscure than that. The current behaviour doesn’t really allow us to tell
Gnus that in XEmacs. 

-- 
“As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious day
the plain folks of the land will reach their heart’s desire at last and the
White House will be adorned by a downright moron.” – H.L. Mencken 

Re: XEmacs, Gnus and mm-coding-system priorities.

[Aidan Kehoe]

[-- Attachment #1: Type: text/plain, Size: 1602 bytes --]


Hi! again, 

 Ar an séiú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

 > [...] I tried your patch with the old mm-util.el and confirmed it works.
 > However, it doesn't work with non-Latin characters:
 > 
 > (let ((mm-coding-system-priorities '(shift_jis)))
 >   (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))

I was wrong, I can work around this. Since latin-unity knows the list of
coding systems it can map into, we can check each entry in
mm-coding-system-priorities for validity before trying to remap with it. So,
for example, if shift_jis is in mm-coding-system-priorities, the latin unity
code should just give up, and mm-find-mime-charset falls back to the code
you had written.

I~ve a revised patch and test set (including your tests from the last mail)
attached that addresses this. I~m also going to post this to gnu.emacs.gnus
because ding@gnus evidently doesn~t like me :-) .

In terms of usability, I~m starting to feel strongly that
mm-coding-systems-priorities should be initialised to '(iso-8859-1
iso-8859-15 iso-8859-2 iso-8859-16 utf-8) for non-East-Asian
locales. Americans don~t care, but for non-English speaking Europeans the
Mule breakage just another reason not to use an Emacs. 

Best regards, 

        - Aidan
-- 
~As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious day
the plain folks of the land will reach their heart~s desire at last and the
White House will be adorned by a downright moron.~ ~ H.L. Mencken 


[-- Attachment #2: Further revision of mm-util.el to behave better under XEmacs. --]
[-- Type: text/x-patch, Size: 4116 bytes --]

--- mm-util.el.orig	2004-12-06 23:20:17.000000000 +0000
+++ mm-util.el	2004-12-06 23:54:27.000000000 +0000
@@ -584,6 +584,73 @@
 		(length (memq (coding-system-base b) priorities)))
 	   t))))
 
+(defun mm-xemacs-find-mime-charset (begin end)
+  "Determine which MIME charset to use to send region as message.
+This uses the XEmacs-specific latin-unity package to better handle the case
+where identical characters from diverse ISO-8859-? character sets can be
+encoded using a single one of the corresponding coding systems. 
+
+It treats `mm-coding-system-priorities' as the list of preferred coding
+systems; a useful example setting for this list in Western Europe would be
+'(iso-8859-1 iso-8859-15 utf-8), which would default to the very standard
+Latin 1 coding system, and only move to coding systems that are less
+supported as is necessary to encode the characters that exist in the buffer.
+
+Latin Unity doesn't know about those non-ASCII Roman characters that are
+available in various East Asian character sets.  As such, its behavior if
+you have a JIS 0212 LATIN SMALL LETTER A WITH ACUTE in a buffer and it can
+otherwise be encoded as Latin 1, won't be ideal.  But this is very much a
+corner case, so don't worry about it.  "
+  (let ((systems mm-coding-system-priorities) csets psets curset chars-region)
+
+    ;; Load the Latin Unity library, if available.
+    (when (and (not (featurep 'latin-unity)) (locate-library "latin-unity"))
+      (require 'latin-unity))
+
+    ;; Now, can we use it?
+    (if (featurep 'latin-unity)
+	(progn 
+	  (assert (featurep 'xemacs) 
+		  (concat "We only expect latin-unity on XEmacs--this code "
+			  "will break on the FSF's Emacs.  "))
+	  (setq csets (latin-unity-representations-feasible-region begin end)
+		psets (latin-unity-representations-present-region begin end)
+		chars-region (delq 'ascii (charsets-in-region begin end)))
+
+	  (catch 'done
+
+	    ;; Pass back the first coding system in the preferred list that
+	    ;; can encode the whole region.
+	    (dolist (curset systems)
+	      (setq curset (latin-unity-massage-name curset 'buffer-default))
+
+	      ;; If the coding system is a universal coding system, then it
+	      ;; can certainly encode all the characters in the region. 
+	      (if (memq curset latin-unity-ucs-list)
+		  (throw 'done (list curset)))
+
+	      ;; If a coding system isn't universal, and isn't in the list
+	      ;; that latin unity knows about, we can't decide whether to
+	      ;; use it here. Leave that until later in mm-find-mime-charset
+	      ;; region function, whence we have been called.
+	      (unless (memq curset latin-unity-coding-systems)
+		(throw 'done nil))
+
+	      ;; Right, we know about this coding system, and it may
+	      ;; conceivably be able to encode all the characters in the
+	      ;; region.
+	      (if (latin-unity-maybe-remap begin end curset csets psets t)
+		  (throw 'done (list curset))))
+
+	    ;; Can't encode using anything from the
+	    ;; mm-coding-system-priorities list. Leave mm-find-mime-charset
+	    ;; to do most of the work.
+	    nil))
+
+      ;; Right, latin unity isn't available; let mm-find-charset-region 
+      ;; take its default action, which equally applies to GNU Emacs. 
+      nil)))
+
 (defun mm-find-mime-charset-region (b e &optional hack-charsets)
   "Return the MIME charsets needed to encode the region between B and E.
 nil means ASCII, a single-element list represents an appropriate MIME
@@ -625,8 +692,12 @@
 			 (setq systems nil
 			       charsets (list cs))))))
 	       charsets))
-	;; Otherwise we're not multibyte, we're XEmacs, or a single
-	;; coding system won't cover it.
+        ;; If we're XEmacs, and some coding system is appropriate,
+        ;; mm-xemacs-find-mime-charset will return an appropriate list.
+        ;; Otherwise, we'll get nil, and the next setq will get invoked.
+        (setq charsets (mm-xemacs-find-mime-charset b e))
+
+        ;; We're not multibyte, or a single coding system won't cover it.
 	(setq charsets
 	      (mm-delete-duplicates
 	       (mapcar 'mm-mime-charset

[-- Attachment #3: Test Lisp for the mm-util.el revision. --]
[-- Type: application/emacs-lisp, Size: 3397 bytes --]

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 206 bytes --]

>>>>> In <b9ypt1n83gc.fsf@jpl.org> Katsumi Yamaoka wrote:

> It is strange that Aidan Kehoe's message replying to me and the
> ding list didn't appear in the list again.  Here it is:

One more is here. :)


[-- Attachment #2: Type: message/rfc822, Size: 10289 bytes --]

[-- Attachment #2.1.1: Type: text/plain, Size: 1602 bytes --]


Hi! again, 

 Ar an séiú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

 > [...] I tried your patch with the old mm-util.el and confirmed it works.
 > However, it doesn't work with non-Latin characters:
 > 
 > (let ((mm-coding-system-priorities '(shift_jis)))
 >   (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))

I was wrong, I can work around this. Since latin-unity knows the list of
coding systems it can map into, we can check each entry in
mm-coding-system-priorities for validity before trying to remap with it. So,
for example, if shift_jis is in mm-coding-system-priorities, the latin unity
code should just give up, and mm-find-mime-charset falls back to the code
you had written.

I~ve a revised patch and test set (including your tests from the last mail)
attached that addresses this. I~m also going to post this to gnu.emacs.gnus
because ding@gnus evidently doesn~t like me :-) .

In terms of usability, I~m starting to feel strongly that
mm-coding-systems-priorities should be initialised to '(iso-8859-1
iso-8859-15 iso-8859-2 iso-8859-16 utf-8) for non-East-Asian
locales. Americans don~t care, but for non-English speaking Europeans the
Mule breakage just another reason not to use an Emacs. 

Best regards, 

        - Aidan
-- 
~As democracy is perfected, the office of president represents, more and
more closely, the inner soul of the people. On some great and glorious day
the plain folks of the land will reach their heart~s desire at last and the
White House will be adorned by a downright moron.~ ~ H.L. Mencken 


[-- Attachment #2.1.2: Further revision of mm-util.el to behave better under XEmacs. --]
[-- Type: text/x-patch, Size: 4116 bytes --]

--- mm-util.el.orig	2004-12-06 23:20:17.000000000 +0000
+++ mm-util.el	2004-12-06 23:54:27.000000000 +0000
@@ -584,6 +584,73 @@
 		(length (memq (coding-system-base b) priorities)))
 	   t))))
 
+(defun mm-xemacs-find-mime-charset (begin end)
+  "Determine which MIME charset to use to send region as message.
+This uses the XEmacs-specific latin-unity package to better handle the case
+where identical characters from diverse ISO-8859-? character sets can be
+encoded using a single one of the corresponding coding systems. 
+
+It treats `mm-coding-system-priorities' as the list of preferred coding
+systems; a useful example setting for this list in Western Europe would be
+'(iso-8859-1 iso-8859-15 utf-8), which would default to the very standard
+Latin 1 coding system, and only move to coding systems that are less
+supported as is necessary to encode the characters that exist in the buffer.
+
+Latin Unity doesn't know about those non-ASCII Roman characters that are
+available in various East Asian character sets.  As such, its behavior if
+you have a JIS 0212 LATIN SMALL LETTER A WITH ACUTE in a buffer and it can
+otherwise be encoded as Latin 1, won't be ideal.  But this is very much a
+corner case, so don't worry about it.  "
+  (let ((systems mm-coding-system-priorities) csets psets curset chars-region)
+
+    ;; Load the Latin Unity library, if available.
+    (when (and (not (featurep 'latin-unity)) (locate-library "latin-unity"))
+      (require 'latin-unity))
+
+    ;; Now, can we use it?
+    (if (featurep 'latin-unity)
+	(progn 
+	  (assert (featurep 'xemacs) 
+		  (concat "We only expect latin-unity on XEmacs--this code "
+			  "will break on the FSF's Emacs.  "))
+	  (setq csets (latin-unity-representations-feasible-region begin end)
+		psets (latin-unity-representations-present-region begin end)
+		chars-region (delq 'ascii (charsets-in-region begin end)))
+
+	  (catch 'done
+
+	    ;; Pass back the first coding system in the preferred list that
+	    ;; can encode the whole region.
+	    (dolist (curset systems)
+	      (setq curset (latin-unity-massage-name curset 'buffer-default))
+
+	      ;; If the coding system is a universal coding system, then it
+	      ;; can certainly encode all the characters in the region. 
+	      (if (memq curset latin-unity-ucs-list)
+		  (throw 'done (list curset)))
+
+	      ;; If a coding system isn't universal, and isn't in the list
+	      ;; that latin unity knows about, we can't decide whether to
+	      ;; use it here. Leave that until later in mm-find-mime-charset
+	      ;; region function, whence we have been called.
+	      (unless (memq curset latin-unity-coding-systems)
+		(throw 'done nil))
+
+	      ;; Right, we know about this coding system, and it may
+	      ;; conceivably be able to encode all the characters in the
+	      ;; region.
+	      (if (latin-unity-maybe-remap begin end curset csets psets t)
+		  (throw 'done (list curset))))
+
+	    ;; Can't encode using anything from the
+	    ;; mm-coding-system-priorities list. Leave mm-find-mime-charset
+	    ;; to do most of the work.
+	    nil))
+
+      ;; Right, latin unity isn't available; let mm-find-charset-region 
+      ;; take its default action, which equally applies to GNU Emacs. 
+      nil)))
+
 (defun mm-find-mime-charset-region (b e &optional hack-charsets)
   "Return the MIME charsets needed to encode the region between B and E.
 nil means ASCII, a single-element list represents an appropriate MIME
@@ -625,8 +692,12 @@
 			 (setq systems nil
 			       charsets (list cs))))))
 	       charsets))
-	;; Otherwise we're not multibyte, we're XEmacs, or a single
-	;; coding system won't cover it.
+        ;; If we're XEmacs, and some coding system is appropriate,
+        ;; mm-xemacs-find-mime-charset will return an appropriate list.
+        ;; Otherwise, we'll get nil, and the next setq will get invoked.
+        (setq charsets (mm-xemacs-find-mime-charset b e))
+
+        ;; We're not multibyte, or a single coding system won't cover it.
 	(setq charsets
 	      (mm-delete-duplicates
 	       (mapcar 'mm-mime-charset

[-- Attachment #2.1.3: Test Lisp for the mm-util.el revision. --]
[-- Type: application/emacs-lisp, Size: 3397 bytes --]

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 396 bytes --]

>>>>> In <16816.39575.977351.530618.dm0@vm.parhasard.net> Aidan Kehoe wrote:

> Reiner also tells me that I need to send a copyright assignment to the FSF
> to get this added to mainline Gnus; where do I get the forms for this? Can I
> download PDFs and print them out?

The attached form is what I received from FSF in September.  It
may be better to inquire to <copyright-clerk (at) fsf.org>.


[-- Attachment #2: Type: text/plain, Size: 823 bytes --]

Please email the following information to fsf-records@gnu.org, and we
will send you the assignment form for your past and future changes.
Please use your full name as the subject line of the message.


[What is the name of the program or package you're contributing to?]


[Did you copy any files or text written by someone else in these changes?
Even if that material is free software, we need to know about it.]


[Do you have an employer who might have a basis to claim to own
your changes?  Do you attend a school which might make such a claim?]


[For the copyright registration, what country are you a citizen of?]


[What year were you born?]


[Please write your email address here.]


[Please write your snail address here.]





[Which files have you changed so far, and which new files have you written
so far?]

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

>>>>> In <16820.62708.663703.64580.z25zdq@parhasard.net> Aidan Kehoe wrote:

>  Ar an séiú lá de mí na Nollaig, scríobh Katsumi Yamaoka:

>> [...] I tried your patch with the old mm-util.el and confirmed it works.
>> However, it doesn't work with non-Latin characters:
>> 
>> (let ((mm-coding-system-priorities '(shift_jis)))
>>   (rfc2047-encode-string (string (make-char 'japanese-jisx0208 38 66))))

> I was wrong, I can work around this. Since latin-unity knows the list of
> coding systems it can map into, we can check each entry in
> mm-coding-system-priorities for validity before trying to remap with it. So,
> for example, if shift_jis is in mm-coding-system-priorities, the latin unity
> code should just give up, and mm-find-mime-charset falls back to the code
> you had written.

That's good.  I also vaguely thought it might be possible.

> I've a revised patch and test set (including your tests from the last mail)
> attached that addresses this. I'm also going to post this to gnu.emacs.gnus
> because ding@gnus evidently doesn't like me :-) .

Too bad.  Maybe the administrator is taking a winter vacation?
Anyway, thank you for the new patch.  I wish we could merge it
by the time Gnus v5.10.7 is released.

> In terms of usability, I'm starting to feel strongly that
> mm-coding-systems-priorities should be initialised to '(iso-8859-1
> iso-8859-15 iso-8859-2 iso-8859-16 utf-8) for non-East-Asian
> locales. Americans don't care, but for non-English speaking Europeans the
> Mule breakage just another reason not to use an Emacs. 

Currently the default value for `mm-coding-systems-priorities'
is non-nil only in the Japanese language environments, which is
done by examining the value for `current-language-environment'.
Is it able to set `(iso-8859-1 iso-8859-15 ...)' by the same
way?  If so, do you know those language names?

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

>>>>> In <b9yy8ga7wc7.fsf@jpl.org> Katsumi Yamaoka wrote:

> Anyway, thank you for the new patch.  I wish we could merge it
> by the time Gnus v5.10.7 is released.

I'm ready to install your patch in the Gnus trunk and the v5-10
branch.  My patch is available at:

http://www.jpl.org/mm-patch-20041207.txt

I hope it is tested by many XEmacs users in the Latin worlds.
There I made slightly modifications:

1. Added some dummy definitions to suppress compile warnings.
2. Provided a small macro used to reduce a funcall overhead in
   Emacs.
3. Corrected the order of arguments specified to the
   latin-unity-massage-name function.
4. Removed the chars-region variable which is unused.

If there's no comment, I will commit it when you drop a signed
paper into a mailbox.

(For the change to v5-10 branch, I think it can be considered to
be a bug fix since Gnus didn't have an ability to unify Latin-9
into Latin-1 in XEmacs.)

P.S.
I found another problem, although it is not related to this
subject.  The following form causes an error in XEmacs, but
Emacs works.

(rfc2047-encode-string (string
			(make-char 'latin-iso8859-1 95)
			(make-char 'japanese-jisx0208 38 66)))

The occasion using mixture text of Latin and Japanese is few,
though.

Re: XEmacs, Gnus and mm-coding-system priorities.

[Aidan Kehoe]

 Ar an seachtú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

 > If there's no comment, I will commit it when you drop a signed
 > paper into a mailbox.

That’s done. It may take a few days more than normal to get to
Massachusetts, though, because our postal service isn’t up to much. 

-- 
“Ah come on now Ted, a Volkswagen with a mind of its own, driving all over
the place and going mad, if that’s not scary I don’t know what is.”

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

>>>>> In <16834.53068.586526.338568.dm0@vm.parhasard.net> Aidan Kehoe wrote:

>  Ar an seachtú lá de mí na Nollaig, scríobh Katsumi Yamaoka: 

>> If there's no comment, I will commit it when you drop a signed
>> paper into a mailbox.

> That’s done. It may take a few days more than normal to get to
> Massachusetts, though, because our postal service isn’t up to much. 

Thanks.  I'll do it ASAP (tomorrow in Japan, maybe).

Re: XEmacs, Gnus and mm-coding-system priorities.

[Katsumi Yamaoka]

>>>>>> In <16834.53068.586526.338568.dm0@vm.parhasard.net> Aidan Kehoe wrote:

>> That’s done. It may take a few days more than normal to get to
>> Massachusetts, though, because our postal service isn’t up to much. 

>>>>> In <b9yoegtnlck.fsf@jpl.org> 
>>>>>	Katsumi Yamaoka <yamaoka@jpl.org> wrote:

> Thanks.  I'll do it ASAP (tomorrow in Japan, maybe).

This is tomorrow of the day. :)
I've installed Aidan Kehoe's change in both the trunk and
the v5-10 branch.

XEmacs hangs in rfc2047-encode-string (Was: XEmacs, Gnus and mm-coding-system priorities.)

[Steinar Bang]

>>>>> Katsumi Yamaoka <yamaoka@jpl.org>:

> P.S.
> I found another problem, although it is not related to this
> subject.  The following form causes an error in XEmacs, but
> Emacs works.

> (rfc2047-encode-string (string
> 			(make-char 'latin-iso8859-1 95)
> 			(make-char 'japanese-jisx0208 38 66)))

> The occasion using mixture text of Latin and Japanese is few,
> though.

After "apt-get dist-upgrade" upgraded XEmacs on my debian system
yesterday, Gnus in XEmacs started hanging when fetching headers.
XEmacs is completely dead, and doesn't respond to `C-g'.  

When I kill the XEmacs process, I get a backtrace in the terminal
window I started XEmacs from. 

All backtraces has shown it being stuck in rfc2047-parse-and-decode,
but the actual headers being processed have been different, so I think
it's just using a lot of time, rather than being completely stuck.

When this happened after the upgrade yesterday, it happened on
ISO-8859-1 decoded headers.  I did a CVS update, and byte compile on
gnus, and that seemed to fix it.  But today it happened again.

One difference since yesterday, is that I have removed mule-ucs, since
it is incompatible with the nXML XML mode.

The debian package version number is
	xemacs21 21.4.16-1

The backtrace from killing a stuck XEmacs process follows:
Lisp backtrace follows:

  string-match("=\\?\\([^][- ()<>@,;:*\\\"/?.=]+\\)\\(?:\\*[^?]+\\)?\\?\\(B\\|Q\\)\\?\\([!->@-~ ]*\\)\\?=" "=?windows-1251?B?0ODn8ODh7vLq8yDw5err4Ozt+/Ug6CBQUiDq7u325e/26OkgLy8g4e7p?=")
  # bind (word)
  rfc2047-parse-and-decode("=?windows-1251?B?0ODn8ODh7vLq8yDw5err4Ozt+/Ug6CBQUiDq7u325e/26OkgLy8g4e7p?=")
  # (unwind-protect ...)
  # (unwind-protect ...)
  # bind (case-fold-search b e start end)
  # (unwind-protect ...)
  # (unwind-protect ...)
  # bind (temp-buffer m string)
  mail-decode-encoded-word-string("=?windows-1251?B?0ODn8ODh7vLq8yDw5err4Ozt+/Ug6CBQUiDq7u325e/26OkgLy8g4e7p?= =?windows-1251?B?6ugg6CDi7vDu4uDy+yDoIPfg8fLuIPP17uTo6+gg8yDs5e3/IOjnIOPr?= =?windows-1251?B?4Ocg4iA=?=")
  byte-code("..." [x gnus-decode-encoded-word-function string eol search-forward "      " move mm-subst-char-in-string ?\r ?- ?\n t] 8)
  # (condition-case ... . ((error x)))
  # bind (x)
  # (unwind-protect ...)
  # bind (eol buffer header references in-reply-to number dependencies force-new)
  byte-code("..." [id number eol id-dep references header read point-at-eol nil ((widen)) (byte-code " `ÄÅ
  Æ#«¤`Sª¡
          {©!?ÇÈÉÇÊÉ\nË$Ë$)§" [x gnus-decode-encoded-word-function string eol search-forward "  " move mm-subst-char-in-string ?\r ?- ?\n t] 8) ((error x)) (byte-code "        `ÄÅ
                                                                           Æ#«¤`Sª¡
                                                                                   {©!?ÇÈÉÇÊÉ\nË$Ë$)§" [x gnus-decode-encoded-word-function string eol search-forward " " move mm-subst-char-in-string ?\r ?- ?\n t] 8) ((error x)) search-forward "    " move string-match "^<[^>]+>$" format "fake+none+%s+%d" "fake+none+%s+%s" int-to-string ?\t 0 (byte-code "Àp!§" [read] 2) ((error)) 1 (byte-code "Àp!§" [read] 2) ((error)) looking-at "Xref: " (?\n nil) "^\\([^ :]+\\): " intern match-string vector "" 9 In-Reply-To 5 gnus-extract-message-id-from-in-reply-to 4 boundp t nnmail-message-id 8 zerop gnus-split-references "\\(<[^<]+>\\)[  ]*\\'" intern-soft "none" ids sequence ref-dep dependencies ref n force-new in-reply-to string num out parent-id allp ref-header replaced gnus-newsgroup-name nnheader-fake-message-id extra xref lines chars date from subject gnus-alter-header-function headers cur buffer x gnus-summary-ignore-duplicates] 16)
  # (condition-case ... . ((error (byte-code "ÀÁÂÃe`\"#¨Äy§" ... 6))))
  # (unwind-protect ...)
  # bind (mail-parse-charset mail-parse-ignored-charsets cur dependencies allp number headers header also-fetch-heads group dependencies force-new sequence)
  gnus-get-newsgroup-headers-xover((4772 4857 4862 4864 5430 5434 5436 5438 5443 5445 5460 5461 5473 5475 5480 5482 5494 5507 5508 5516 5517 5524 5529 5530 5531 5532 5534 5548 5549 5550 5555 5564 5795 5798 5800 5801 5809 6309 6310 6371 6382 6383 6923 6925 6934 6937 6943 6979 6984 6985 6986 6990 7004 7007 7009 7012 7015 7020 7022 7023 7085 7108 7110 7112 7130 7131 7133 7134 7135 7140 7173 7174 7175 7178 7179 7180 7182 7185 7192 7193 7213 7214 7220 7331 7691 7736 7750 7756 7759 7762 7763 7774 7787 7796 7800 7805 7816 7843 7844 7846 7872 7897 7900 7906 7917 7919 7927 7940 7957 7991 7992 8001 8018 8020 8029 8051 8057 8063 8065 8082 8083 8085 8092 8093 8094 8105 8109 8111 8112 8115 8116 8144 8146 8147 8148 8161 8193 8239 8259 8260 8292 8332 8362 8365 8366 8367 8368 8369 8373 8390 8404 8409 8417 8422 8426 8429 8432 8436 8440 8451 8458 8459 8463 8474 8479 8480 8487 8490 8491 8495 8500 8505 8514 8526 8531 8536 8597 8599 8600 8601 8608 8617 8628 8638 8643 8644 8662 8665 8705 8732 8733 8742 8749 8758 8771 8772 8773 8786 8792 8794 8795 8796 8798 8800 8802 8814 8816 8817 8821 8822 8824 8825 8829 8833 8844 8849 8870 8871 8874 8876 8877 8909 8910 8912 8916 8918 8923 8927 8931 8935 8936 8945 8953 8955 8976 8978 8981 8986 9002 9012 9019 9028 9029 9032 9038 9046 9049 9069 9070 9072 9074 9076 9079 9099 9105 9124 9135 9150 9164 9238 9333 9360 9361 9551 9560 9588 9593 9621 9625 9645 9646 10038 10076 10078 10092 10132 10170 10176 10219 10228 10237 10244 10247 10273 10941 10947 10960 10975 10985 10995 10996 11001 11002 11003 11004 11012 11017 11018 11023 11024 11026 11027 11029 11030 11042 11053 11056 11058 11059 11061 11062 11066 11068 11069 11070 11071 11072 11076 11082 11085 11089 11090 11101 11206 11207 11208 11219 11228 11229 11243 11247 11248 11259 11269 11269 11290 11292 11294 11295 11296 11297 11325 11330 11331 11334 11335 11336 11337 11338 11339 11340 11341 11342 11343 11344 11345 11346 11347 11348 11349 11350 11351 11352 11353 11354 11355 11356 11357 11358 11359 11360 11361 11362 11363 11364 11365 11366 11367 11368 11369 11370 11371 11372 11373 11374 11375 11376 11377 11378 11379 11380 11381 11382 11383 11384 11385 11386 11387 11388 11389 11390 11391 11392 11393 11394 11395 11396 11397 11398 11399 11400 11401 11402 11403 11404 11405 11406 11407 11408 11409 11410 11411 11412 11413 11414 11415 11416 11417 11418 11419 11420 11421 11422 11423 11424 11425 11426 11427 11428 11429 11430 11431 11432 11433 11434 11435 11436 11437 11438 11439 11440 11441 11442 11443 11444 11445 11446 11447 11448 11449 11450 11451 11452 11453 11454 11455 11456 11457 11458 11459 11460 11461 11462 11463 11464 11465 11466 11467 11468 11469 11470 11471 11472 11473 11474 11475) nil nil "nntp+news.gmane.org:gmane.linux.acpi.devel")
  # bind (marked-articles gnus-command-method gnus-newsgroup-name gnus-newsgroup-dependencies gnus-newsgroup-headers gnus-newsgroup-scored gnus-use-cache gnus-summary-expunge-below gnus-summary-mark-below gnus-orphan-score gnus-headers gnus-score articles arts category predicate info marks score-param method group)
  gnus-agent-fetch-group-1("nntp+news.gmane.org:gmane.linux.acpi.devel" (nntp "news.gmane.org"))
  # (condition-case ... . ((error (byte-code "  ÂÃ!\"!¬¤ÅÆ!¨Â§" ... 5)) (quit (byte-code "Ã\n!¨ ÄÅ!\"!¬¥ÇÈÉ\"¨Ã§" ... 5))))
  # bind (gnus-agent-fetching)
  # (unwind-protect ...)
  # (unwind-protect ...)
  # bind (methods groups group gnus-command-method)
  #<compiled-function nil "...(114)" [gnus-plugged gnus-agent-covered-methods groups group methods gnus-command-method error "No servers are covered by the Gnus agent" "Can't fetch articles while Gnus is unplugged" nil gnus-server-opened gnus-open-server gnus-online gnus-groups-from-server (...) t gnus-agent-start-fetch gnus-group-level gnus-agent-fetch-group-1 err (gnus-agent-fetch-group-1 group gnus-command-method) (... ...) gnus-run-hooks gnus-agent-fetched-hook gnus-message 6 "Finished fetching articles into the Gnus agent" gnus-agent-fetching gnus-agent-handle-level debug-on-error debug-on-quit] 5 ("/home/sb/cvs/gnus.xemacs/lisp/gnus-agent.elc" . 64578) nil>()
  call-interactively(gnus-agent-fetch-session)
  # (condition-case ... . error)
  # (catch top-level ...)

Re: XEmacs hangs in rfc2047-encode-string

[Reiner Steib]

On Fri, Dec 31 2004, Steinar Bang wrote:

> All backtraces has shown it being stuck in rfc2047-parse-and-decode,
> but the actual headers being processed have been different, so I think
> it's just using a lot of time, rather than being completely stuck.
>
> When this happened after the upgrade yesterday, it happened on
> ISO-8859-1 decoded headers.  I did a CVS update, and byte compile on
> gnus, and that seemed to fix it.  But today it happened again.
[...]
> The debian package version number is
> 	xemacs21 21.4.16-1
[...]
> The backtrace from killing a stuck XEmacs process follows:
> Lisp backtrace follows:
>
>   string-match("=\\?\\([^][- ()<>@,;:*\\\"/?.=]+\\)\\(?:\\*[^?]+\\)?\\?\\(B\\|Q\\)\\?\\([!->@-~ ]*\\)\\?=" "=?windows-1251?B?0ODn8ODh7vLq8yDw5err4Ozt+/Ug6CBQUiDq7u325e/26OkgLy8g4e7p?=")
>   # bind (word)
>   rfc2047-parse-and-decode("=?windows-1251?B?0ODn8ODh7vLq8yDw5err4Ozt+/Ug6CBQUiDq7u325e/26OkgLy8g4e7p?=")
[...]

We got several reports like this one from people using XEmacs 21.4.16.
I guess it's a bug in XEmacs' RegExp handling.  Please make a full
XEmacs bug report and send it to the XEmacs developers.

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: XEmacs hangs in rfc2047-encode-string

[Steinar Bang]

>>>>> Steinar Bang <sb@dod.no>:

> After "apt-get dist-upgrade" upgraded XEmacs on my debian system
> yesterday, Gnus in XEmacs started hanging when fetching headers.
> XEmacs is completely dead, and doesn't respond to `C-g'.  
[snip!]
> One difference since yesterday, is that I have removed mule-ucs, since
> it is incompatible with the nXML XML mode.

I reinstalled mule-ucs, and the problem persists.

Re: XEmacs hangs in rfc2047-encode-string

[Arnaud Giersch]

Vendredi le 31 décembre 2004, vers 10:01:40 (CET), Steinar Bang a
écrit:

> After "apt-get dist-upgrade" upgraded XEmacs on my debian system
> yesterday, Gnus in XEmacs started hanging when fetching headers.
> XEmacs is completely dead, and doesn't respond to `C-g'.  

I've got the same problem here.  A workaround (if you can afford it)
is to use xemacs21-nomule instead of xemacs21-mule.

-- 
Arnaud

Re: XEmacs hangs in rfc2047-encode-string

[Steinar Bang]

>>>>> Reiner Steib <reinersteib+gmane@imap.cc>:

> We got several reports like this one from people using XEmacs 21.4.16.
> I guess it's a bug in XEmacs' RegExp handling.  Please make a full
> XEmacs bug report and send it to the XEmacs developers.

Ok, I've done that.

But it would be nice if a workaround could be found in Gnus.  Even if
it was fixed today, it'll be a while until a new xemacs works its way
into debian testing.

Re: XEmacs hangs in rfc2047-encode-string

[Steinar Bang]

>>>>> Arnaud Giersch <arnaud.giersch@free.fr>:

> I've got the same problem here.  A workaround (if you can afford it)
> is to use xemacs21-nomule instead of xemacs21-mule.

My current workaround is to use GNU Emacs for Gnus (ie. the debian
package "emacs21"). 

That works, as well.

Re: XEmacs hangs in rfc2047-encode-string

[Andrey Slusar]

Fri, 31 Dec 2004 15:41:48 +0100, Steinar Bang wrote:

> > We got several reports like this one from people using XEmacs 21.4.16.
> > I guess it's a bug in XEmacs' RegExp handling.  Please make a full
> > XEmacs bug report and send it to the XEmacs developers.

> Ok, I've done that.

> But it would be nice if a workaround could be found in Gnus.  Even if
> it was fixed today, it'll be a while until a new xemacs works its way
> into debian testing.

  Apply the patch and recompile xemacs:
--8<---------------cut here---------------start------------->8---
Index: regex.c
===================================================================
RCS file: /pack/xemacscvs/XEmacs/xemacs/src/regex.c,v
retrieving revision 1.25.2.10
retrieving revision 1.25.2.9
diff -u -r1.25.2.10 -r1.25.2.9
--- src/regex.c	2004/10/08 00:40:33	1.25.2.10
+++ src/regex.c	2004/09/23 03:03:43	1.25.2.9
@@ -4766,20 +4766,16 @@
 
 		/* Go through the first `min (num_regs, regs->num_regs)'
 		   registers, since that is all we initialized.  */
-		for (mcnt = 1;
-		     mcnt < MIN (num_nonshy_regs, regs->num_regs);
-		     mcnt++)
+		for (mcnt = 1; mcnt < MIN (num_regs, regs->num_regs); mcnt++)
 		  {
-		    int ireg = bufp->external_to_internal_register[mcnt];
-
-		    if (REG_UNSET (regstart[ireg]) || REG_UNSET (regend[ireg]))
+		    if (REG_UNSET (regstart[mcnt]) || REG_UNSET (regend[mcnt]))
 		      regs->start[mcnt] = regs->end[mcnt] = -1;
 		    else
 		      {
 			regs->start[mcnt]
-			  = (regoff_t) POINTER_TO_OFFSET (regstart[ireg]);
+			  = (regoff_t) POINTER_TO_OFFSET (regstart[mcnt]);
 			regs->end[mcnt]
-			  = (regoff_t) POINTER_TO_OFFSET (regend[ireg]);
+			  = (regoff_t) POINTER_TO_OFFSET (regend[mcnt]);
 		      }
 		  }
 	      } /* regs && !bufp->no_sub */
--8<---------------cut here---------------end--------------->8---

 This is fixed problems. I am added this patch to official freebsd
xemacs21-mule port.

-- 
Regards,
Andrey.

Re: XEmacs hangs in rfc2047-encode-string

[Steinar Bang]

>>>>> Andrey Slusar <anrays@gmail.com>:

>   Apply the patch and recompile xemacs:
[snip!]
>  This is fixed problems. I am added this patch to official freebsd
> xemacs21-mule port.

Do you mean fixed in XEmacs CVS?  Do you have any URLs to the original
XEmacs bug report?  I plan to report this as a debian bug on the
xemacs package.

Thanx!

Re: XEmacs hangs in rfc2047-encode-string

[Andrey Slusar]

Sat, 01 Jan 2005 10:04:57 +0100, Steinar Bang wrote:

> >   Apply the patch and recompile xemacs:
> [snip!]
> >  This is fixed problems. I am added this patch to official freebsd
> > xemacs21-mule port.

> Do you mean fixed in XEmacs CVS?  Do you have any URLs to the original
> XEmacs bug report?  I plan to report this as a debian bug on the
> xemacs package.

  Original report is http://thread.gmane.org/gmane.emacs.xemacs.beta/17437
Not answered :(

-- 
Regards,
Andrey.

Re: XEmacs hangs in rfc2047-encode-string

[Steinar Bang]

>>>>> Andrey Slusar <anrays@gmail.com>:

>   Original report is http://thread.gmane.org/gmane.emacs.xemacs.beta/17437

From looking at the thread, what's needed is a minimal test case,
that'll provoke the loop.

I'm wondering if calling rfc2047-encode-string with a string
containing a problem header would be enough?

Re: XEmacs hangs in rfc2047-encode-string

[Andrey Slusar]

Sat, 01 Jan 2005 20:25:45 +0100, Steinar Bang wrote:

> >   Original report is http://thread.gmane.org/gmane.emacs.xemacs.beta/17437

> From looking at the thread, what's needed is a minimal test case,
> that'll provoke the loop.

> I'm wondering if calling rfc2047-encode-string with a string
> containing a problem header would be enough?

  This bug is reproduced not a only in the gnus's rfc2047-encode-string,
this is a global xemacs regexps problem.
  I am reproduced this, when i am using the semantic 1.4.4 too. You are
propose change all the hanged in the xemacs regexps? This is a
unfortunately.

-- 
Regards,
Andrey.

Re: XEmacs hangs in rfc2047-encode-string

[Steinar Bang]

>>>>> Andrey Slusar <anrays@gmail.com>:

>   This bug is reproduced not a only in the gnus's
> rfc2047-encode-string, this is a global xemacs regexps problem.
>   I am reproduced this, when i am using the semantic 1.4.4 too. You
> are propose change all the hanged in the xemacs regexps? This is a
> unfortunately.

Not sure I understand what you're saying here, but what I propose is
to create a minimal testcase that can reproduce the infloop, that the
XEmacs developers can use to step throught the debugger.

It doesn't _have_ to be rfc2047-encode-string or something from
semantic.  Just a regexp that can throw XEmacs into the infloop (or
whatever freezes it).