Re: Reply misinterprets Commas in encoded-words (Gnus violates RFC 2047)

[Katsumi Yamaoka <yamaoka@jpl.org>]

[-- Attachment #1: Type: text/plain, Size: 2226 bytes --]

>>>>> In <v9sm4n6ls5.fsf@marauder.physik.uni-ulm.de> Reiner Steib wrote:

> Hi,

> back in September 2004, Stefan Wiens reported the following bug in
> gnus.gnus-bug but there was no response or fix yet:

> ,----
>| From: =?iso-8859-1?q?Wiens=2C_Stef=AAn?= <s.wi@gmx.net>
>| Subject: Reply misinterprets Commas in encoded-words
>| Newsgroups: gnus.gnus-bug
>| Date: Sat, 18 Sep 2004 03:34:06 +0200
>| Message-ID: <87pt4kbbsh.fsf@xenon.eswe.dyndns.org>
>| 
>| To reproduce, please try to "reply" or "wide reply" to this message
>| and have a look at the "To" and "Cc" headers in your Message buffer.
> `----

> The problem is that Gnus decodes »=?iso-8859-1?q?Wiens=2C_Stef=AAn?=«
> to »Wiens, Stefªn <s.wi@gmx.net>« in the article buffer and when doing
> a (wide) reply, this leads to:

>| To: Wiens
>| Cc: Stefªn <s.wi@gmx.net>,  bugs@gnus.org

> Note that this problem is not artifical (as the »ª« might suggest):
> Many people in Germany or other countries have non-ascii in their
> names.  If the mail software of the sender writes the full name in the
> From/Cc header in the form »Lastname, Fírstname« the user cannot reply
> correctly without manual correction.

> In the German Gnus newsgroup this has been discussed several times:
> (a) http://www.google.de/groups?as_umsgid=uoek8x11f.fsf@M.web.de&hl=en
> (b) http://www.google.de/groups?as_umsgid=87brbdhc4d.fsf@ABG3595C.abg.fsc.net&hl=en

Though I cannot read German articles, I may be able somewhat to
help.

> According to Stefan Wiens and Dirk Nimmich (tin developer), Gnus'
> behavior violates RFC 2047 (see thread (a)).  Could somebody look at
> this problem, please?  Stefan, maybe you could summarize this and the
> other problems with Gnus RFC-2047 en/decoder again if necessary.

> Bye, Reiner.

Well, it seems to be difficult to improve the
message-get-reply-headers function, etc. so that it may parse
correctly any words including commas.  But it is easy to make
Gnus' rfc2047 encoder include the quotes in the encoded words
when the quoted text contains non-ASCII characters and commas.
An example of the encoded word is in the Cc header of this
message.  The patch for the Gnus CVS trunk is here:


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 2006 bytes --]

--- rfc2047.el~	2004-11-23 21:49:55 +0000
+++ rfc2047.el	2005-02-14 07:13:18 +0000
@@ -35,6 +35,7 @@
 
 (require 'qp)
 (require 'mm-util)
+(require 'ietf-drums)
 ;; Fixme: Avoid this (used for mail-parse-charset) mm dependence on gnus.
 (require 'mail-prsvr)
 (require 'base64)
@@ -337,13 +338,23 @@
 		  ;; Does it need encoding?
 		  (goto-char start)
 		  (if (re-search-forward encodable-regexp end 'move)
-		      ;; It needs encoding.  Strip the quotes first,
-		      ;; since encoded words can't occur in quotes.
-		      (progn
-			(goto-char end)
-			(delete-backward-char 1)
-			(goto-char start)
-			(delete-char 1)
+		      ;; It needs encoding.
+		      (let ((tspecials (progn
+					 (goto-char (1+ start))
+					 (re-search-forward
+					  (concat "[" ietf-drums-tspecials "]")
+					  (1- end) t))))
+			(if tspecials
+			    ;; Don't strip the quotes which should be
+			    ;; included in the encoded word, since there
+			    ;; are special characters.
+			    (goto-char start)
+			  ;; Strip the quotes first, since encoded words
+			  ;; can't occur in quotes.
+			  (goto-char end)
+			  (delete-backward-char 1)
+			  (goto-char start)
+			  (delete-char 1))
 			(when last-encoded
 			  ;; There was a preceding quoted word.  We need
 			  ;; to include any separating whitespace in this
@@ -354,7 +365,9 @@
 			  (setq start (point)
 				end (1+ end)))
 			;; Adjust the end position for the deleted quotes.
-			(rfc2047-encode start (- end 2))
+			(rfc2047-encode start (if tspecials
+						  end
+						(- end 2)))
 			(setq last-encoded t)) ; record that it was encoded
 		    (setq last-encoded  nil)))
 		 ((eq ?. csyntax)
@@ -766,8 +779,7 @@
   (let* ((rfc2047-encoding-type 'mime)
 	 (rfc2047-encode-max-chars nil)
 	 (string (rfc2047-encode-string value)))
-    (if (string-match "[][()<>@,;:\\\"/?=]" ;; tspecials
-		      string)
+    (if (string-match (concat "[" ietf-drums-tspecials "]") string)
 	(format "%s=%S" param string)
       (concat param "=" string))))