Other IDN-fixes (was: Hashcash-generation/check for IDN-domains)

["Arne Jørgensen" <arne@arnested.dk>]

[-- Attachment #1: Type: text/plain, Size: 1385 bytes --]

asjo@koldfront.dk (Adam Sjøgren) writes:

> I (sort-of) recently acquired the domain 'sjøgren.org'.
>
> I just tried sending an email to adam@sjøgren.org - Gnus nicely asked
> me if it should translate sjøgren.org to xn--sjgren-cya.org, and I
> happily accepted.

Your posting just reminded me that I've had and used some fixes for
the IDN-handling for some weeks now (since i bought jøssen.dk).

The patch doesn't solve your problem as far as I can tell.

It fixes the following things though:

1. Decoding articles with more than one IDN-address in the same field
   (i.e. "To: adam@sjøgren.dk, arne@jøssen.dk")

   Currently only the last address is decoded, but the patch fixes a
   regexp in `article-decode-idna-rhs' so all are handled.

2. Also encode and decode address in Reply-To:, Mail-Reply-To: and
   Mail-Followup-To:.

3. When `message-use-idna is 'ask you will be asked whether to replace
   the same idna domain once for every time it is in a header (i.e.
   twice about jøssen.dk for "To: arne@jøssen.dk, palle@jøssen.dk")
   but the replacement is done the first time for both/all.

   The patch only asks once for the same idna domain in each header,
   but will still ask twice about jøssen.dk if it is present in both
   say To: and Reply-To: (that would require a bigger rewrite of the
   idna handling).

Kind regards,
-- 
Arne Jørgensen <http://arnested.dk/>


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: idna.patch --]
[-- Type: text/x-patch, Size: 4647 bytes --]

Index: lisp/ChangeLog
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/ChangeLog,v
retrieving revision 7.819
diff -u -p -r7.819 ChangeLog
--- lisp/ChangeLog	27 Sep 2005 02:16:37 -0000	7.819
+++ lisp/ChangeLog	27 Sep 2005 14:25:06 -0000
@@ -1,3 +1,18 @@
+2005-09-27  Arne J^[,Ax^[(Brgensen  <arne@arnested.dk>
+
+	* message.el (message-remove-duplicates): New function.
+	Implementation borrowed from `gnus-remove-duplicates'.
+	(message-idna-to-ascii-rhs): Also encode idna addresses in
+	Reply-To:, Mail-Reply-To: and Mail-Followup-To:.
+	(message-idna-to-ascii-rhs-1): When `message-use-idna' is 'ask
+	only ask about the same idna domain once per header and also tell
+	in what header to replace the idna domain.
+
+	* gnus-art.el (article-decode-idna-rhs): Also decode idna
+	addresses in Reply-To:, Mail-Reply-To: and Mail-Followup-To:.
+	(article-decode-idna-rhs): Fix regexp so that all idna-address in
+	a header is decoded and not just the last one.
+
 2005-09-27  Katsumi Yamaoka  <yamaoka@jpl.org>
 
 	* gnus-art.el (gnus-mime-display-single): Don't modify text if it
Index: lisp/gnus-art.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/gnus-art.el,v
retrieving revision 7.127
diff -u -p -r7.127 gnus-art.el
--- lisp/gnus-art.el	27 Sep 2005 02:16:36 -0000	7.127
+++ lisp/gnus-art.el	27 Sep 2005 14:25:11 -0000
@@ -2385,20 +2385,22 @@ If PROMPT (the prefix), prompt for a cod
 (autoload 'idna-to-unicode "idna")
 
 (defun article-decode-idna-rhs ()
-  "Decode IDNA strings in RHS in From:, To: and Cc: headers in current buffer."
+  "Decode IDNA strings in RHS in various headers in current buffer.
+The following headers are decoded: From:, To:, Cc:, Reply-To:,
+Mail-Reply-To: and Mail-Followup-To:."
   (when gnus-use-idna
     (save-restriction
       (let ((inhibit-point-motion-hooks t)
 	    (inhibit-read-only t))
 	(article-narrow-to-head)
 	(goto-char (point-min))
-	(while (re-search-forward "@.*\\(xn--[-A-Za-z0-9.]*\\)[ \t\n\r,>]" nil t)
+	(while (re-search-forward "@[^ \t\n\r,>]*\\(xn--[-A-Za-z0-9.]*\\)[ \t\n\r,>]" nil t)
 	  (let (ace unicode)
 	    (when (save-match-data
 		    (and (setq ace (match-string 1))
 			 (save-excursion
 			   (and (re-search-backward "^[^ \t]" nil t)
-				(looking-at "From\\|To\\|Cc")))
+				(looking-at "From\\|To\\|Cc\\|Reply-To\\|Mail-Reply-To\\|Mail-Followup-To")))
 			 (setq unicode (idna-to-unicode ace))))
 	      (unless (string= ace unicode)
 		(replace-match unicode nil nil nil 1)))))))))
Index: lisp/message.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/message.el,v
retrieving revision 7.95
diff -u -p -r7.95 message.el
--- lisp/message.el	25 Sep 2005 21:27:48 -0000	7.95
+++ lisp/message.el	27 Sep 2005 14:25:16 -0000
@@ -2088,6 +2088,14 @@ With prefix-argument just set Follow-Up,
 
 ;;; End of functions adopted from `message-utils.el'.
 
+(defun message-remove-duplicates (list)
+  (let (new)
+    (while list
+      (or (member (car list) new)
+	  (setq new (cons (car list) new)))
+      (setq list (cdr list)))
+    (nreverse new)))
+
 (defun message-remove-header (header &optional is-regexp first reverse)
   "Remove HEADER in the narrowed buffer.
 If IS-REGEXP, HEADER is a regular expression.
@@ -5027,13 +5035,15 @@ subscribed address (and not the addition
   (let ((field (message-fetch-field header))
 	rhs ace  address)
     (when field
-      (dolist (address (mail-header-parse-addresses field))
-	(setq address (car address)
-	      rhs (downcase (or (cadr (split-string address "@")) ""))
-	      ace (downcase (idna-to-ascii rhs)))
+      (dolist (rhs 
+	       (message-remove-duplicates
+		(mapcar (lambda (rhs) (or (cadr (split-string rhs "@")) ""))
+			(mapcar 'downcase
+				(mapcar 'car (mail-header-parse-addresses field))))))
+	(setq ace (downcase (idna-to-ascii rhs)))
 	(when (and (not (equal rhs ace))
 		   (or (not (eq message-use-idna 'ask))
-		       (y-or-n-p (format "Replace %s with %s? " rhs ace))))
+		       (y-or-n-p (format "Replace %s with %s in %s:? " rhs ace header))))
 	  (goto-char (point-min))
 	  (while (re-search-forward (concat "^" header ":") nil t)
 	    (message-narrow-to-field)
@@ -5053,6 +5063,8 @@ See `message-idna-encode'."
 	(message-idna-to-ascii-rhs-1 "From")
 	(message-idna-to-ascii-rhs-1 "To")
 	(message-idna-to-ascii-rhs-1 "Reply-To")
+	(message-idna-to-ascii-rhs-1 "Mail-Reply-To")
+	(message-idna-to-ascii-rhs-1 "Mail-Followup-To")
 	(message-idna-to-ascii-rhs-1 "Cc")))))
 
 (defun message-generate-headers (headers)