Editing UTF-8 mail messages

Editing UTF-8 mail messages

[Karl Eichwalder]

[-- Attachment #1: Type: text/plain, Size: 339 bytes --]

Editing UTF-8 encoded mail messages does not seem to work reliably.
I'll append such a message mailed to myself using lynx with "wide"
ncurses support.

The problem: Press e (gnus-summary-edit-article), then C-c C-c.  Saving
back the message Gnus turns it into a base64 encoding and worse, it
does not seem to preserve the UTF-8 charset.


[-- Attachment #2: UTF-8 message with greek characters --]
[-- Type: application/x-gzip, Size: 1931 bytes --]

[-- Attachment #3: Type: text/plain, Size: 273 bytes --]



-- 
ke@suse.de (work) / keichwa@gmx.net (home):              |
http://www.suse.de/~ke/                                  |      ,__o
Free Translation Project:                                |    _-\_<,
http://www.iro.umontreal.ca/contrib/po/HTML/             |   (*)/'(*)

Re: Editing UTF-8 mail messages

[Jesper Harder]

[-- Attachment #1: Type: text/plain, Size: 1113 bytes --]

Karl Eichwalder <keichwa@gmx.net> writes:

> Editing UTF-8 encoded mail messages does not seem to work reliably.
> I'll append such a message mailed to myself using lynx with "wide"
> ncurses support.
>
> The problem: Press e (gnus-summary-edit-article), then C-c C-c.  Saving
> back the message Gnus turns it into a base64 encoding and worse, it
> does not seem to preserve the UTF-8 charset.

You are seeing two different problems:

1. When you do a plain `e' Gnus, mangles the encoding.  This seems to be
   one of the usual MULE-related \201-problems.

   In this case it's caused by `mm-get-part' (in mm-decode.el), which
   uses the macro `mm-with-unibyte-current-buffer-mule4'.  This macro
   does *not* make the buffer unibyte temporarily.

   I don't understand the point of `mm-w-u-c-b-mule4' -- it doesn't
   work.  Why is it ever used?  I suspect that the use of it in several
   places could be causing some of the occasional \201-problems.

   `mm-with-unibyte-current-buffer' *does* work, though.  And if we use
   that macro instead, the problem is fixed (but maybe it breaks older
   Emacsen?).


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: mm-decode.el.diff --]
[-- Type: text/x-patch, Size: 446 bytes --]

--- /home/harder/gnus/lisp/mm-decode.el	Thu Jul 25 21:34:42 2002
+++ mm-decode.el	Sun Oct 13 06:12:36 2002
@@ -953,7 +953,7 @@
   "Return the contents of HANDLE as a string."
   (mm-with-unibyte-buffer
     (insert (with-current-buffer (mm-handle-buffer handle)
-	      (mm-with-unibyte-current-buffer-mule4
+	      (mm-with-unibyte-current-buffer
 		(buffer-string))))
     (mm-decode-content-transfer-encoding
      (mm-handle-encoding handle)

[-- Attachment #3: Type: text/plain, Size: 426 bytes --]


2. If you want to preserve the CTE and charset, you should use `C-u 3 e'.

   Unfortunately, this seems to be broken for text/plain messages.  The
   buffer-local variable `gnus-article-mime-handles' is not set for
   text/plain.  I don't know if it's omitted on purpose -- does someone
   know?.

   If it isn't supposed to be set, then here's a patch that will make
   Gnus construct a MIME handle, when you do `C-u 3 e'.


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #4: gnus-sum.el.diff --]
[-- Type: text/x-patch, Size: 546 bytes --]

--- /home/harder/gnus/lisp/gnus-sum.el	Thu Oct  3 20:25:04 2002
+++ gnus-sum.el	Sun Oct 13 06:09:41 2002
@@ -8938,7 +8938,10 @@
 	    (and (gnus-buffer-live-p gnus-article-buffer)
 		 (with-current-buffer gnus-article-buffer
 		   (prog1
-		       gnus-article-mime-handles
+		       (or gnus-article-mime-handles
+			   (and (gnus-buffer-live-p gnus-original-article-buffer)
+				(with-current-buffer gnus-original-article-buffer
+				  (mm-dissect-buffer t))))
 		     (setq gnus-article-mime-handles nil))))))
      (t
       (setq force t)))

Re: Editing UTF-8 mail messages

[Jesper Harder]

Jesper Harder <harder@ifa.au.dk> writes:

> 1. When you do a plain `e' Gnus, mangles the encoding.  This seems to be
>    one of the usual MULE-related \201-problems.
>
>    In this case it's caused by `mm-get-part' (in mm-decode.el), which
>    uses the macro `mm-with-unibyte-current-buffer-mule4'.  This macro
>    does *not* make the buffer unibyte temporarily.
>
>    I don't understand the point of `mm-w-u-c-b-mule4' -- it doesn't
>    work.  Why is it ever used?  I suspect that the use of it in several
>    places could be causing some of the occasional \201-problems.
>
> -	      (mm-with-unibyte-current-buffer-mule4
> +	      (mm-with-unibyte-current-buffer

Hmm, it might not be the right fix after all.  

It does fix the utf-8 editing problem, and I think also another problem
that someone reported in gnu.emacs.gnus about utf-8 format=flowed
messages being mangled.

*But* it introduces \201's in message/rfc822-forwarding of non-ascii
messages :-(  

Hmm, is the " *mm*"-buffer associated with a MIME-handle supposed to be
unibyte or multibyte?