Message forwarding annoyance and message-forward-show-mml

Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

It seems that the changes to message forwarding in the last months are
going from bad to worse.  We seem to be reliving the early pgnus
nightmare all over again.

So I press `S o m' to forward a relatively complex message.  What do I
get?  A bunch of MML!  No, Virginia, I almost never want Gnus to muck
with the original message -- for one because it's so easy to fuck up
and send garbage.  But even when it doesn't fuck up, I resent the idea
of destructuring a forwarded message to MML only to re-assemble it
again two seconds later without intermediate changes.  In most cases,
I don't want to edit the messages I forward.

Looking for ways to break the legs of "helpful" MML-ification of
forwarded messages, I stumble upon `message-forward-show-mml'.  Great,
said I, setting this to nil will surely make things sensible again.
But what do I get:

    <$part type=message/rfc822 disposition=inline buffer=" *Gnus forward*<3>">
    <$/part>
    (# intentionally transformed to $)

Now I don't even get to *see* the message I'm forwarding.  If it's not
MML, I'm not supposed to see it at all lest I be made into a pillar of
salt by raw MIME.  And I don't even want to know about the "<3>"
following the buffer name.

Could this please be fixed to behave sensibly, specifically so that
pressing `S o m' inserts this:

    <$part type=message/rfc822 disposition=inline>
    ... *raw* message (except for quoting of MML-like keywords ...
    <$/part>

Pretty please?

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

>>>>> "Hrvoje" == Hrvoje Niksic <hniksic@iskon.hr> writes:

[...]

Hrvoje> Looking for ways to break the legs of "helpful" MML-ification of
Hrvoje> forwarded messages, I stumble upon `message-forward-show-mml'.  Great,
Hrvoje> said I, setting this to nil will surely make things sensible again.
Hrvoje> But what do I get:

Hrvoje>     <$part type=message/rfc822 disposition=inline buffer=" *Gnus forward*<3>">
Hrvoje>     <$/part>
Hrvoje>     (# intentionally transformed to $)

Hrvoje> Now I don't even get to *see* the message I'm forwarding.  If it's not
Hrvoje> MML, I'm not supposed to see it at all lest I be made into a pillar of
Hrvoje> salt by raw MIME.  And I don't even want to know about the "<3>"
Hrvoje> following the buffer name.

Hrvoje> Could this please be fixed to behave sensibly, specifically so that
Hrvoje> pressing `S o m' inserts this:

Hrvoje>     <$part type=message/rfc822 disposition=inline>
Hrvoje>     ... *raw* message (except for quoting of MML-like keywords ...
Hrvoje>     <$/part>

Hrvoje> Pretty please?

What is the definition of the *raw* message?  What if the original
message contains non-ASCII characters or other base64- or qp-encoded
parts?  If not decoded, it is still non-readable.  If decoded, it has
to be re-encoded when you send it.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> What is the definition of the *raw* message?

Un-decoded, un-MML-ified, the way it is seen in the mailbox or
delivered by NNTP.

> What if the original message contains non-ASCII characters or other
> base64- or qp-encoded parts?

Leave base64 and qp parts as they are.  Non-ASCII characters should
also be left as they are (their octet values unchanged), making sure
that no additional (en|de)coding happens.

> If not decoded, it is still non-readable.

That's why it's called "raw".

If one wanted to edit complex MIME messages, MML would indeed be
preferable.  I dislike the fact that the very complex and error-prone
disassembling and assembling have been made the default.

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > What is the definition of the *raw* message?
> 
> Un-decoded, un-MML-ified, the way it is seen in the mailbox or
> delivered by NNTP.
> 
> > What if the original message contains non-ASCII characters or other
> > base64- or qp-encoded parts?
> 
> Leave base64 and qp parts as they are.  Non-ASCII characters should
> also be left as they are (their octet values unchanged), making sure
> that no additional (en|de)coding happens.

I recall the problem in forwarding an 8-bit-encoded message.  The
8-bit raw text (esp. koi8-r and utf-8) might not be safe in an
Emacs/MULE multibyte buffer. That's why the original text is put into
an external buffer.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> I recall the problem in forwarding an 8-bit-encoded message.  The
> 8-bit raw text (esp. koi8-r and utf-8) might not be safe in an
> Emacs/MULE multibyte buffer. That's why the original text is put
> into an external buffer.

Not safe in what sense?  Does Mule crash, or what?

Could at least the people who don't suffer Mule (or those who use
XEmacs) be spared the damage?  It's _really_ nice to see the message
you're forwarding.

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > I recall the problem in forwarding an 8-bit-encoded message.  The
> > 8-bit raw text (esp. koi8-r and utf-8) might not be safe in an
> > Emacs/MULE multibyte buffer. That's why the original text is put
> > into an external buffer.
> 
> Not safe in what sense?  Does Mule crash, or what?

Emacs/MULE messes up some 8-bit characters in the range of \200-\237,
because these characters are used for emacs-mule coding-system.

> Could at least the people who don't suffer Mule (or those who use
> XEmacs) be spared the damage?  It's _really_ nice to see the message
> you're forwarding.

Anyway, people can use show-mml to forward 8-bit message.  I will fix
it. See what happens.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> Hrvoje Niksic <hniksic@iskon.hr> writes:
> 
> > Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> > 
> > > I recall the problem in forwarding an 8-bit-encoded message.  The
> > > 8-bit raw text (esp. koi8-r and utf-8) might not be safe in an
> > > Emacs/MULE multibyte buffer. That's why the original text is put
> > > into an external buffer.
> > 
> > Not safe in what sense?  Does Mule crash, or what?
> 
> Emacs/MULE messes up some 8-bit characters in the range of
> \200-\237, because these characters are used for emacs-mule
> coding-system.

How does it mess up?

And, how does that work with regular, non-forwarded messages?  For
instance, what happens in Emacs/MULE when you reply to a message that
contains a literal \220?

> > Could at least the people who don't suffer Mule (or those who use
> > XEmacs) be spared the damage?  It's _really_ nice to see the
> > message you're forwarding.
> 
> Anyway, people can use show-mml to forward 8-bit message.  I will
> fix it. See what happens.

Thanks a bunch!

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > Emacs/MULE messes up some 8-bit characters in the range of
> > \200-\237, because these characters are used for emacs-mule
> > coding-system.
> 
> How does it mess up?

For example, a literal text, \221\345\256, is shown as 瀹 in a
multibyte buffer, because \221 is the escape code for chinese-gb2312,
I guess.

> And, how does that work with regular, non-forwarded messages?  For
> instance, what happens in Emacs/MULE when you reply to a message
> that contains a literal \220?

I am not sure.  A literal \220 could hardly survive when Gnus (using
Emacs/MULE) copies the text between multibyte and unibyte buffers.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> Hrvoje Niksic <hniksic@iskon.hr> writes:
> 
> > Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> > 
> > > Emacs/MULE messes up some 8-bit characters in the range of
> > > \200-\237, because these characters are used for emacs-mule
> > > coding-system.
> > 
> > How does it mess up?
> 
> For example, a literal text, \221\345\256, is shown as 瀹 in a
> multibyte buffer, because \221 is the escape code for
> chinese-gb2312, I guess.

But can't you escape "\221\345\256" to whatever is needed for MULE to
regard them as such?  Doesn't MULE have a "Control-1" escape?

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> But can't you escape "\221\345\256" to whatever is needed for MULE to
> regard them as such?  Doesn't MULE have a "Control-1" escape?

I guess not. That's a difference between Emacs and XEmacs.

s.

Re: Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> Hrvoje Niksic <hniksic@iskon.hr> writes:
> 
> > But can't you escape "\221\345\256" to whatever is needed for MULE to
> > regard them as such?  Doesn't MULE have a "Control-1" escape?
> 
> I guess not.

That sounds incredible.  :-(

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > Hrvoje Niksic <hniksic@iskon.hr> writes:
> > 
> > > But can't you escape "\221\345\256" to whatever is needed for MULE to
> > > regard them as such?  Doesn't MULE have a "Control-1" escape?
> > 
> > I guess not.
> 
> That sounds incredible.  :-(

In a multibyte buffer, the charset of \345 is ascii and \201 cannot
exist before \345.  In a unibyte buffer, the charset of \201 followed
by \345 is latin-iso8859-1, the charset of \201 followed by \201 is
ascii.  At least I can not find a charset like control-1 in XEmacs.
That is why it is difficult to handle mixed text (containing both
control (e.g. \201) and non-ascii (e.g. å) characters in the same
buffer).  To forward message with show-mml being nil, the current
solution is switch the buffer between multibyte and unibyte mode.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Hrvoje Niksic]

Shenghuo ZHU <zsh@cs.rochester.edu> writes:

> > > > But can't you escape "\221\345\256" to whatever is needed for MULE to
> > > > regard them as such?  Doesn't MULE have a "Control-1" escape?
> > > 
> > > I guess not.
> > 
> > That sounds incredible.  :-(
> 
> In a multibyte buffer, the charset of \345 is ascii and \201 cannot
> exist before \345.  In a unibyte buffer, the charset of \201
> followed by \345 is latin-iso8859-1, the charset of \201 followed by
> \201 is ascii.  At least I can not find a charset like control-1 in
> XEmacs.  That is why it is difficult to handle mixed text
> (containing both control (e.g. \201) and non-ascii (e.g. å)
> characters in the same buffer).

Could you present this problem to FSF Emacs developers?  Dave Love for
one used to follow this list.

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
> > > > > But can't you escape "\221\345\256" to whatever is needed for MULE to
> > > > > regard them as such?  Doesn't MULE have a "Control-1" escape?
> > > > 
> > > > I guess not.
> > > 
> > > That sounds incredible.  :-(
> > 
> > In a multibyte buffer, the charset of \345 is ascii and \201 cannot
> > exist before \345.  In a unibyte buffer, the charset of \201
> > followed by \345 is latin-iso8859-1, the charset of \201 followed by
> > \201 is ascii.  At least I can not find a charset like control-1 in
> > XEmacs.  That is why it is difficult to handle mixed text
> > (containing both control (e.g. \201) and non-ascii (e.g. å)
> > characters in the same buffer).
> 
> Could you present this problem to FSF Emacs developers?  Dave Love for
> one used to follow this list.

It has been discussed before. Hope it can be fixed soon.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Dave Love]

>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:

 ZSH> It has been discussed before. Hope it can be fixed soon.

(with-temp-buffer
  (insert ?\221 ?\345 ?\256)
  (find-charset-region (point-min) (point-max)))
  => (eight-bit-control eight-bit-graphic)

OK?

[I have been meaning to check how the existence of the new charsets
affects Gnus.  Someone is welcome to beat me to it.]

Re: Message forwarding annoyance and message-forward-show-mml

[Shenghuo ZHU]

Dave Love <d.love@dl.ac.uk> writes:

> >>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:
> 
>  ZSH> It has been discussed before. Hope it can be fixed soon.
> 
> (with-temp-buffer
>   (insert ?\221 ?\345 ?\256)
>   (find-charset-region (point-min) (point-max)))
>   => (eight-bit-control eight-bit-graphic)
> 
> OK?
> 
> [I have been meaning to check how the existence of the new charsets
> affects Gnus.  Someone is welcome to beat me to it.]

Is it new in Emacs 21?  I got (chinese-gb2312) in Emacs 20.7.

Shenghuo

Re: Message forwarding annoyance and message-forward-show-mml

[Dave Love]

>>>>> "ZSH" == Shenghuo ZHU <zsh@cs.rochester.edu> writes:

 ZSH> Is it new in Emacs 21?  I got (chinese-gb2312) in Emacs 20.7.

Yes.

Re: Message forwarding annoyance and message-forward-show-mml

[Dave Love]

>>>>> "Hrvoje" == Hrvoje Niksic <hniksic@iskon.hr> writes:

 Hrvoje> Shenghuo ZHU <zsh@cs.rochester.edu> writes:
 >> Hrvoje Niksic <hniksic@iskon.hr> writes:
 >> 
 >> > But can't you escape "\221\345\256" to whatever is needed for MULE to
 >> > regard them as such?  Doesn't MULE have a "Control-1" escape?
 >> 
 >> I guess not.

 Hrvoje> That sounds incredible.  :-(

No more so than some things in XEmacs MULE, I think.  I guess this
just wasn't a problem that bit sufficiently for Handa to address it
before -- he's more interested in things like working Latin-2 :-).
MIMEography is rather a special application -- which isn't to say it
shouldn't be done right and rough edges in the framework smoothed off,
of course.  When we install Gnus 5.8, I hope Handa will be able to
take a look and advise on it and there will be more incentive to
address other problems it throws up.

Re: Message forwarding annoyance and message-forward-show-mml

[Vladimir Volovich]

"Hrvoje" == Hrvoje Niksic writes:

 Hrvoje> For instance, what happens in Emacs/MULE when you reply to a
 Hrvoje> message that contains a literal \220?

this octet comes from some MIME charset, so it is first translated to
mule internal encoding (when you are preparing a reply or viewing a
message with that literal \220); when you send a reply, a multibyte
buffer is then translated to some MIME charset.

problems could occur if the file with that message which contains
arbitrary octets in the range 0x80-0x9F is opened in multibyte mode.
(these are really annoying problems for large mbox which happens to
get some message with such literal octets).

	Best regards, -- Vladimir.

Re: Message forwarding annoyance and message-forward-show-mml

[Russ Allbery]

Hrvoje Niksic <hniksic@iskon.hr> writes:

> So I press `S o m' to forward a relatively complex message.  What do I
> get?  A bunch of MML!  No, Virginia, I almost never want Gnus to muck
> with the original message -- for one because it's so easy to fuck up and
> send garbage.  But even when it doesn't fuck up, I resent the idea of
> destructuring a forwarded message to MML only to re-assemble it again
> two seconds later without intermediate changes.  In most cases, I don't
> want to edit the messages I forward.

For what it's worth, I really like seeing the forwarded message in MML and
do edit it somewhat regularly, so if this is changed, an option to keep
the current behavior would be nice.

-- 
Russ Allbery (rra@stanford.edu)             <http://www.eyrie.org/~eagle/>