MML: The Summation

MML: The Summation

[Lars Magne Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 1832 bytes --]


To prematurely sum up the pros and cons of using MML (or something
similar) for Message's MIME interface:

Cons:

1) You can't expect people to learn a markup language.
2) MML is ugly.
3) Inband marking of elements is bound to fail.  Look at the breakage
   done by sendmail's "From " separator.

Pros:

1) I like languages, not applications.
2) MML is cute.
3) Doing MIME is dull, boring and tedious.  By having a simple markup
   language that gives us the power of MIME without the bogosity,
   anyone can write scripts that do MML, which can then just be
   pushed to Message for Mimification.
4) Doing it in a non-textual way (text props, overlays, markers,
   invisible text) sucks because it doesn't allow the users
   to edit the stuff properly.  What if you are composing a complex
   MIME thing in one buffer, and then you decide to copy the text
   to a different buffer?  

That's basically it.  I like the idea of just saying

<#part>
And then writing some Chinese text, and then saying

<#part>
And write some Japanese text, and then saying

<#part type=image/jpeg filename="~/rms.jpg">--==-=-=




to include an image.  If I don't like the order of these paragraphs, I 
can just use the normal Emacs editing tools (C-k, C-y) and move them
wherever I want to -- up, down, to another buffer, or just kill them
altogether.

I think MML gives us the flexibility to compose any MIME message, in a 
very low-impact way.

Of course, one doesn't necessarily have to have just one interface.
For instance, the trivial "include this attachment with this mail" can 
be handled using a different mechanism -- for instance, a header line
`X-Attachment: type=image/jpeg filename="~/rms.jpg"' can be used in
these trivial cases.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: MML: The Summation

[Norman Walsh]

| That's basically it.  I like the idea of just saying
| 
| <#part>
| And then writing some Chinese text, and then saying
| 
| <#part>
| And write some Japanese text, and then saying
| 
| <#part type=image/jpeg filename="~/rms.jpg">--==-=-=
| 
| to include an image.  If I don't like the order of these paragraphs, I 
| can just use the normal Emacs editing tools (C-k, C-y) and move them
| wherever I want to -- up, down, to another buffer, or just kill them
| altogether.

Yes, please!  I understand the desire to hide MIME complexity from
users, but that could be done with another interface, on top of the
MML layer.
                                        Cheers,
                                          norm
-- 
Norman Walsh <ndw@nwalsh.com>      | A new scientific truth does not
http://nwalsh.com/                 | triumph by convincing its
                                   | opponents and making them see the
                                   | light, but rather because its
                                   | opponents eventually die, and a
                                   | new generation grows up that is
                                   | familiar with it (Planck 1949)

Re: MML: The Summation

[Matt Armstrong]

---------------------------------------------------------------------
Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> To prematurely sum up the pros and cons of using MML (or something
> similar) for Message's MIME interface:

I like MML as a foundation.  X-Attachment: <blah blah> can come later.


-- 
matta

Re: MML: The Summation

[Edward J. Sabol]

Excerpts from [ding]: (17-Nov-98) MML: The Summation by Lars Magne Ingebrigtsen
> Cons:
> 1) You can't expect people to learn a markup language.
> 2) MML is ugly.
> 3) Inband marking of elements is bound to fail.  Look at the breakage
> done by sendmail's "From " separator.

I definitely agree with both (1) and (2) here, but I think (3) is the
probably worst "con" of the bunch. It's the one that made me say "ewwwww!"
aloud when I first read the postings about MML.

> Pros:
> 1) I like languages, not applications.

Oh, pshaw. Gnus is an application. (Please, no semantic arguments. I
understand what you're saying.)

> 3) Doing MIME is dull, boring and tedious.  By having a simple markup
> language that gives us the power of MIME without the bogosity,
> anyone can write scripts that do MML, which can then just be
> pushed to Message for Mimification.

Nobody said that a sciptable interface to message MIME-ing was a bad idea. In
fact, I think it's a great idea, and I think MML fulfills this nicely. I'm
just saying it makes a lousy user interface for on-the-fly message-mode
composition.

> 4) Doing it in a non-textual way (text props, overlays, markers,
> invisible text) sucks because it doesn't allow the users
> to edit the stuff properly.  What if you are composing a complex
> MIME thing in one buffer, and then you decide to copy the text
> to a different buffer?  

I definitely agree that the user should be able to edit the buffer properly
and cut, copy, and paste MIME things between message buffers. But surely this
could be implemented? It would probably be difficult to implement, but
impossible? I find that hard to believe. You might have to overload a few
functions probably or add some hooks or both. I suppose that might seem
pretty inelegant. However, your expertise with text properties, overlays,
markers, and invisible text is much greater than mine, so I'll take your word
on it.

> That's basically it.  I like the idea of just saying
>
> <#part>
> And then writing some Chinese text, and then saying
>
> <#part>
> And write some Japanese text, and then saying
>
> <#part type=image/jpeg filename="~/rms.jpg">

I think that with the user interface that I'm dreaming of the user wouldn't
have to enter "<part>" in his text in order to switch from Chinese text to
Japanese text. The very fact that the Chinese text is using a Chinese font
and the Japanese text is in a Japanese font gives enough information for the
MIME user-interface to know that they are two different parts and to have the
appropriate MIME information inserted either after the user hits `C-c C-c' to
send the message or on the fly when the user changes fonts using some
invisible text or other marker. But then there's the question of
multipart/alternative and nested multiparts and how to compose those. That
wouldn't be quite so easy, but I don't think it needs to be and I'm sure
something could be figured out. Like possibly highlighting some number of
adjacent parts with transient-mark-mode on and then hitting some key combo to
signify that the parts are alternative or nested.

> Of course, one doesn't necessarily have to have just one interface.
> For instance, the trivial "include this attachment with this mail" can 
> be handled using a different mechanism -- for instance, a header line
> `X-Attachment: type=image/jpeg filename="~/rms.jpg"' can be used in
> these trivial cases.

To be honest, I don't think I like this "trivial" interface all that much
either, nor is it particularly needed since `C-c C-a filename RET' can just
as easily enter the MML stuff.

Just my two cents worth,
Ed

Re: MML: The Summation

[Matt Armstrong]

"Edward J. Sabol" <sabol@alderaan.gsfc.nasa.gov> writes:

> > That's basically it.  I like the idea of just saying
> >
> > <#part>
> > And then writing some Chinese text, and then saying
> >
> > <#part>
> > And write some Japanese text, and then saying
> >
> > <#part type=image/jpeg filename="~/rms.jpg">
> 
> I think that with the user interface that I'm dreaming of the user
> wouldn't have to enter "<part>" in his text in order to switch from
> Chinese text to Japanese text. The very fact that the Chinese text
> is using a Chinese font and the Japanese text is in a Japanese font
> gives enough information for the MIME user-interface to know that
> they are two different parts and to have the appropriate MIME
> information inserted...<snip>.

Things brings up an interesting point.  Emacs supports just one coding
system per buffer, so how would Lars' example above actually work?



-- 
matta

Re: MML: The Summation

[Lars Magne Ingebrigtsen]

Matt Armstrong <matta@geoworks.com> writes:


> Things brings up an interesting point.  Emacs supports just one coding
> system per buffer, so how would Lars' example above actually work?

No, you can have as many coding systems in each buffer as you feel
like. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: MML: The Summation

[Lars Magne Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 1975 bytes --]

"Edward J. Sabol" <sabol@alderaan.gsfc.nasa.gov> writes:

> I definitely agree that the user should be able to edit the buffer properly
> and cut, copy, and paste MIME things between message buffers. But surely this
> could be implemented? It would probably be difficult to implement, but
> impossible? I find that hard to believe.

This is Emacs; nothing is impossible.

However, Emacs' strength is its brilliant text editing commands --
they work; they work well and they work all the time.  The newer bits
-- text props, overlays, invisible text -- do not work nearly as
well.  Text props have a tendency to bleed if one is not careful; they 
aren't always preserved; they aren't saved; but most importantly of
all (in my opinion), is that the user usually has no way of inspecting 
them.

A text prop based MIME interface (with the required write file hooks
to preserve them) is certainly possible, but...

> I think that with the user interface that I'm dreaming of the user wouldn't
> have to enter "<part>" in his text in order to switch from Chinese text to
> Japanese text. The very fact that the Chinese text is using a Chinese font
> and the Japanese text is in a Japanese font gives enough information for the
> MIME user-interface to know that they are two different parts and to have the
> appropriate MIME information inserted either after the user hits `C-c C-c' to
> send the message or on the fly when the user changes fonts using some
> invisible text or other marker.

Yes.  But if I want to write a Norwegian character and a Japanese
glyph in the same article, Message would have to break that up into a
multipart message, either automatically or by user intervention.  For
instance, Message could go over an article paragraph by paragraph, and 
see whether each paragraph can validly be written in some charset or
other, and if not, it could ask the user for help.  But I'm (as
always) a bit leery of interfaces that try to guess what the user
wants.  

[-- Attachment #2: Type: text/plain, Size: 119 bytes --]


If I want to talk about Morioka (守岡 知彦) and one of my sisters,
that can't be done in the same part...

[-- Attachment #3: Type: text/plain, Size: 43 bytes --]


... but I can mention my sister Åse here.

[-- Attachment #4: Type: text/plain, Size: 827 bytes --]


But perhaps this is a rather esoteric question, and not something that 
should be a primary consideration when designing the MIME user
interface. 

> But then there's the question of multipart/alternative and nested
> multiparts and how to compose those. That wouldn't be quite so easy,
> but I don't think it needs to be and I'm sure something could be
> figured out. Like possibly highlighting some number of adjacent
> parts with transient-mark-mode on and then hitting some key combo to
> signify that the parts are alternative or nested.

I'm against marking parts and then issuing commands.  :-)  I like to
enter commands to make things happen; not futzing around with the
cursor a lot, and then making things happen.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: MML: The Summation

[Graham Murray]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> 
> If I want to talk about Morioka (守岡 知彦) and one of my sisters,
> that can't be done in the same part...

Sorry if this is not actually a gnus question. In the above all I see
between the parenthesis are square boxes rather than the characters. Is
this what I should be seeing, or does this indicate that I have not
setup by emacs or X correctly? C-x = on the first character shows
0xde69, so I suspect that it is a display problem. Should this work on
FSF Emacs or is it only work in XEmacs?

Re: MML: The Summation

[Kai.Grossjohann]

Graham Murray <graham@barnowl.demon.co.uk> writes:


  > Sorry if this is not actually a gnus question. In the above all I see
  > between the parenthesis are square boxes rather than the
  > characters. [...]

You need a font with these characters.  The GNU intlfonts package
contains fonts for them.  (Available from ftp.gnu.org and its mirrors.)

kai
-- 
Life is hard and then you die.

Re: MML: The Summation

[George J McNinch]

>>>>> "Kai" == Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:

    >> Graham Murray <graham@barnowl.demon.co.uk> writes:
    >> Sorry if this is not actually a gnus question. In the above all
    >> I see between the parenthesis are square boxes rather than the
    >> characters. [...]

    Kai> You need a font with these characters.  The GNU intlfonts
    Kai> package contains fonts for them.  (Available from ftp.gnu.org
    Kai> and its mirrors.)

Just so I'm clear on this; if

(featurep 'mule)
=> nil

then I'm out of luck seeing Asian language alphabets? I installed
(most of) the GNU intlfonts, but see only crud in the message in
question.

(emacs-version)
=> XEmacs 20.4 "Emerald" [Lucid] (mips-sgi-irix6.2) [snip]

Best,
George

Re: MML: The Summation

[Kai.Grossjohann]

George J McNinch <McNinch.1@nd.edu> writes:

  > Just so I'm clear on this; if
  > 
  > (featurep 'mule)
  > => nil
  > 
  > then I'm out of luck seeing Asian language alphabets?

Well, I don't know XEmacs, but MULE is for Asian languages, so you
must be right.

Install XEmacs compiled with MULE support.

kai
-- 
Life is hard and then you die.

Re: MML: The Summation

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:


 >> I think that with the user interface that I'm dreaming of the user
 >> wouldn't have to enter "<part>" in his text in order to switch
 >> from Chinese text to Japanese text.

I think that this is a BAD THING (TM) Gnus MUST NOT try to change the
structure of a message if user did not asked this explicitly. For
example, what should happen if one writes a mixed
chinese-japanese-russian-norwegian text? In which each paragraph and
each sentence contains quotes from all those languages (e.g. a
multilingual dictionary). Gnus in no way should try to divide that
message into parts.

All that Gnus should do is: maintain a list of coding system to
charset transformations. And use that list to encode the text from
Emacs buffer to the charset which can fit all symbols used in that
buffer. For example, if i write a mixed russian-japanese text, gnus
should use a japanese encoding in mime part, because the japanese
encoding contains russian characters as well. In short, gnus should
not use any `auto-magic', and should try to find a charset which
contains all characters appearing in a buffer using a translation
list. This is how SEMI does things.

 >> The very fact that the Chinese text is using a Chinese font and
 >> the Japanese text is in a Japanese font gives enough information
 >> for the MIME user-interface to know that they are two different
 >> parts and to have the appropriate MIME information inserted either
 >> after the user hits `C-c C-c' to send the message or on the fly
 >> when the user changes fonts using some invisible text or other
 >> marker.

No, please. It is incorrect and broken broken broken.

	Best regards, -- Vladimir.

Re: MML: The Summation

[Lars Magne Ingebrigtsen]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:


> In short, gnus should not use any `auto-magic', and should try to
> find a charset which contains all characters appearing in a buffer
> using a translation list. This is how SEMI does things.

What does it do when it can't find a charset that covers all
characters?  For instance, I may have written a message that quotes
Sami text (which is iso-8859-9, I believe) and responds with text that 
contains Norwegian text (iso-8859-1).

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: MML: The Summation

[Vladimir Volovich]

"LMI" == Lars Magne Ingebrigtsen writes:


 >> In short, gnus should not use any `auto-magic', and should try to
 >> find a charset which contains all characters appearing in a buffer
 >> using a translation list. This is how SEMI does things.
 LMI> What does it do when it can't find a charset that covers all
 LMI> characters?  For instance, I may have written a message that
 LMI> quotes Sami text (which is iso-8859-9, I believe) and responds
 LMI> with text that contains Norwegian text (iso-8859-1).

AFAIR, if the message contains a really multilingual text :) for which
SEMI could not figure out a single MIME charset which includes all
glyphs, it does not permit to send such a message. [forcing a user
e.g. to break a message into parts] This is imho more safe than
`automagical' splitting the message into MIME parts. At least i'd
prefer gnus to inform me that it is not able to find a proper charset
than not having an idea how the message would be sent. Or at least,
before sending such a message, gnus should show the new parts which
were created, so that the user will have a control on this.

	Best regards, -- Vladimir.

Re: MML: The Summation

[Edward J. Sabol]

Excerpts from mail: (18-Nov-98) Re: MML: The Summation by Vladimir Volovich
> I think that this is a BAD THING (TM) Gnus MUST NOT try to change the
> structure of a message if user did not asked this explicitly. For example,
> what should happen if one writes a mixed chinese-japanese-russian-norwegian
> text? In which each paragraph and each sentence contains quotes from all
> those languages (e.g. a multilingual dictionary). Gnus in no way should try
> to divide that message into parts.

Why not? Seriously, I see nothing wrong with it. Please enlighten me as to
why it is so horribly bad. There's no single charset that includes Chinese,
Japanese, Russian, and Norwegian glyphs, is there? (I'm showing my
international charset ignorance here. My apologies.) Assuming the answer is
no, then how in the world could you possibly send the e-mail and have your
recipient be able to see the same glyphs that you see if you're not going to
make the e-mail multipart? (If the answer was yes, then of course there
should only be one charset. I'm not saying we should use more charsets than
is absolutely necessary.)

> All that Gnus should do is: maintain a list of coding system to charset
> transformations. And use that list to encode the text from Emacs buffer to
> the charset which can fit all symbols used in that buffer. For example, if
> i write a mixed russian-japanese text, gnus should use a japanese encoding
> in mime part, because the japanese encoding contains russian characters as
> well. In short, gnus should not use any `auto-magic', and should try to
> find a charset which contains all characters appearing in a buffer using a
> translation list. This is how SEMI does things.

I'm sorry. I think I might have had my terminology wrong in my previous
e-mail. Yes, of course, the number of parts and charsets used should be
minimized, and if there's one charset which encompasses all the
characters/glyphs/coding systems used in the buffer, then that's the charset
that should be used. But what if there is no single charset? What if you need
two or three charsets? Why should I have to know these things and be forced
to type "<part>" into my message buffer in order for Gnus to switch charsets?
I'm not saying that manual control over such things be removed (if someone
wants to type "<part>", more power to them), but I would like Gnus to take
care of some of these details for me.

Re: MML: The Summation

[Vladimir Volovich]

"EJS" == Edward J Sabol writes:


 >> I think that this is a BAD THING (TM) Gnus MUST NOT try to change
 >> the structure of a message if user did not asked this
 >> explicitly. For example, what should happen if one writes a mixed
 >> chinese-japanese-russian-norwegian text? In which each paragraph
 >> and each sentence contains quotes from all those languages (e.g. a
 >> multilingual dictionary). Gnus in no way should try to divide that
 >> message into parts.
 EJS> Why not? Seriously, I see nothing wrong with it. Please
 EJS> enlighten me as to why it is so horribly bad.

Just imagine how many parts will result from a mixed text of the form:

[word_in_chinese] [word_in_japanese] [word_in_russian] [word_in_norwegian]
[word_in_chinese] [word_in_japanese] [word_in_russian] [word_in_norwegian]
[word_in_chinese] [word_in_japanese] [word_in_russian] [word_in_norwegian]
[word_in_chinese] [word_in_japanese] [word_in_russian] [word_in_norwegian]
[word_in_chinese] [word_in_japanese] [word_in_russian] [word_in_norwegian]

;-) And would it be possible at all to reconstruct the original
message (recover from all those parts)? If the user did not explicitly
specify a part structure of a message, who can do this?

 EJS> There's no single charset that includes Chinese, Japanese,
 EJS> Russian, and Norwegian glyphs, is there? (I'm showing my
 EJS> international charset ignorance here. My apologies.)

Well, i'm also not sure whether MIME officially registered some of
Unicode encodings as allowable charsets for MIME parts, but if it is
so, then yes, there is a single charset which includes all those
languages, and much more. :-)

	Best regards, -- Vladimir.

Re: MML: The Summation

[Raja R Harinath]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:
> Well, i'm also not sure whether MIME officially registered some of
> Unicode encodings as allowable charsets for MIME parts, but if it is
> so, then yes, there is a single charset which includes all those
> languages, and much more. :-)


According to RFC 1700 (STD 02), the allowed character sets include:

  Name: ISO-10646-UCS-2
  Source: the 2-octet Basic Multilingual Plane, aka Unicode
	  this needs to specify network byte order: the standard
	  does not specify (it is a 16-bit integer space)

  Name: ISO-10646-UCS-4
  Source: the full code space. (same comment about byte order,
	  these are 31-bit numbers.

  Name: ISO-10646-UTF-1
  Source: Universal Transfer Format (1), this is the multibyte
	  encoding, that subsets ASCII-7. It does not have byte
	  ordering issues.

- Hari
-- 
Raja R Harinath ------------------------------ harinath@cs.umn.edu
"When all else fails, read the instructions."      -- Cahn's Axiom
"Our policy is, when in doubt, do the right thing."   -- Roy L Ash

Re: MML: The Summation

[Lars Magne Ingebrigtsen]

[-- Attachment #1: Type: text/plain, Size: 777 bytes --]

Vladimir Volovich <vvv@vvv.vsu.ru> writes:

> Well, i'm also not sure whether MIME officially registered some of
> Unicode encodings as allowable charsets for MIME parts, but if it is
> so, then yes, there is a single charset which includes all those
> languages, and much more. :-)

But we have to consider that messages should be possible to read,
shouldn't we?  :-)

I think a typical situation where you get a message with several
charsets is when you quote messages written by people who use other
charsets.  I don't know heads nor tails of Japanese, but if I respond
to a message from Katsumi Yamaoka / 山岡 克美 (who posted using
iso-2022-jp, and is likely not to be able to read Unicode replies),
I'd like to respond using iso-2022-jp.  And that I can do.


[-- Attachment #2: Type: text/plain, Size: 352 bytes --]


However, if I then naïvely uses a iso-8859-1 character in my response, 
I either have to find some superencoding that can encode all of this,
or I have to post multipart.  Since I can't read Unicode, I don't
really see I have any real option here.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: MML: The Summation

[Norbert Koch]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Vladimir Volovich <vvv@vvv.vsu.ru> writes:
> 
> > Well, i'm also not sure whether MIME officially registered some of
> > Unicode encodings as allowable charsets for MIME parts, but if it is
> > so, then yes, there is a single charset which includes all those
> > languages, and much more. :-)
> 
> But we have to consider that messages should be possible to read,
> shouldn't we?  :-)

Nah, we could promote it as a (even better: *the*) new form of
encryption, well sort of :-)
 
> I think a typical situation where you get a message with several
> charsets is when you quote messages written by people who use other
> charsets.  I don't know heads nor tails of Japanese, but if I respond
> to a message from Katsumi Yamaoka / 山岡 克美 (who posted using
> iso-2022-jp, and is likely not to be able to read Unicode replies),
> I'd like to respond using iso-2022-jp.  And that I can do.

Even though you don't know it? Cute :-) 

-- 
Somebody should have told him that being a physicist, on a planet
where the smartest animals hate being alive so much, means never
having to say you're sorry.

Re: MML: The Summation

[Edward J. Sabol]

Excerpts from mail: (18-Nov-98) Re: MML: The Summation by Lars Magne Ingebrigtsen
> But if I want to write a Norwegian character and a Japanese glyph in the
> same article, Message would have to break that up into a multipart message,
> either automatically or by user intervention.

Right. And I'm advocating that it should be automatic and not require any
user intervention (i.e., typing "<part>").

> For instance, Message could go over an article paragraph by paragraph, and
> see whether each paragraph can validly be written in some charset or other,
> and if not, it could ask the user for help.

Yes, that's pretty much what I was suggesting. Does it have to be on a
paragraph-by-paragraph basis though? I'm no MIME expert, so forgive the basic
question, but is it not theoretically possible to compose the MIME parts so
that a Japanese character could be next to a Chinese character on the same
line?

> But I'm (as always) a bit leery of interfaces that try to guess what the
> user wants.

I'm not sure where the guesswork comes in at, but I see I'm losing this
battle, so I'll just crawl back in my lurker hole.

Re: MML: The Summation

[Vladimir Volovich]

"EJS" == Edward J Sabol writes:


 EJS> is it not theoretically possible to compose the MIME parts so
 EJS> that a Japanese character could be next to a Chinese character
 EJS> on the same line?

Why not?

	Best regards, -- Vladimir.

Re: MML: The Summation

[Lars Magne Ingebrigtsen]

"Edward J. Sabol" <sabol@alderaan.gsfc.nasa.gov> writes:

> Yes, that's pretty much what I was suggesting. Does it have to be on a
> paragraph-by-paragraph basis though? I'm no MIME expert, so forgive the basic
> question, but is it not theoretically possible to compose the MIME parts so
> that a Japanese character could be next to a Chinese character on the same
> line?

It is, although I'd guess that most MIME viewers don't display that
properly.  Gnus doesn't, for instance.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen