From_ line problem -- when does it occur?

From_ line problem -- when does it occur?

[Kai Großjohann]

We all know the infamous From_ line problem.  To reiterate: since
From_ lines mark the beginning of a message in a Unix mbox format
file, every "From " at the beginning of a line which does not start a
new message gets a ">" prepended to it, giving ">From ".  The problem
with this is that you can't say whether ">From " in a message is an
escaped From_ line or whether it was in the message in the first
place.

It is clear that the problem can be avoided: you change the file
format.  For example, if each message contains a header which gives
the length of the message, From_ lines within that message need not be
escaped.  Or you could use an algorithm which replaces "From " by
">From " and ">From " by ">>From " and so on.  Or you use a different
format altogether.  Or you use a one-file-per-message storage
structure.

I wonder when does this problem actually occur when using Gnus?
Obviously, it can't be avoided in some situations, but what about the
other situations?

When movemail reads mail from a POP server, it produces babyl output,
so the problem might be avoided in this case.  (Unless the POP server
reads from a Unix mbox file which has this problem.)

But if movemail reads from a Unix mbox file which uses the stupid
From_ line algorithm, the problem will persist within Gnus.

Does the problem in any way depend on the backend used?  I mean,
suppose you are reading incoming mail from a place without the From_
line problem (a babyl file, say): will the From_ line problem appear
in an nnfolder or nnmbox file?  Or does Gnus use a header or something
clever to prevent the problem?

Conversely: suppose movemail is reading a From_ line infected file;
does it help to use nnml as a backend?

kai
-- 
I like BOTH kinds of music.

Re: From_ line problem -- when does it occur?

[Florian Weimer]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> I wonder when does this problem actually occur when using Gnus?

AFAIK Gnus never quotes /^From / lines, but some mail delivery agents
do. It's very likely that if you get your mail via POP, the message
has already been mangled by a MDA.  There's no /^From / manling for
news articles.

AFAIK there were some attempts to get rid of the /^From / problem
(the Content-Length:/Content-Lines: header, for example), but all of
them had rather nasty consequences (read: you could shredder your mail
folder in horrible wayss) when you used old and new software in turn.

Re: From_ line problem -- when does it occur?

[Aaron M. Ucko]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> escaped.  Or you could use an algorithm which replaces "From " by
> ">From " and ">From " by ">>From " and so on.  Or you use a different

I'm pretty sure *something* does this, though I don't remember just
what.

> When movemail reads mail from a POP server, it produces babyl output,
> so the problem might be avoided in this case.  (Unless the POP server

It is, provided you're using a From_-safe backend and a suitable
version of movemail.  (The movemail that comes with FSF Emacs 19
gratuitously escapes From_ lines; some XEmacs versions probably have
this problem too, but I don't know which offhand.)

> reads from a Unix mbox file which has this problem.)

Well, yeah, there's not much you can do there.

> Does the problem in any way depend on the backend used?  I mean,
> suppose you are reading incoming mail from a place without the From_
> line problem (a babyl file, say): will the From_ line problem appear
> in an nnfolder or nnmbox file?  Or does Gnus use a header or something
> clever to prevent the problem?

I think nnmbox and nnfolder use Lines: or Content-Length: or something
to avoid this problem.

> Conversely: suppose movemail is reading a From_ line infected file;
> does it help to use nnml as a backend?

No.  (I saw From_ escaping with nnml and the Emacs 19 movemail.)

-- 
Aaron M. Ucko, KB1CJC <amu@mit.edu> (finger amu@monk.mit.edu)

Re: From_ line problem -- when does it occur?

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* amu@MIT.EDU (Aaron M. Ucko)  on Sun, 22 Aug 1999
| I'm pretty sure *something* does this, though I don't remember just
| what.

The MTA or local delivery agent does this if a Content-Length header does
not exist or is not generated.  It is unusual for an MUA to do it, but
movemail is an unusual beast.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.10 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3wDVHgl+vIlSVSNkRAkTkAJ0QioQZL9sr59lriVuMQKPToJqCcQCgiw2k
morOKxdrZwTJkVEE0FrgmhA=
=pO0+
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Caution: Happy Fun Ball may suddenly
Minion of Nathan - Nathan says Hi! \ accelerate to dangerous speeds.
PGP Key: at a key server near you!  \ 

Re: From_ line problem -- when does it occur?

[Kai Großjohann]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> * amu@MIT.EDU (Aaron M. Ucko)  on Sun, 22 Aug 1999
> | I'm pretty sure *something* does this, though I don't remember just
> | what.
> 
> The MTA or local delivery agent does this if a Content-Length header does
> not exist or is not generated.  It is unusual for an MUA to do it, but
> movemail is an unusual beast.

I think Aaron was talking about replacing "From " with ">From " and
">From " with ">>From " and ">>From " with ">>>From " and so on.

I think you are talking about the "From " to ">From " part only.

kai
-- 
I like BOTH kinds of music.

Re: From_ line problem -- when does it occur?

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann)  on Sun, 22 Aug 1999
| I think you are talking about the "From " to ">From " part only.

I wasn't sure, but yeah, that is what I was talking about.  It is better
than the old way of doing things, but becuase you are not going to get
consistent implementation it should still be considered to be a destructive 
transformation.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.10 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3wX6Agl+vIlSVSNkRAr+fAJ9MqQMxFUtL/RPNq7S0ROq9hxXRNwCg2lss
sr/taZvohE6cbiFpz9Ei7v8=
=Z2zN
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Warning: pregnant women, the elderly, and
Minion of Nathan - Nathan says Hi! \ children under 10 should avoid prolonged
PGP Key: at a key server near you!  \ exposure to Happy Fun Ball.

Re: From_ line problem -- when does it occur?

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Stainless Steel Rat <ratinox@peorth.gweep.net>  on Mon, 23 Aug 1999
| I wasn't sure, but yeah, that is what I was talking about.  It is better
| than the old way of doing things, but becuase you are not going to get

Allow me to rephrase: His way is better than the old way of doing things.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.10 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3wYYVgl+vIlSVSNkRAoPaAJ0azYxnwZbxarJ7Rgv8ss2he6Ig0gCgjG2b
RJvt1JQm+cCqofNXEGUtcG0=
=pB6d
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Caution: Happy Fun Ball may suddenly
Minion of Nathan - Nathan says Hi! \ accelerate to dangerous speeds.
PGP Key: at a key server near you!  \ 

Re: From_ line problem -- when does it occur?

[Aaron M. Ucko]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> I think Aaron was talking about replacing "From " with ">From " and
> ">From " with ">>From " and ">>From " with ">>>From " and so on.

Indeed I was, though I'm still not sure what actually does this.  (The
guy who mentioned this behavior to me described it as standard, but I
think he was confused.)

-- 
Aaron M. Ucko, KB1CJC <amu@mit.edu> (finger amu@monk.mit.edu)

Re: From_ line problem -- when does it occur?

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* amu@MIT.EDU (Aaron M. Ucko)  on Mon, 23 Aug 1999
| Indeed I was, though I'm still not sure what actually does this.  (The
| guy who mentioned this behavior to me described it as standard, but I
| think he was confused.)

There are two things that the delivery agent can do before it dumps a
message into a Unix mbox spool file (aside from doing nothing at all).  The 
first is the "old way".  If it sees the magic regexp "^From " anywhere in
the message other than the first line (the envelopeish thingie), it
prepends ">", turning it into "^>From ", and avoiding problems with MUAs.

The second is the "new way".  This entails generating a Content-Length
header with the byte count of the body of the message.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.10 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3waqfgl+vIlSVSNkRAtgoAKCgXA99Bw7WqcR+28+WNOhI4xjb4ACg2R4W
SmS7sV+jXNtA9AVrjvYNGic=
=qFd7
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 

Re: From_ line problem -- when does it occur?

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Just to follow up on this, as I got distracted and failed to complete the
thought.  Nesting of ">" is nonstandard.  There are probably delivery
agents out there that can be set up to do this (like procmail).  I do not
know of any that do so by default.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.10 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3wav7gl+vIlSVSNkRAtYsAJ9E+dPZ9+T952akba6SiHs8ojEWBwCfU0cI
2KU0tc7x2VZY5Nw9/d4lDvw=
=Plxk
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \