Mail without `From:' lines

Mail without `From:' lines

[Kai Grossjohann]

Hi,

if an incoming mail message does not have a `From:' line (only `From '
which is `X-From-Line:' after nnml munges it), the summary buffer
shows `nobody' as sender.  Reply doesn't work, either.

WIBNI Gnus were to use the information from X-From-Line in lieu of the
missing `From ' line?

Happens both with Gnus 5.1 and Sgnus 0.80 on Emacs 19.30 I think.

kai
-- 
Gleep!

Re: Mail without `From:' lines

[Lars Magne Ingebrigtsen]

Kai Grossjohann <grossjoh@ls6.informatik.uni-dortmund.de> writes:

> WIBNI Gnus were to use the information from X-From-Line in lieu of the
> missing `From ' line?

This is on the Red Gnus todo list.  Or rather, the mail backends will
fudge a missing From: header from the "From " separator.

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Mail without `From:' lines

[Sudish Joseph]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:
> Kai Grossjohann <grossjoh@ls6.informatik.uni-dortmund.de> writes:
> > WIBNI Gnus were to use the information from X-From-Line in lieu of the
> > missing `From ' line?
>
> This is on the Red Gnus todo list.  Or rather, the mail backends will
> fudge a missing From: header from the "From " separator.

This isn't a good idea, as you're effectively setting the reply
address for the message to the envelope sender--and that's only
allowed for bounces, I think (or it might not be, I don't have the
time to dig into 821/822 to check this :).  In fact, I'm pretty
certain that you're not supposed to reply to the conents of Sender:,
and that's a


4.4.4.  AUTOMATIC USE OF FROM / SENDER / REPLY-TO

    For systems which automatically  generate  address  lists  for
    replies to messages, the following recommendations are made:

        o   The "Sender" field mailbox should be sent  notices  of
            any  problems in transport or delivery of the original
            messages.  If there is no  "Sender"  field,  then  the
            "From" field mailbox should be used.

        o   The  "Sender"  field  mailbox  should  NEVER  be  used
            automatically, in a recipient's reply message.

        o   If the "Reply-To" field exists, then the reply  should
            go to the addresses indicated in that field and not to
            the address(es) indicated in the "From" field.



Kai's original idea is better as it only effects what gets displayed
in the summary, no (illegal in this instance?) munging of headers.

-Sudish

Re: Mail without `From:' lines

[Sudish Joseph]

[ Argh, I thought the Supercite keymap was hanging off C-c C-s, with
  unfortunate results.  Hmm, SC's keymap isn't to be found, a side
  effect of the change of hook names in message.el?  ]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:
> Kai Grossjohann <grossjoh@ls6.informatik.uni-dortmund.de> writes:
> > WIBNI Gnus were to use the information from X-From-Line in lieu of the
> > missing `From ' line?
>
> This is on the Red Gnus todo list.  Or rather, the mail backends will
> fudge a missing From: header from the "From " separator.

I wrote:
mishap> This isn't a good idea, as you're effectively setting the reply
mishap> address for the message to the envelope sender--and that's only
mishap> allowed for bounces, I think (or it might not be, I don't have the
mishap> time to dig into 821/822 to check this (*).  In fact, I'm pretty
mishap> certain that you're not supposed to reply to the conents of Sender:,
mishap> and that's a

Anyways, the above paragraph was going to be changed to end on the
lines of: "you're not supposed to reply to the contents of Sender,
and Sender's a whole lot better for this purpose than the envelope
recipient (as mailing list expanders are required to rewrite
env. rec.); so I'd assume that replying _manually_ to env. recip. is
strictly taboo."

Here's 822 on replying to Sender.

822> 4.4.4.  AUTOMATIC USE OF FROM / SENDER / REPLY-TO
822>
822>     For systems which automatically  generate  address  lists  for
822>     replies to messages, the following recommendations are made:
822>
822>         o   The "Sender" field mailbox should be sent  notices  of
822>             any  problems in transport or delivery of the original
822>             messages.  If there is no  "Sender"  field,  then  the
822>             "From" field mailbox should be used.
822>
822>         o   The  "Sender"  field  mailbox  should  NEVER  be  used
822>             automatically, in a recipient's reply message.

Kai's original idea is better as it only effects what gets displayed
in the summary, no (illegal in this instance?) munging of headers.

Basically, this kind of thing gives me the shivers, since it could
well backfire in the manner that Pine's use of Newsgroup has.  No, I
haven't thought it through, but it still gives me the creeps. :-)

-Sudish "

(*) And I should have stuck to that, instead of losing more time now,
greeping 822 and then rewriting this :-)

Re: Mail without `From:' lines

[Sudish Joseph]

I gibbered:
> Anyways, the above paragraph was going to be changed to end on the
> lines of: "you're not supposed to reply to the contents of Sender,
> and Sender's a whole lot better for this purpose than the envelope
> recipient (as mailing list expanders are required to rewrite
> env. rec.); so I'd assume that replying _manually_ to env. recip. is
> strictly taboo."

Um, I meant envelope sender wherever I said envelope recipient, of
course.

Also, I think I'd prefer having a blank display to even Kai's
suggestion of using env. sender--IBM's vnet mail gateway rewrites it
to a source-routed form (advertising itself, of course) that is so
ugly to look at that a blank would be far better. :-)

-Sudish

Re: Mail without `From:' lines

[Per Abrahamsen]

RFC 822 is irrelevant since a message without a "From: " is not an RFC
822 message in the first place.

>>>>> "RFC822" == Standard for ARPA Internet Text Messages wrote:

RFC822> A.3.  COMPLETE HEADERS

RFC822>      A.3.1.  Minimum required

RFC822>      Date:     26 Aug 76 1429 EDT        Date:     26 Aug 76 1429 EDT
RFC822>      From:     Jones@Registry.Org   or   From:     Jones@Registry.Org
RFC822>      Bcc:                                To:       Smith@Registry.Org

The question is how to transform a piece of random junk in unix mbox
format into something resembling an RFC 822 message.  Using the
information in the `From ' message separator to generate a missing
`From:' and `Date:' header has worked fine for me when converting old
mbox.out files, and works fine for VM and many other mail agents.

And the right term is `message separator', the `From ' lines in an
mbox.out has nothing to do with sendmail envelope addresses.

Re: Mail without `From:' lines

[Sudish Joseph]

Per Abrahamsen <abraham@dina.kvl.dk> writes:
> RFC 822 is irrelevant since a message without a "From: " is not an RFC
> 822 message in the first place.

Hardly.  Not when your goal is to generate an 822 format message.  The
issue isn't what format the original crud is in, the issue is what
format it will be transformed into.

> The question is how to transform a piece of random junk in unix mbox
> format into something resembling an RFC 822 message.  Using the

Just like it's Pine's intent to transform a random piece of junk into
a format that's suitable for following up to as a USENET message.  We
all know how that went, never mind that Pine's usage of Newsgroup
makes more sense in theory.

> information in the `From ' message separator to generate a missing
> `From:' and `Date:' header has worked fine for me when converting old
> mbox.out files, and works fine for VM and many other mail agents.

Hitting "r" on such a message is going to cause problems for anything
other than personal mail (yes, that's 822 behaviour, but then your
stated goal is to transform to 822 format).

I have yet to see an MDA-generated UNIX From_ line contain anything
other than the envelope sender (+ctime date), I'd be grateful if you
showed me such an example.

To put it simply:
Anything that can be achieved by munging together a From: header by
inspecting From_ can be done by inspecting From_ whenever you do not
see a From:.  If that doesn't please you, put the information in an
X-Bogosity: header.  Don't mess up a message that might later be
processed by an agent other than GNUS.

-Sudish

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "SJ" == Sudish Joseph <sudish@VNET.IBM.COM> writes:

SJ> Just like it's Pine's intent to transform a random piece of junk into
SJ> a format that's suitable for following up to as a USENET message.  We
SJ> all know how that went, never mind that Pine's usage of Newsgroup
SJ> makes more sense in theory.

Pine wasn't dealing with random junk and it wasn't doing any
transformations, and it went against an established tradition for how
to use a particular header.  None of these are relevant for the issue
at hand, quite the contrary Gnus would be following an established
tradition, which mailers like VM has shown works well in practice.

It seems to me that converting the 'From ' line in the absence of a
`From:' header will be the right thing in all real examples, and no
worse than not converting it in hypothetical examples.

SJ> To put it simply:
SJ> Anything that can be achieved by munging together a From: header by
SJ> inspecting From_ can be done by inspecting From_ whenever you do not
SJ> see a From:.  If that doesn't please you, put the information in an
SJ> X-Bogosity: header.  Don't mess up a message that might later be
SJ> processed by an agent other than GNUS.

The `From ' line is an artifact of the mbox format, so leaving it
alone is not an option with other backends.  Converting is to an `X-'
header will break if the message is later read by another mail agent
(like VM), who will then not be able to access the information.  The
only way to preserve the information is to convert the broken message
into standard (RFC 822) format.

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "SJ" == Sudish Joseph <sudish@VNET.IBM.COM> writes:

SJ> I have yet to see an MDA-generated UNIX From_ line contain anything
SJ> other than the envelope sender (+ctime date), I'd be grateful if you
SJ> showed me such an example.

Using /bin/mail in research unix to send mail to a local user will
result in a mail with no From: line.  The intention is that one should
use the `From ' line for replies.  The system administrator at
research.att.com insists that this is the correct behaviour, and that
everyone else are wrong.  They were first.

However, I'm more concerned about files create with `FCC:' by mail
agents, that's the only source for `From:' less mail I know of
outside AT&T.

Re: Mail without `From:' lines

[Kai Grossjohann]

>>>>> Kai Grossjohann <grossjoh@ls6.informatik.uni-dortmund.de>
>>>>> writes:

  Kai> WIBNI Gnus were to use the information from X-From-Line in lieu of the
  Kai> missing `From ' line?

>>>>> Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

  Lars> This is on the Red Gnus todo list.  Or rather, the mail backends will
  Lars> fudge a missing From: header from the "From " separator.

>>>>> On 03 May 1996 18:58:25 -0400, "Sudish Joseph"
>>>>> <sudish@VNET.IBM.COM> said:

  Sudish> This isn't a good idea, as you're effectively setting the
  Sudish> reply address for the message to the envelope sender--and
  Sudish> that's only allowed for bounces, I think [...]

I'm talking about the cases where you have neither of `From:',
`Sender:' and `Reply-To:', you only have the `From ' line to go on.
Then surely using the address given in the `From ' line can't be worse
than using `' (the empty string) as reply address?

kai
-- 
Gleep!

Re: Mail without `From:' lines

[Per Persson]

Kai Grossjohann <grossjoh@charly.informatik.uni-dortmund.de> writes:

   I'm talking about the cases where you have neither of `From:',
   `Sender:' and `Reply-To:', you only have the `From ' line to go on.
   Then surely using the address given in the `From ' line can't be worse
   than using `' (the empty string) as reply address?

On mailinglists, the "From " line is mostly a pointer to the owner of
the mailinglist. I don't think that this persons wants to recieve
mails because something is broken on the actual senders side. I know I
wouldn't, I know I would hate it, I know I would track down the dimwit
who coded the feature and force him to learn texinfo.

-- 
Bahnhof; http://www.bahnhof.se/      <pp@bahnhof.se> | a   difference   of
  (work) phone(voice/fax):      +46 18 100899/103737 | opinion is possible
 PFAWWW; http://pfawww.pp.se/pp/   <pp@pfawww.pp.se> | ^. .^     anum meum  
 (!work) phone(voice)    :             +46 18 247473 | ( @ )       aperies

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

PP> On mailinglists, the "From " line is mostly a pointer to the owner of
PP> the mailinglist. I don't think that this persons wants to recieve
PP> mails because something is broken on the actual senders side. 

If the mailing list sends out mail without `From: ' lines, he deserves
to get a direct feed of `talk.bizarre' forwarded to his personal mail
account.

Re: Mail without `From:' lines

[Per Abrahamsen]

I just tried to a message without a From: header by talking to
sendmail directly.  Sendmail automatically created a From: header,
using the envelope sender.  

Here is some extracts from the test (indentation added by me):

kleene% telnet smtp smtp
	mail from: amanda
	250 amanda... Sender ok
	rcpt to: abraham
	250 abraham... Recipient ok
	help data
	data
	354 Enter mail, end with "." on a line by itself
	test
	.
	250 NAA26313 Message accepted for delivery
	quit

kleene% cat /var/spool/mail/abraham 
	From amanda Mon May  6 13:58:56 1996
	Date: Mon, 6 May 1996 13:57:50 +0200
	From: amanda
	Message-Id: <199605061157.NAA26313@elc1.dina.kvl.dk>
	Apparently-To: abraham
	 
	test

kleene% 

Re: Mail without `From:' lines

[Sudish Joseph]

Per Abrahamsen <abraham@dina.kvl.dk> writes:
> It seems to me that converting the 'From ' line in the absence of a
> `From:' header will be the right thing in all real examples, and no

Here's a real example: *Every* message I receive from the internet has
envelope sender set to a fixed address of this form:
<@smtp.relay,relay.id@vm.node.that.doesn't.speak.tcp.ip>
(also a nice real world example of a valid use of source routes)

> worse than not converting it in hypothetical examples.

Not necessarily.  Using the envelope sender makes sense for local mail
and mail sent to directly connected hosts.  True, this accounts for
the vast majority of mail messages (and ~ 10% of my mail).

Then again, the percentages of messages w/o From: lines is vanishingly
small.  I'd prefer that GNUS did not tear, damage or spindle the
headers (other than X-blah) of any message I receive.  I'll detect the
lack of a From: line when I try to reply to any such errant message,
and I'll be able to make a far better estimate of the actual return
address than GNUS.

Unless, of course, someone wants to write that cool routine to parse
Received: headers to see where the mail originated, and then figure
out how to reply to it.  This is far more likely to be correct than
using envelope sender.

> The `From ' line is an artifact of the mbox format, so leaving it
> alone is not an option with other backends.  Converting is to an `X-'
> header will break if the message is later read by another mail agent
> (like VM), who will then not be able to access the information.  The
> only way to preserve the information is to convert the broken message
> into standard (RFC 822) format.

GNUS current X-From-Line preserves that information.  Putting it in
From: destroys information (namely, the far more important information
that the original message did not have a From: line).  I do not want
to reply to From_ blindly.  I'd prefer that my MUA choked and died on
such messages.

Oh, yes. Please note that replies send to envelope sender can
disappear w/o any fuss.  You won't even have the benefit of a bounce
to know that you must resend.

At least make this an option that defaults to nil.
-Sudish

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "SJ" == Sudish Joseph <sudish@VNET.IBM.COM> writes:

SJ> Here's a real example: *Every* message I receive from the internet has
SJ> envelope sender set to a fixed address of this form:
SJ> <@smtp.relay,relay.id@vm.node.that.doesn't.speak.tcp.ip>
SJ> (also a nice real world example of a valid use of source routes)

How does it From: header looks?

If it looks right, the example is irrelevant.

SJ> I'll detect the
SJ> lack of a From: line when I try to reply to any such errant message,

Have you ever received such a message?  I have been unable to generate
one with sendmail or /bin/mail here.  As I noted in another message,
sendmail insist on putting the envelope address in the From: line if
there isn't one in the first place.

SJ> At least make this an option that defaults to nil.

Why should the default be something that is wrong in all examples you
and me can think of? 

Re: Mail without `From:' lines

[Per Persson]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

   If the mailing list sends out mail without `From: ' lines, he deserves
   to get a direct feed of `talk.bizarre' forwarded to his personal mail
   account.

It's not up to the mailinglist to add a "From: " header.

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

PP> It's not up to the mailinglist to add a "From: " header.

I think the mailing list should reject such submissions, but I suspect
most mailing lists will add a `From: ' header based on the envelope if
you actually try.

Re: Mail without `From:' lines

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

PP> It's not up to the mailinglist to add a "From: " header.

Correct; it is up to the MTA that initially handles each message to
ensure that a valid From: header is generated if one does not already
exist (cf, RFC822).

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3
Charset: noconv

iQCVAwUBMY4r356VRH7BJMxHAQHMBAQArElgLx/VTplNsM6Fc+DYU+wAzRrsVwWL
b6Xq+x6FpWci2BVa58qV7f1KHKBe01e5IUxugNUOpm/ju8QPU+I5/BU9OBVvZ/AQ
hN0oE4M+Z6zNjxob4Hu5JVqGS4opFvbrIFw2fPMf9aK4sa9dk8p9m2nxRLD3Kqv2
4lwoV1atDmg=
=oqmY
-----END PGP SIGNATURE-----
-- 
Rat <ratinox@ccs.neu.edu>          \ When not in use, Happy Fun Ball should be
PGP Public Key: Ask for one today!  \ returned to its special container and
http://www.ccs.neu.edu/home/ratinox/ \ kept under refrigeration.

Re: Mail without `From:' lines

[Per Persson]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

   >>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

   PP> It's not up to the mailinglist to add a "From: " header.

   I think the mailing list should reject such submissions, but I suspect
   most mailing lists will add a `From: ' header based on the envelope if
   you actually try.

These are the relevant headers from your mail;

>From ding-request@ifi.uio.no  Mon May  6 19:13:59 1996
Sender: abraham@dina.kvl.dk
From: Per Abrahamsen <abraham@dina.kvl.dk>

1) the list adds an appropriate "From " header.
2) your MUA or your MTA adds a "Sender: " header, in perfect
   conditions this should be done by the MTA.
3) your MUA adds a "From: " header--if missing, the MTA might do it.

An MUA should use option three first, then option two if three is
missing which is shouldn't be.

An MUA should NOT use the "From " header, just as it shouldn't use the
"Return-Path: " header. The "From " header isn't added by the MUA or
the MTA, it's actually added my the MDA. 

This isn't really relevant to this mailinglist, it should be fought
over on comp.mail.headers. IMHO, GNUS should NOT use a header which is
likly to return something you don't want to use, at least not without
querying the user first.

-- 
Bahnhof; http://www.bahnhof.se/      <pp@bahnhof.se> | a   difference   of
  (work) phone(voice/fax):      +46 18 100899/103737 | opinion is possible
 PFAWWW; http://pfawww.pp.se/pp/   <pp@pfawww.pp.se> | ^. .^     anum meum  
 (!work) phone(voice)    :             +46 18 247473 | ( @ )       aperies

Re: Mail without `From:' lines

[Sudish Joseph]

Per Abrahamsen <abraham@dina.kvl.dk> writes:
> How does it From: header looks?
>
> If it looks right, the example is irrelevant.

The example simply points out that your notion of "real world" is
incorrect.

> SJ> I'll detect the
> SJ> lack of a From: line when I try to reply to any such errant message,
>
> Have you ever received such a message?  I have been unable to generate
> one with sendmail or /bin/mail here.  As I noted in another message,
> sendmail insist on putting the envelope address in the From: line if
> there isn't one in the first place.

Precisely why I would prefer to have that one glaring exception be
handled manually, instead of having GNUS decide for me.

> SJ> At least make this an option that defaults to nil.
>
> Why should the default be something that is wrong in all examples you
> and me can think of?

Uh, why is it "wrong"?  Would you rather that the default was to set
up a situation where a user can reply to a mail only to have it be
delivered to some auto-scanner that sinks it unread?

We agree that the lack of "From:" is an extremely rare incident.  Then
why do you wish that the user forgo the pleasure of handling this
once-in-a-lifetime incident herself?  Besides, like you've shown by
your sendmail example, the message would have to arrive by a really
weird route to not have a From: line in the first place.  Given this,
the envelope sender information is unlikely to be pointing to the
originating host; definitely not an address that should be
auto-selected for replies.

-Sudish

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

PP> From ding-request@ifi.uio.no  Mon May  6 19:13:59 1996
PP> Sender: abraham@dina.kvl.dk
PP> From: Per Abrahamsen <abraham@dina.kvl.dk>

Since it has a From: header, the example is irrelevant.  
Nobody is proposing that Gnus use `From' instead of `From:'.

Re: Mail without `From:' lines

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----

>>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

PP> 1) the list adds an appropriate "From " header.

'From<space>' is actually generated by the delivery agent on "your" (the
recipient's) mail server when the mail is delivered into the system
mailbox.  Not all mail systems use the Unix mail delimiters; PMDF uses a
string of four ^A characters to begin and end every message, thus no
'From<space>' is necessary (and is often not generated).

PP> 2) your MUA or your MTA adds a "Sender: " header, in perfect
PP>    conditions this should be done by the MTA.

MTAs are technically the only entities that are supposed to add
'Sender:'; this is what the -s switch to sendmail is for, for instance.
MUAs should not generate 'Sender:' fields, and they should ignore this
header in messages received.

PP> 3) your MUA adds a "From: " header--if missing, the MTA might do it.

The MTA *must* do it if the MUA does not.  It is required that the MTA
do it if the MUA has not provided a 'From:' header.

-----BEGIN PGP SIGNATURE-----
Version: 2.6.3
Charset: noconv

iQCVAwUBMY465p6VRH7BJMxHAQHkMAP/XmuThI1d0x6qkbkQ8DVSEsdEidNic6hy
ecOUgoO+dIp890pb0jUN94jpwXuDWBFW0l7ABFKz1Ca9aWIT45VS/anG0bwPXkeE
7T2shy18s4ct13hS8m43p0oIagxnlw5GcSThJlFeGNtvtVIOllm0JWiiF4UyAhVT
dJJeYq2kvAo=
=pv1s
-----END PGP SIGNATURE-----
-- 
Rat <ratinox@ccs.neu.edu>          \ Happy Fun Ball may stick to certain types
PGP Public Key: Ask for one today!  \ of skin.
http://www.ccs.neu.edu/home/ratinox/ \ 

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "SJ" == Sudish Joseph <sudish@VNET.IBM.COM> writes:

SJ> The example simply points out that your notion of "real world" is
SJ> incorrect.

No, your example support what I have said.  External mail has a
`From:' header. 

SJ> Precisely why I would prefer to have that one glaring exception be
SJ> handled manually, instead of having GNUS decide for me.

It is *not* a glaring exception!  Messages in mbox format without
From: lines are quite common.  They are just not created by mail
delivery agents, but other programs such as mail and news readers
using `FCC: '.

My point has has been that incoming, external mail without `From: '
is rare, and that we should set the default for the common case,
namely `From:' less messages generated by other programs or perhaps
local mailers.

Re: Mail without `From:' lines

[Per Persson]

Per Abrahamsen <abraham@dina.kvl.dk> writes:

   Since it has a From: header, the example is irrelevant.  Nobody is
   proposing that Gnus use `From' instead of `From:'.

You didn't read the rest of my mail, did you?

/pp.

Re: Mail without `From:' lines

[Per Abrahamsen]

>>>>> "PP" == Per Persson <pp@pfawww.pp.se> writes:

PP> You didn't read the rest of my mail, did you?

Yes I did.  The MTA will add a From: if there isn't one, using the
envelope sender if necessary.  This is true for mailing lists too.