Re: More POP madness

Re: More POP madness

[Chris Tessone]

[-- Attachment #1: Type: text/plain, Size: 247 bytes --]


Well, here's an incoming file with one message in it. I sent this
message to tessone@imsa.edu, it was saved to my mailbox on
cappio.imsa.edu (postoffice.imsa.edu) and then was pop'd to
pollux.imsa.edu by fetchmail, read in by Gnus, and ignored.


[-- Attachment #2: this message was lost --]
[-- Type: message/rfc822, Size: 1563 bytes --]

From: Chris Tessone <tessone@fnal.gov>
To: tessone@imsa.edu
Subject: woo
Date: Thu, 08 Apr 1999 18:17:22 -0500
Message-ID: <t6qvhf6vh19.fsf@fsui02.fnal.gov>


hi

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/


[-- Attachment #3: Type: text/plain, Size: 301 bytes --]


-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Kai.Grossjohann]

Chris Tessone <tessone@imsa.edu> writes:

  > Well, here's an incoming file with one message in it. I sent this
  > message to tessone@imsa.edu, it was saved to my mailbox on
  > cappio.imsa.edu (postoffice.imsa.edu) and then was pop'd to
  > pollux.imsa.edu by fetchmail, read in by Gnus, and ignored.

This message does not have a From_ line.  (In Incoming files, each
message should start with the five characters "From " and end with an
empty line.)

How do you fetch stuff from the POP server?  Via movemail or via
pop3.el? 

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Chris Tessone]

But Gnus didn't give a message about the mailbox being corrupt...

Anyway, I use fetchmail from Gnus:

      nnmail-spool-file '((pop :program
"/home/fla/tessone/bin/fetchmail -L /hom
e/fla/tessone/.fetchmaillog --mda procmail"
                               :server "cappio.imsa.edu"
                               :password "secret!"
                               :user "tessone"))

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Kai.Grossjohann]

Chris Tessone <tessone@fnal.gov> writes:

  >       nnmail-spool-file '((pop :program
  > "/home/fla/tessone/bin/fetchmail -L /hom
  > e/fla/tessone/.fetchmaillog --mda procmail"
  >                                :server "cappio.imsa.edu"
  >                                :password "secret!"
  >                                :user "tessone"))

Hm.  So the ouput of fetchmail doesn't seem to be in mbox format.
Hm.  I'm not sure which program should add the From_ line, but either
fetchmail or procmail should do that.

(Or create output in babyl or MMDF format.)

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Colin Rafferty]

Kai Grossjohann writes:

> Chris Tessone <tessone@fnal.gov> writes:
>>       nnmail-spool-file '((pop :program
>> "/home/fla/tessone/bin/fetchmail -L /hom
>> e/fla/tessone/.fetchmaillog --mda procmail"
>>                                :server "cappio.imsa.edu"
>>                                :password "secret!"
>>                                :user "tessone"))

> Hm.  So the ouput of fetchmail doesn't seem to be in mbox format.
> Hm.  I'm not sure which program should add the From_ line, but either
> fetchmail or procmail should do that.

I run fetchmail->procmail externally, and just point Gnus to the spool 
that procmail creates.

I use the following in my .fetchmailrc:

    mda "procmail -f fetchmail"

-- 
Colin

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Kai.Grossjohann@CS.Uni-Dortmund.DE  on Fri, 09 Apr 1999
| Hm.  So the ouput of fetchmail doesn't seem to be in mbox format.

Kai, you just got through telling me that what a POP3 retrieval program
spits out does not need to be in mbox format.  In fact, you pointed out
that strictly by RFCs, what a POP3 server spits out cannot be in mbox
format because the SMTP envelope is not part of an RFC 822 message.

Which is it?

| Hm.  I'm not sure which program should add the From_ line, but either
| fetchmail or procmail should do that.

Nope.  What *should* generate that line is the SMTP agent on the sending
end.  The retrieval mechanism should not need to generate the envelope
again.

Which is why I maintain that POP servers that strip that information are
broken.

Sorry for being particularly bitchy this evening, but I spent 8 friggin'
hours fixing my company's mail system and worrying if any of the almost
1300 messages in the queue were lost.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3DprUgl+vIlSVSNkRAtQNAJ9MmFhSsERQOg6G2nJlk+9nUvVXMwCg2TwT
TPdULRDQoz7nxsbrcJfuGLo=
=aMyn
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ If Happy Fun Ball begins to smoke, get
Minion of Nathan - Nathan says Hi! \ away immediately. Seek shelter and cover
PGP Key: at a key server near you!  \ head.

Re: More POP madness

[Christopher K Davis]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> * Kai.Grossjohann@CS.Uni-Dortmund.DE  on Fri, 09 Apr 1999
> | Hm.  So the ouput of fetchmail doesn't seem to be in mbox format.

> Kai, you just got through telling me that what a POP3 retrieval program
> spits out does not need to be in mbox format.  In fact, you pointed out
> that strictly by RFCs, what a POP3 server spits out cannot be in mbox
> format because the SMTP envelope is not part of an RFC 822 message.

> Which is it?

fetchmail is not a POP3 server.  If fetchmail (or a fetchmail/procmail
combo) is specified to output an mbox format file, then that's what it
should do.

-- 
Christopher Davis * <ckd-sig@ckdhr.com> * <URL:http://www.ckdhr.com/ckd/>
Put location information in your DNS! <URL:http://www.ckdhr.com/dns-loc/>

Re: More POP madness

[Kai.Grossjohann]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

  > * Kai.Grossjohann@CS.Uni-Dortmund.DE  on Fri, 09 Apr 1999
  > | Hm.  So the ouput of fetchmail doesn't seem to be in mbox format.
  > 
  > Kai, you just got through telling me that what a POP3 retrieval program
  > spits out does not need to be in mbox format.  In fact, you pointed out
  > that strictly by RFCs, what a POP3 server spits out cannot be in mbox
  > format because the SMTP envelope is not part of an RFC 822 message.
  > 
  > Which is it?

Er, hm.  I was tired when I wrote that.  Of course, the POP3 server
spits out RFC 822, which is not in mbox format.  But Gnus expects mbox
format.  So, one of the programs involved (fetchmail or procmail, I
gather) should transform the output of the POP3 server to mbox format.

Right?

  > | Hm.  I'm not sure which program should add the From_ line, but either
  > | fetchmail or procmail should do that.
  > 
  > Nope.  What *should* generate that line is the SMTP agent on the sending
  > end.  The retrieval mechanism should not need to generate the envelope
  > again.
  > 
  > Which is why I maintain that POP servers that strip that information are
  > broken.

Are you saying that the POP3 RFC is broken because it specifies that
the output of a RETR command must be an RFC 822 message (which does
not contain an envelope)?

  > Sorry for being particularly bitchy this evening, but I spent 8 friggin'
  > hours fixing my company's mail system and worrying if any of the almost
  > 1300 messages in the queue were lost.

No problem.  What does friggin' mean?  I mean, I can sort of glork the
meaning from context, but one never knows...

Losing mail is awful.  Good luck.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Stephen Zander]

>>>>> "Kai" == Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:
    Kai> No problem.  What does friggin' mean?  I mean, I can sort of
    Kai> glork the meaning from context, but one never knows...

The polite substitute for one of my favourite Old English (Saxon I
think) words... :)

-- 
Stephen
---
Long noun chains don't automatically imply security. - Bruce Schneier

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Kai.Grossjohann@CS.Uni-Dortmund.DE  on Sat, 10 Apr 1999
| Are you saying that the POP3 RFC is broken because it specifies that
| the output of a RETR command must be an RFC 822 message (which does
| not contain an envelope)?

I am saying that a POP server that strips *ANYTHING* from a message is
broken.  POP is *NOT* allowed to strip anything from a message.  What is on 
the POP server is exactly what should appear on the POP client.  Once
properly downloaded, the message may be massaged into something the MUA
groks.

In other words, there is no good reason for pop3-movemail or fetchmail to
generate artificial SMTP envelopes.  The whole process should not be
required except for the fact that some POP servers do the RFC 822 thing
while ignoring the mail handling ubercommandment, 'thou shalt not remove
anything from a message in transit.'
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EKh3gl+vIlSVSNkRAgWPAJ0ZgHd/RAuiHB2VqSKMb7/QNRzIdgCcCvW4
5dmWdP5sVlvK6I8ds8RgHAg=
=NgUh
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ If Happy Fun Ball begins to smoke, get
Minion of Nathan - Nathan says Hi! \ away immediately. Seek shelter and cover
PGP Key: at a key server near you!  \ head.

Re: More POP madness

[Christopher K Davis]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> I am saying that a POP server that strips *ANYTHING* from a message is
> broken.  POP is *NOT* allowed to strip anything from a message.

A *message* does not include the delimeter.

> What is on the POP server is exactly what should appear on the POP
> client.  Once properly downloaded, the message may be massaged into
> something the MUA groks.

And if the POP server uses an out-of-band delimeter in its spool storage 
(maildir, database, index file with byte counts)--then what?

> In other words, there is no good reason for pop3-movemail or fetchmail
> to generate artificial SMTP envelopes.

Of course not, they're not doing SMTP (well, I suppose fetchmail is when 
it's injecting mail into the local mail system, but I don't think that's 
what you mean).

You're confusing an SMTP envelope ("MAIL From:<foo@example.com>") with
an mbox delimeter ("From foo@example.com date").  The fact that for
hysterical raisins having to do with UUCP envelopes ("From joe date
remote from uuhost") the SMTP envelope sender information gets reflected 
in the mbox delimiter is irrelevant.

After all, in an MMDF-delimeter world, the sender information isn't in
the delimeter.  According to RFC 821, the Return-Path: header is the
place to preserve the SMTP sender envelope information.

> The whole process should not be required except for the fact that some
> POP servers do the RFC 822 thing while ignoring the mail handling
> ubercommandment, 'thou shalt not remove anything from a message in
> transit.'

The message, when it arrived via SMTP, *HAD NO DELIMITER*.  *If* one
got added as part of the spool format on the POP server host (again,
depending on the spool format delimiters may not be necessary) then it
should not (according to the specs) be sent to the POP client as if it
were part of the message.

-- 
Christopher Davis * <ckd-sig@ckdhr.com> * <URL:http://www.ckdhr.com/ckd/>
Put location information in your DNS! <URL:http://www.ckdhr.com/dns-loc/>

Re: More POP madness

[Kai.Grossjohann]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

  > * Kai.Grossjohann@CS.Uni-Dortmund.DE  on Sat, 10 Apr 1999
  > | Are you saying that the POP3 RFC is broken because it specifies that
  > | the output of a RETR command must be an RFC 822 message (which does
  > | not contain an envelope)?
  > 
  > I am saying that a POP server that strips *ANYTHING* from a message is
  > broken.  POP is *NOT* allowed to strip anything from a message.  What is on 
  > the POP server is exactly what should appear on the POP client.  Once
  > properly downloaded, the message may be massaged into something the MUA
  > groks.

I suppose endless debate will ensue about whether or not the envelope
sender and/or the mbox delimiter (which Christopher points out are not
the same thing) are actually part of the message.

If one thinks that it is not part of the message, then stripping it
does not mean stripping anything from the message.

Let me go on record saying that IMHO the From_ line is just a
delimiter and not part of the message.  It is useful information,
though, and thus I'm happy that Gnus preserves it as X-From-Line
header.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Look, this is really very simple: mail handling and delivery is outside the
scope of RFC 822.

A message delivered to an Internet host, even if delivered 'locally',
regardless of its contents, is delivered by SMTP by defintion.  By
definition it must have an envelope.  That envelope and the information it
contains have become part of the message, information that is necessary for
the proper handling of that message.  If that information is removed for
any reason, the message can no longer be properly handled.  Whatever agent
is responsible for this destruction of information is broken.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EObygl+vIlSVSNkRAmFsAKDFXbBBDNvJSPCJGI9J4/AzOW0egwCgqjEB
6yx6UOcaOKHBUT4HVC0OuhI=
=XuJV
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Ingredients of Happy Fun Ball include an
Minion of Nathan - Nathan says Hi! \ unknown glowing substance which fell to
PGP Key: at a key server near you!  \ Earth, presumably from outer space.

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Kai.Grossjohann@CS.Uni-Dortmund.DE  on Sun, 11 Apr 1999
| I suppose endless debate will ensue about whether or not the envelope
| sender and/or the mbox delimiter (which Christopher points out are not
| the same thing) are actually part of the message.

They most certainly are.  They are not part of an RFC 822 message, but a
messge in transit is more than an RFC 822 message.  And a message on a POP
server is still in transit because final delivery has not been realized.

Look at it this way: POP is part of a store-and-forward delivery mechanism.
A message sitting on a server somewhere is in the stored state awaiting
forwarding to the next node in its trip.  A POP server that assumes it is
forwarding directly to a client is making an assumption it has no business
making.  For all it knows, it could be forwarding mail into an internal
Internet-style network's SMTP gateway.  If the POP server strips the
envelope, the message cannot be properly delivered on the other side --
envelopes are generated by the originating host, nobody else.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EOjlgl+vIlSVSNkRAm/JAJ4/32b+9jFmWJw1atI28MXPLeYQvQCg6ce2
L4thqTmN2uOQi/ifi+eMpeI=
=kpm9
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball may stick to certain types
Minion of Nathan - Nathan says Hi! \ of skin.
PGP Key: at a key server near you!  \ 

Re: More POP madness

[Kai.Grossjohann]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

  > They most certainly are.  They are not part of an RFC 822 message,
  > but a messge in transit is more than an RFC 822 message.  And a
  > message on a POP server is still in transit because final delivery
  > has not been realized.

The POP3 RFC specifically requires messages to conform to RFC 822,
whether or not it should do so or not.

Have I mentioned `endless argument' before? :-)

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Kai.Grossjohann]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

  > [...]  A POP server that assumes it is forwarding directly to a
  > client is making an assumption it has no business making.  For all
  > it knows, it could be forwarding mail into an internal
  > Internet-style network's SMTP gateway.  If the POP server strips
  > the envelope, the message cannot be properly delivered on the
  > other side -- envelopes are generated by the originating host,
  > nobody else.

Hm.  Apparently, POP3 wasn't designed this way, else the writers of
the RFC wouldn't have forbidden the envelopes to persist.  Hm.

So, ought POP3 to be changed, then?

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Christopher K Davis]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> Look, this is really very simple: mail handling and delivery is outside
> the scope of RFC 822.

True.

> A message delivered to an Internet host, even if delivered 'locally',
> regardless of its contents, is delivered by SMTP by defintion.

Not true.  It could be delivered locally, by SMTP, by UUCP rmail, by
UUCP BSMTP (which is admittedly a form of SMTP), by QMTP, by RJE (hey,
*I* remember BITNET)....

However, the essential point (which is what I think you had in mind)
that all of these protocols have some notion of an envelope sender and
one or more envelope recipients is still valid.

> By definition it must have an envelope.  That envelope and the
> information it contains have become part of the message, information
> that is necessary for the proper handling of that message.

Agreed.

> If that information is removed for any reason, the message can no longer
> be properly handled.  Whatever agent is responsible for this destruction
> of information is broken.

But then why are you accepting messages with MMDF delimiters?  Those
don't contain the envelope information, and therefore are, by this
definition, broken.

Besides, RFC 822 defines a location and format for the envelope sender--the
Return-Path header.  RFC 821 specifies that a receiver-SMTP is responsible
for inserting a Return-Path.  RFC 1123, when clarifying these issues, makes
the generation of Return-Path a MUST requirement for the receiver-SMTP.

So, of the information that an mbox delimiter conveys, the sender
envelope address is contained in the Return-Path header (and I would be
willing to argue that a message lacking such a header, transferred via
POP3, is in fact broken) and the date is contained in the timestamp of
the last-added Received header.

In summary, an undelimited RFC 822 message containing a Return-Path
header has all the information you claim should be passed on to a POP3
client, and therefore isn't broken by the definition you used in this
message.

-- 
Christopher Davis * <ckd-sig@ckdhr.com> * <URL:http://www.ckdhr.com/ckd/>
Put location information in your DNS! <URL:http://www.ckdhr.com/dns-loc/>

Re: More POP madness

[Russ Allbery]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> A message delivered to an Internet host, even if delivered 'locally',
> regardless of its contents, is delivered by SMTP by defintion.  By
> definition it must have an envelope.  That envelope and the information
> it contains have become part of the message, information that is
> necessary for the proper handling of that message.  If that information
> is removed for any reason, the message can no longer be properly
> handled.

My MTA puts that information in Return-Path and Delivered-To RFC 822
headers.  It does not generate a "From " line, nor should it.

-- 
Russ Allbery (rra@stanford.edu)         <URL:http://www.eyrie.org/~eagle/>

Re: More POP madness

[Dan Christensen]

It seems to be a fact that many pop servers produce message which are
not in valid mbox format (e.g. without a From_ line or with unescaped
From_ lines in the message body and no Content-Length header).  It
seems to me that there are two questions here.

1) Is this behaviour "correct" or "broken"?

2) What should the retrieving program (fetchmail, pop3-movemail, etc) 
do when this happens?

I happen to believe the many people who have argued that the behaviour
is reasonable.  But I don't think that we are going to come to a
consensus on question 1.

However, I believe that we can answer question 2 regardless of our
views on question 1.  For example, suppose for argument's sake that we
believe that the behaviour in question is broken and that we are
writing a program that retrieves messages from a pop server and
(sometimes) writes the output to a file that is expected to be in mbox
format.  As disciplined programmers who believe in writing robust
software, we should ensure that our output is valid whenever it is
possible for us to do so.  In the case in question, we are perfectly
aware of the message boundaries, since we handle one message at a
time, and so it is reasonable for us to add phony From_ separators and
Content-Length lines or to escape From_ lines in the message body in
order to ensure that our output is usable.  We have the information
to do so, and we are the last agent who will have that ability.
Moreover, it is not difficult to do from the programmer's point of
view, it does not alter any input which we believe is correct, and
it will not disrupt any existing users in the catastrophic way that
an invalid mbox does.  

Rat's idea of adding Content-Length headers qualifies in my mind
as a good fix to the problem of unescaped From_ lines.  And adding
artificial From_ separators also seems like a reasonable behaviour
to me when they are needed.

Dan

-- 
Dan Christensen
jdc@math.jhu.edu

Re: More POP madness

[Chris Tessone]

>>>>> "Kai" == Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:

    Kai> Have I mentioned `endless argument' before? :-)

*nods* I doubt this argument will resolve much.

Getting back to the topic, any ideas as to why Gnus is dropping this
mail? Lars? Anyone?

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Christopher K Davis <ckd@ckdhr.com>  on Sun, 11 Apr 1999
| But then why are you accepting messages with MMDF delimiters?  Those
| don't contain the envelope information, and therefore are, by this
| definition, broken.

MMDF is a storage-side delimiter, information *ADDED* to a message during
transmission.  Adding information is okay.  Destructive removal of
information is what is not okay.  If a POP server spits out MMDF
delimiters, the POP client should be accept them.

| Besides, RFC 822 defines a location and format for the envelope sender--
| the Return-Path header.

Ignore RFC 822.  It has nothing to do with message handling.

| RFC 821 specifies that a receiver-SMTP is responsible for inserting a
| Return-Path.  RFC 1123, when clarifying these issues, makes the
| generation of Return-Path a MUST requirement for the receiver-SMTP.

But neither do away with the envelope around the entire message,
specifically because removal of information is a Bad Thing.  Besides, there
are many legacy systems still in operation that cannot be made to comply
with RFC 1123.  RFC 821 is still the least common denominator.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EVPEgl+vIlSVSNkRApBEAKDrhlpksn2NX6p5hRJJgTOsnhijIACfYhGV
Tsgey5y40+C2R5hA33f+8Xw=
=q4GI
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Do not taunt Happy Fun Ball.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Dan Christensen <jdc@chow.mat.jhu.edu>  on Sun, 11 Apr 1999
| 1) Is this behaviour "correct" or "broken"?

It is broken only if a) that information exists on the server, and b) it is
destructively removed -- that is, it is utterly impossible to regenerate
that information.

| 2) What should the retrieving program (fetchmail, pop3-movemail, etc)
| do when this happens?

'Be liberal in what they accept'.  Ie, deal with the situation as best they
can.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EVRugl+vIlSVSNkRAiqsAJ9o+vVrtM1qp24YoEsG2QtxNgBhtgCfcuw0
jVZd1anISK9BIElUUH3C6eM=
=PkWv
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Kai.Grossjohann@CS.Uni-Dortmund.DE  on Sun, 11 Apr 1999
| Hm.  Apparently, POP3 wasn't designed this way, else the writers of
| the RFC wouldn't have forbidden the envelopes to persist.  Hm.

One could (and will :) argue that POP was not designed.  It happened.

What I am harping about is something that it outside the context of
specific instances of standards, but a long-standing 'meta-standard'.  To
wit: you do not destroy information for any reason.  Re-code it, put it
somewhere else, whatever, as long as the transformation is obvious and
easilly reversible, but do not remove it entirely.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EVVygl+vIlSVSNkRAvD0AJ9XgNlXQdl7qlkHP4qZLxgTzX3/dACfdiaZ
4Z6mg5z6gHA/rPQMYBrt4SY=
=kJ29
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Do not use Happy Fun Ball on concrete.
Minion of Nathan - Nathan says Hi! \ 
PGP Key: at a key server near you!  \ 

Re: More POP madness

[Christopher K Davis]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> MMDF is a storage-side delimiter, information *ADDED* to a message
> during transmission.  Adding information is okay.  Destructive removal
> of information is what is not okay.  If a POP server spits out MMDF
> delimiters, the POP client should be accept them.

Are you now willing to accept that a delimiterless storage format on the 
server can exist, and that a delimiterless message transferred via POP3
is not a sign of a "broken" server?

> | Besides, RFC 822 defines a location and format for the envelope sender--
> | the Return-Path header.

> Ignore RFC 822.  It has nothing to do with message handling.

It has everything to do with the format of the message being transferred.
You are complaining that information is lost if an mbox delimiter isn't
sent over POP3.  I am pointing out that the information is *not* lost, and
that RFC 822 has a place for it to live that allows the information to be
kept with the message as POP3's RETR command sends it without having to
violate the spec or include any delimiter whatsoever.

> | RFC 821 specifies that a receiver-SMTP is responsible for inserting a
> | Return-Path.  RFC 1123, when clarifying these issues, makes the
> | generation of Return-Path a MUST requirement for the receiver-SMTP.

> But neither do away with the envelope around the entire message,
> specifically because removal of information is a Bad Thing.  Besides,
> there are many legacy systems still in operation that cannot be made
> to comply with RFC 1123.  RFC 821 is still the least common
> denominator.

Yes, and RFC 821 specifies the use of Return-Path to hold the sender
envelope information.  RFC 1123 (which is nearly 10 years old, for Ghu's 
sake) simply clarifies the level of requirement to a MUST--and, as I
said, I'm more than willing to consider a Return-Path-less message to be 
broken.

-- 
Christopher Davis * <ckd-sig@ckdhr.com> * <URL:http://www.ckdhr.com/ckd/>
Put location information in your DNS! <URL:http://www.ckdhr.com/dns-loc/>

Re: More POP madness

[Christopher K Davis]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

> What I am harping about is something that it outside the context of
> specific instances of standards, but a long-standing 'meta-standard'.  To
> wit: you do not destroy information for any reason.  Re-code it, put it
> somewhere else, whatever, as long as the transformation is obvious and
> easilly reversible, but do not remove it entirely.

You mean like changing

MAIL From:<foo@example.com>

and putting the important information into the message as

Return-Path: <foo@example.com>

for future use?

Thank you, we apparently agree.

-- 
Christopher Davis * <ckd-sig@ckdhr.com> * <URL:http://www.ckdhr.com/ckd/>
Put location information in your DNS! <URL:http://www.ckdhr.com/dns-loc/>

Re: More POP madness

[Lars Magne Ingebrigtsen]

Chris Tessone <tessone@fnal.gov> writes:

> Getting back to the topic, any ideas as to why Gnus is dropping this
> mail? Lars? Anyone?

Nope.  If the popper generates something that Gnus doesn't understand, 
then Gnus should complain.

Did `M-x nnmail-split-history' say anything?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: More POP madness

[Chris Tessone]

>>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

    Lars> Nope.  If the popper generates something that Gnus doesn't
    Lars> understand, then Gnus should complain.

    Lars> Did `M-x nnmail-split-history' say anything?

Bloody hell. I think perhaps the reason I did see the mail was because 
they were going into zombie groups. However, I'm used to Gnus making
new mail groups visible on restarting when a new mail group is
created. I made only a few minor modifications to my ~/.gnus.el, so
this was not really expected behaviour.

Note that this is only cursory testing, exclusively messages sent to
myself from myself. I've set gnus-subscribe-newsgroup-method to
gnus-subscribe-alphabetically, and we'll see if this helps. (Whereas
my Fermilab account has access to news, my IMSA one does not, so the
Gnus doing POP will only be subscribing mail groups and is not
connected to a news server.)

Chris

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Chris Tessone]

Okay, I was wrong. It wasn't just zombie groups. I received the
following e-mails, which I then pop'd to Gnus:

>From ph@postoffice.imsa.edu  Wed Apr 21 02:00:15 1999
>From ph@postoffice.imsa.edu  Wed Apr 21 03:00:47 1999
>From ph@imsa.edu  Wed Apr 21 04:00:25 1999
>From owner-perl-unicode@perl.org  Wed Apr 21 10:06:40 1999
>From owner-chicago-pm@Mailing-List.net  Wed Apr 21 10:47:58 1999
>From owner-chicago-pm@Mailing-List.net  Wed Apr 21 10:58:28 1999
>From owner-chicago-pm@Mailing-List.net  Wed Apr 21 11:05:35 1999

Only one of the mails (from ph@postoffice.imsa.edu) was put in the
proper group (mail.imsa.phadmin). The others were discarded by
Gnus. M-x nnmail-split-history returns:

mail.imsa.phadmin:346

And that's all.

Chris

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[François Pinard]

Chris Tessone <tessone@fnal.gov> writes:

> >>>>> "Lars" == Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

>     Lars> Nope.  If the popper generates something that Gnus doesn't
>     Lars> understand, then Gnus should complain.

>     Lars> Did `M-x nnmail-split-history' say anything?

> Bloody hell.  I think perhaps the reason I did see the mail was because
> they were going into zombie groups.

I will remember for a long time the difficulty I had to get things
working properly for me with Gnus.  Playing with `F' to see new newsgroups,
knowing to look at zombies, and also understanding that there is a possible
confusion between killed groups and non-existing groups, gave me a hard time.

It is much easier for me now (and I could not live without Gnus now :-), but
I keep thinking that the manual was not very helpful to me, as a newcomer,
at the beginning.  Some users might be just like me, and not very good at
guess work.  I still think the manual should explain things better for those
doing their initial setup and starting to get acquainted with the package.

-- 
François Pinard                            mailto:pinard@iro.umontreal.ca
Join the free Translation Project!    http://www.iro.umontreal.ca/~pinard

Re: More POP madness

[Kai.Grossjohann]

Chris Tessone <tessone@fnal.gov> writes:

  > Only one of the mails (from ph@postoffice.imsa.edu) was put in the
  > proper group (mail.imsa.phadmin). The others were discarded by
  > Gnus. M-x nnmail-split-history returns:

Do you have a catch-all entry at the end of your nnmail-split-methods?

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Chris Tessone]

>>>>> "Kai" == Kai Grossjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> writes:

    Kai> Do you have a catch-all entry at the end of your
    Kai> nnmail-split-methods?

Yes.

As I said, I have made no major changes to my .gnus.el from the one I
use on FNALU (the system from which I'm sending this mail), so I
didn't expect things to break.

Chris

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Kai.Grossjohann@CS.Uni-Dortmund.DE  on Sat, 10 Apr 1999
| The value of mail-source-crash-box is ~/.emacs-mail-crash-box, and M-x
| apropos-value RET gnus-crash-box RET does not return a variable.

Oops.  Another case of nnmail-foo or gnus-foo becoming mail-source-foo.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3EKdqgl+vIlSVSNkRAjQXAJ9CmSNTJ0OoFSMrn3yXARghSiYU0wCgi/7f
5x2TZH6zDQZ+j2seXPfVtJg=
=DgIL
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Happy Fun Ball contains a liquid core,
Minion of Nathan - Nathan says Hi! \ which, if exposed due to rupture, should
PGP Key: at a key server near you!  \ not be touched, inhaled, or looked at.

Re: More POP madness

[Kai.Grossjohann]

Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

  > * Chris Tessone <tessone@fnal.gov>  on Thu, 08 Apr 1999
  > | I don't see what difference it makes. That's what Gnus' POP3 stuff is
  > | chosing as the filename, and Gnus says that it's reading in the
  > | file...
  > 
  > Because if you copy to ~/.emacs-mail-crash-box, Gnus won't read it.  Gnus
  > uses ~/.gnus-crash-box.

The value of mail-source-crash-box is ~/.emacs-mail-crash-box, and M-x
apropos-value RET gnus-crash-box RET does not return a variable.

I'm willing to bet a beer against a coke (I don't drink beer) that
0.80 reads ~/.emacs-mail-crash-box.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Chris Tessone <tessone@fnal.gov>  on Thu, 08 Apr 1999
| I don't see what difference it makes. That's what Gnus' POP3 stuff is
| chosing as the filename, and Gnus says that it's reading in the
| file...

Because if you copy to ~/.emacs-mail-crash-box, Gnus won't read it.  Gnus
uses ~/.gnus-crash-box.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3DpkGgl+vIlSVSNkRAg1TAJ4hBzBxX9S1Vh1hTVO26j652jqSCACgiWJR
LAFqM/aft+WEmxZm9YfdPK0=
=/+0c
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ Ingredients of Happy Fun Ball include an
Minion of Nathan - Nathan says Hi! \ unknown glowing substance which fell to
PGP Key: at a key server near you!  \ Earth, presumably from outer space.

Re: More POP madness

[Kai.Grossjohann]

Chris Tessone <tessone@imsa.edu> writes:

  > Secondly, how can I retrieve the mail from the Incoming file if Gnus
  > *does* lose it? Making a new nndoc group didn't work --- it kept
  > giving me an error when I tried to open the new group.

If you can't `G f' the Incoming file, then the file is broken.  Have a
look at it (with C-x C-f) and see if you can salvage stuff.

Gnus should warn you, though.  But we don't know what's wrong with
that file, yet, so maybe it will help if you tell us what it looks
like and where you think the problem is.  (You might be able to find
the problem by splitting the file into two then using `G f' on both
files, and so on until you find the problem.)

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: More POP madness

[Dmitry Yaitskov]

Chris Tessone <tessone@imsa.edu> wrote:

> Okay, I have POP working now. Minor problem though: occasionally Gnus
> will POP my mail from the mail server and then claim I have no new
> mail when I'm positive I do --- that is, I've typed "from" on the
> mailserver, seen messages, typed 'g' in Gnus, then typed "from" on the 
> mailserver again and not seen messages. The Mail/Incoming... file is
> there and the mail's in it, but it's not being incorporated into
> groups.

I have a consistently repeatable situation that is similar somewhat. I
have several (actually, 4) different pop3 accounts that I read mail
from (using pop3.el but I don't think it matters here... dunno
though). And, new mail that has been received from the last account
(last one in the nnmail-spool-file) does not make it past the crashbox
when it first arrives. I.e. I see that x messages are being read from
account y, but it does not show up anywhere. Repeating 'g' though
'fixes' the situation - mail is put into its right place. The actual
account does not seem to matter, just the fact that it is listed last.

-- 
Cheers,
-Dima.

Re: More POP madness

[Chris Tessone]

>>>>> "Rat" == Stainless Steel Rat <ratinox@peorth.gweep.net> writes:

    Rat> Gnus normally uses `.gnus-crash-box' so try that instead.

I don't see what difference it makes. That's what Gnus' POP3 stuff is
chosing as the filename, and Gnus says that it's reading in the
file...

Chris

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Stainless Steel Rat]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

* Chris Tessone <tessone@fnal.gov>  on Thu, 08 Apr 1999
| Okay, I copied the Incoming file to ~/.emacs-mail-crash-box, Gnus said
| it was processing it, then the same thing happened. No new mail.

Gnus normally uses `.gnus-crash-box' so try that instead.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.5 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE3DT1qgl+vIlSVSNkRAq9iAJ4zFqaHwx9HQk6SP6Zs21v1ghJZpACfVS6o
zUuJbqXeMZQVEEKBOaa7p+w=
=S3CX
-----END PGP SIGNATURE-----

-- 
Rat <ratinox@peorth.gweep.net>    \ When not in use, Happy Fun Ball should be
Minion of Nathan - Nathan says Hi! \ returned to its special container and
PGP Key: at a key server near you!  \ kept under refrigeration.

Re: More POP madness

[Chris Tessone]

Okay, I copied the Incoming file to ~/.emacs-mail-crash-box, Gnus said 
it was processing it, then the same thing happened. No new mail.

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Chris Tessone]

>>>>> "Alexandre" == Alexandre Oliva <oliva@dcc.unicamp.br> writes:

    Alexandre> Maybe it was just a duplicate message, dropped by gnus?

There were eight messages in the mailbox, all of them ones I'd not
received before.

    Alexandre> Just copy the Incoming file to ~/.emacs-mail-crash-box
    Alexandre> (or whatever your mail-source-crash-box is set to) and
    Alexandre> type `g' again.

Okay, I'll try that.

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/

Re: More POP madness

[Alexandre Oliva]

On Apr  8, 1999, Chris Tessone <tessone@imsa.edu> wrote:

> mail when I'm positive I do --- that is, I've typed "from" on the
> mailserver, seen messages, typed 'g' in Gnus, then typed "from" on the 
> mailserver again and not seen messages.

Maybe it was just a duplicate message, dropped by gnus?

> Secondly, how can I retrieve the mail from the Incoming file if Gnus
> *does* lose it?

Just copy the Incoming file to ~/.emacs-mail-crash-box (or whatever
your mail-source-crash-box is set to) and type `g' again.

-- 
Alexandre Oliva http://www.dcc.unicamp.br/~oliva IC-Unicamp, Brasil
{oliva,Alexandre.Oliva}@dcc.unicamp.br  aoliva@{acm.org,computer.org}
oliva@{gnu.org,kaffe.org,{egcs,sourceware}.cygnus.com,samba.org}
*** E-mail about software projects will be forwarded to mailing lists

More POP madness

[Chris Tessone]

Okay, I have POP working now. Minor problem though: occasionally Gnus
will POP my mail from the mail server and then claim I have no new
mail when I'm positive I do --- that is, I've typed "from" on the
mailserver, seen messages, typed 'g' in Gnus, then typed "from" on the 
mailserver again and not seen messages. The Mail/Incoming... file is
there and the mail's in it, but it's not being incorporated into
groups.

Secondly, how can I retrieve the mail from the Incoming file if Gnus
*does* lose it? Making a new nndoc group didn't work --- it kept
giving me an error when I tried to open the new group.

Chris

-- 
Chris Tessone              tessone@imsa.edu              tessone@fnal.gov
PH/Postmaster SNPC                                  Voice: (630) 907-5547
Student Network System Engineer         Illinois Math and Science Academy
http://www.imsa.edu/~tessone/              http://home.fnal.gov/~tessone/