Unix mbox format

Unix mbox format

[Lars Magne Ingebrigtsen]

Septembet 0.44 is going to be stricter on what it considers delimiters
in Unix mbox files.  It still uses the `rmail-unix-mail-delimiter' as
the delimiter regexp, but it also takes a peek at the next line.  The
next line has to start with "[^ :]+:" for Gnus to consider it a real
delimiter.  (It's reasonable to assume that a header line should
follow the delimiter.)

Is this a bad idea?  A good idea?  Should we use a different regexp to
match header lines?

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Unix mbox format

[michael lamoureux]

 "lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

lars> The next line has to start with "[^ :]+:" for Gnus to consider
lars> it a real delimiter.  (It's reasonable to assume that a header
lars> line should follow the delimiter.)

Well, I think the next line is the From line (the one without the ":")

Here's a brief clip from my VM INBOX file (I prepended a space to all
of the lines, for paranoia's sake).

...
 The release is just one short month away, and it should be as bug free
 as possible.  
  
 -- 
   "Yes.  The journey through the human heart 
      would have to wait until some other time."
  
 From ding-request@ifi.uio.no Mon Feb 26 07:57:36 1996
 Received: from srvr7.engin.umich.edu (root@srvr7.engin.umich.edu [141.212.2.69]) by srvr5.engin.umich.edu (8.7.4/8.7.3) with ESMTP id HAA17221 for <lamour@srvr5.engin.umich.edu>; Mon, 26 Feb 1996 07:57:35 -0500 (EST)
 Received: from ifi.uio.no (ifi.uio.no [129.240.64.2]) by srvr7.engin.umich.edu (8.7.4/8.7.3) with SMTP id HAA01287 for <lamour@engin.umich.edu>; Mon, 26 Feb 1996 07:56:25 -0500 (EST)
...


lars> Is this a bad idea?  A good idea?  Should we use a different
lars> regexp to match header lines?

Probably.


fyi,
Michael

Re: Unix mbox format

[Jason L Tibbitts III]

>>>>> "ml" == michael lamoureux <lamour@engin.umich.edu> writes:

ml>  "lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:
lars> The next line has to start with "[^ :]+:" for Gnus to consider it a
lars> real delimiter.  (It's reasonable to assume that a header line should
lars> follow the delimiter.)

ml> Well, I think the next line is the From line (the one without the ":")

No, the delimiter itself is the From_ line.  (Look at
rmail-unix-mail-delimiter.)  The next line should (must?) be a header line,
and so should include a colon.

This may or may not be unrelated, but SGnus currently doesn't properly
handle the Content-Length: header; I use Solaris and do not escape lines
starting with From.  Instead I rely on the Content-Length: header.  MH
(patched to deal with C-L: headers) will properly split my inbox but SGnus
will not.  This is really, really annoying when dealing with bounces.  I've
tried to track down the problem but have had no luck.  Has anyone else seen
this?

 - J<

Re: Unix mbox format

[Lars Magne Ingebrigtsen]

michael lamoureux <lamour@engin.umich.edu> writes:

> lars> The next line has to start with "[^ :]+:" for Gnus to consider
> lars> it a real delimiter.  (It's reasonable to assume that a header
> lars> line should follow the delimiter.)
> 
> Well, I think the next line is the From line (the one without the ":")

No, the delimiter *is* the "From " line.  :-)  

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."

Re: Unix mbox format

[michael lamoureux]

 "lars" == Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

lars> No, the delimiter *is* the "From " line.  :-)  

Wow...where was my coffee mug that day?


duh,
Michael

Re: Unix mbox format

[Hallvard B Furuseth]

> Septembet 0.44 is going to be stricter on what it considers delimiters
> in Unix mbox files.  It still uses the `rmail-unix-mail-delimiter' as
> the delimiter regexp, but it also takes a peek at the next line.  The
> next line has to start with "[^ :]+:"

Hrrm.  That's multi-line.  Try "[^\n :]+:".  Or maybe "[^\n :]+ *:",
which is wrong by rfc822 but wouldn't surprise me anyway.

> for Gnus to consider it a real delimiter.  (It's reasonable to assume
> that a header line should follow the delimiter.)
> 
> Is this a bad idea?  A good idea?  Should we use a different regexp to
> match header lines?

Just be sure you are not too restrictive.  It's better to split some
letters that contain "From " in the body than concatenate two letters.
If the first letter is long and uninteresting, the user won't even see
the second.  For example, you are sure the buffer has never been
narrowed to contain the "From " line but not the following header line?


Regards,

Hallvard

Re: Unix mbox format

[Lars Magne Ingebrigtsen]

Hallvard B Furuseth <h.b.furuseth@usit.uio.no> writes:

> Hrrm.  That's multi-line.  Try "[^\n :]+:".  Or maybe "[^\n :]+ *:",
> which is wrong by rfc822 but wouldn't surprise me anyway.

I'm currently using "[^\000-\037\177-\377 :]+:", which is what rfc822
says is a legal header name.  Perhaps that's too strict, though.
Maybe "[^\n :]+ *:" might be a better choice...  Yes, I think I'll go
with that instead.

> Just be sure you are not too restrictive.  It's better to split some
> letters that contain "From " in the body than concatenate two letters.
> If the first letter is long and uninteresting, the user won't even see
> the second.

Yes, it's much better to split a bit too much than incorrectly append
one mail to another even once.

-- 
  "Yes.  The journey through the human heart 
     would have to wait until some other time."