"From " -> ">From " in imap mail-source avoidable?

"From " -> ">From " in imap mail-source avoidable?

[Helmut Waitzmann]

I sent two messages to me, each of them contained a line
beginning with the word "From".

The first message I read with the nnimap-backend, and everything
was fine.

Then I respooled it into the nnml-archive ('M-x
gnus-summary-respool-article') and everthing was fine again:


   From: Helmut Waitzmann <Helmut.Waitzmann@web.de>
   Subject: "From" am Anfang einer Zeile kritisch?
   To: Helmut Waitzmann <Helmut.Waitzmann@web.de>
   Date: 16 Jul 2002 19:15:41 +0200
   Xref: meine.maschine.example.com mail.private.misc:208
   Return-Path: <Helmut.Waitzmann@web.de>
   X-Sieve: cmu-sieve 2.0
   Return-path: <Helmut.Waitzmann@web.de>
   Envelope-to: MeineZweiteEmailAdresse@example.com
   User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2
   MIME-Version: 1.0
   Content-Type: text/plain; charset=iso-8859-1
   Content-Transfer-Encoding: quoted-printable

   Was geschieht mit folgender Zeile, wenn ich die Nachricht mit dem
   nnimap-Backend lese und mittels M-x gnus-summary-respool-article
   anschließend ins nnml-backend transportiere?

   From mir

The second message was grabbed by the imap mail-source specifier
from the imap server

   (list
    'imap
    ':server "mein.imap.server"
    ':mailbox "INBOX"
    ':predicate "UNDELETED"
    ':fetchflag "\\Deleted"
    ':dontexpunge nil
    )

and put into the nnml-archive:

   From: Helmut Waitzmann <Helmut.Waitzmann@web.de>
   X-From-Line: imap Tue Jul 16 19:20:34 2002
   Subject: "From" am Anfang einer Zeile kritisch?
   To: Helmut Waitzmann <Helmut.Waitzmann@web.de>
   Date: 16 Jul 2002 19:19:25 +0200
   Xref: meine.maschine.example.com mail.private.misc:209
   Return-Path: <Helmut.Waitzmann@web.de>
   X-Sieve: cmu-sieve 2.0
   Return-path: <Helmut.Waitzmann@web.de>
   Envelope-to: MeineZweiteEmailAdresse@example.com
   User-Agent: Gnus/5.0808 (Gnus v5.8.8) Emacs/21.2
   X-Gnus-Mail-Source: imap:imap.example.com:INBOX
   MIME-Version: 1.0
   Content-Type: text/plain; charset=us-ascii

   Was geschieht mit folgender Zeile, wenn ich die Nachricht mit dem
   imap-mail-source ins nnml-Backend hole?

   >From mir

And as you can see, the last line has been mangled.  I guess,
that the mail fetching with mail-sources is the culprit:  nnimap
as well as nnml let the message intact, as can be seen from the
first message.  Is there a way to avoid this?  It looks like
mail-sources first puts all messages into a unix mailbox file.
But as I understand, that shouldn't be necessary, as can be seen
from the first example.

Re: "From " -> ">From " in imap mail-source avoidable?

[Kai Großjohann]

Helmut Waitzmann <Helmut.Waitzmann@web.de> writes:

> And as you can see, the last line has been mangled.  I guess,
> that the mail fetching with mail-sources is the culprit:  nnimap
> as well as nnml let the message intact, as can be seen from the
> first message.  Is there a way to avoid this?  It looks like
> mail-sources first puts all messages into a unix mailbox file.
> But as I understand, that shouldn't be necessary, as can be seen
> from the first example.

Maybe mail-sources could be changed to use a Babyl file or an MMDF
file instead of an mbox file.

What do people think?

It could be an option.

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: "From " -> ">From " in imap mail-source avoidable?

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> Maybe mail-sources could be changed to use a Babyl file or an MMDF
> file instead of an mbox file.

I don't know the format of those files, but what are our constraints?
Must multiple messages be represented in a single file?  Can we find
the size of a whole message before adding it to the file?

> It could be an option.

If a format like "1234\nmessage-consisting-of-1234-bytes\n..." is
possible, I don't see why we'd want to use anything else.


paul

Re: "From " -> ">From " in imap mail-source avoidable?

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
>> Maybe mail-sources could be changed to use a Babyl file or an MMDF
>> file instead of an mbox file.
>
> I don't know the format of those files, but what are our
> constraints?  Must multiple messages be represented in a single
> file?

I guess that a new function along the lines of
nnmail-process-*-mail-format could easily be written.  In fact, now I
see that there is nnmail-process-maildir-mail-format which kind of
suggests that it needn't be a single file.

> Can we find the size of a whole message before adding it to the
> file?

Ah, I think the mbox format currently uses a Content-length or Lines
header, if present.  So it could just be added to the file by imap.
Cool.

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: "From " -> ">From " in imap mail-source avoidable?

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> I guess that a new function along the lines of
> nnmail-process-*-mail-format could easily be written.  In fact, now I
> see that there is nnmail-process-maildir-mail-format which kind of
> suggests that it needn't be a single file.

The *source* needn't be a single file, but then all messages are
dumped into a single file, and FAIK, there may be a good reason for
this.

>> Can we find the size of a whole message before adding it to the
>> file?
>
> Ah, I think the mbox format currently uses a Content-length or Lines
> header, if present.  So it could just be added to the file by imap.
> Cool.

I'd like to avoid destructive modification of the message, if
possible.  But then, I don't personally use this code anyway.


paul

Re: "From " -> ">From " in imap mail-source avoidable?

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
>> I guess that a new function along the lines of
>> nnmail-process-*-mail-format could easily be written.  In fact, now I
>> see that there is nnmail-process-maildir-mail-format which kind of
>> suggests that it needn't be a single file.
>
> The *source* needn't be a single file, but then all messages are
> dumped into a single file, and FAIK, there may be a good reason for
> this.

Well, but movemail.c writes Babyl format when reading from a POP
server.  So _at least_ it would work to change mail-sources to write
Babyl format.

But I don't understand why it would be a problem to write maildir
format, instead, if nnmail-process-maildir-mail-format groks that.

>>> Can we find the size of a whole message before adding it to the
>>> file?
>>
>> Ah, I think the mbox format currently uses a Content-length or Lines
>> header, if present.  So it could just be added to the file by imap.
>> Cool.
>
> I'd like to avoid destructive modification of the message, if
> possible.  But then, I don't personally use this code anyway.

Well, if you have the choice between replacing "From" lines with
">From" lines in the message body (now) or adding a Lines or
Content-length header (my suggestion) -- what's worse?

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: "From " -> ">From " in imap mail-source avoidable?

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> But I don't understand why it would be a problem to write maildir
> format, instead, if nnmail-process-maildir-mail-format groks that.

Actually, what I don't get is why when you're using a maildir
mail-source Gnus bothers to copy it to another spool file at all.
Couldn't Gnus read the messages from the Maildir directly?  (doesn't
need to be locked, etc)

> Well, if you have the choice between replacing "From" lines with
> ">From" lines in the message body (now) or adding a Lines or
> Content-length header (my suggestion) -- what's worse?

Agreed, but I think in the case of Maildir, we don't even need an
intermediate storage, right?

-- 
Josh Huber

Re: "From " -> ">From " in imap mail-source avoidable?

[Kai Großjohann]

Josh Huber <huber+dated+1027863384.faf1e4@alum.wpi.edu> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
>
>> But I don't understand why it would be a problem to write maildir
>> format, instead, if nnmail-process-maildir-mail-format groks that.
>
> Actually, what I don't get is why when you're using a maildir
> mail-source Gnus bothers to copy it to another spool file at all.
> Couldn't Gnus read the messages from the Maildir directly?  (doesn't
> need to be locked, etc)

Well, Gnus also comes with nnmaildir, which does what you want.  So
the user has the choice.

Maybe some users want to store messages in nnfolder but the MTA
delivers to a maildir.

>> Well, if you have the choice between replacing "From" lines with
>> ">From" lines in the message body (now) or adding a Lines or
>> Content-length header (my suggestion) -- what's worse?
>
> Agreed, but I think in the case of Maildir, we don't even need an
> intermediate storage, right?

Yes and no.  mail-sources always needs an intermediate storage: it
copies the messages from the spots specified in mail-sources to temp
files, then from there into the groups.  This avoids problems with
locking.  As mail-sources works for many types of sources, it uses
the same mechanism always, to avoid the locking problems.

kai
-- 
A large number of young women don't trust men with beards.  (BFBS Radio)

Re: "From " -> ">From " in imap mail-source avoidable?

[Josh Huber]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Well, Gnus also comes with nnmaildir, which does what you want.  So
> the user has the choice.
>
> Maybe some users want to store messages in nnfolder but the MTA
> delivers to a maildir.

Well, I deliver into a maildir from my MDA, and want to use it as a
mail-source in Gnus since I like splitting with Gnus.

> Yes and no.  mail-sources always needs an intermediate storage: it
> copies the messages from the spots specified in mail-sources to temp
> files, then from there into the groups.  This avoids problems with
> locking.  As mail-sources works for many types of sources, it uses
> the same mechanism always, to avoid the locking problems.

Right, okay.  Consistancy is good, but I wonder how hard it would be
to skip that part with maildirs? (since you don't need the locking
required for other mail-sources)

I guess I wouldn't mind as much if a format other than mbox was used
for temporary storage.  Those ">From" lines really irk me :)

ttyl,

-- 
Josh Huber

Re: "From " -> ">From " in imap mail-source avoidable?

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> But I don't understand why it would be a problem to write maildir
> format, instead, if nnmail-process-maildir-mail-format groks that.

I don't think it would be a problem; even if there weren't already
n-p-m-m-f, something like it could be written.

> Well, if you have the choice between replacing "From" lines with
> ">From" lines in the message body (now) or adding a Lines or
> Content-length header (my suggestion) -- what's worse?

I'd say From mangling is worse, but avoiding header mangling as well
would be better still, IMO.


paul