procmail and nnfolder

procmail and nnfolder

[Jason R. Mastaler]

I'm still trying to find the "right" relationship between Gnus and
procmail.  I currently use procmail to shuffle my incoming mail info
different folders, one mail folder for each mailing list.  So the ding
list goes into ~/Mail/lists/ding for example.  I use Gnus to read
these lists with nnfolder as the backend.  So, my group for the ding
list is 'nnfolder:lists.ding' .  Reading through the Gnus manual today
I noticed in the "Mail and Procmail" section:

  If you use nnfolder or any other backend that store more than a
  single article in each file, you should never have procmail add mails
  to the file that Gnus sees. Instead, procmail should put all
  incoming mail in nnmail-procmail-directory. To arrive at the file
  name to put the incoming mail in, append nnmail-procmail-suffix to
  the group name. The mail backends will read the mail from these
  files. 

So does this mean I've been doing what the manual says not to?  (It
looks like it)  If so, why is this bad?  I've never noticed any side
effects from this combination.  I'm also a bit confused with at the
proposed solution of having procmail filter incoming mail in a 
nnmail-procmail-directory instead.

Here is the included example:

  (setq nnfolder-directory "~/fMail/")
  (setq nnmail-spool-file 'procmail)
  (setq nnmail-procmail-directory "~/incoming/")
  (setq gnus-secondary-select-methods '((nnfolder "")))
  (setq nnmail-procmail-suffix "")

Why are nnfolder-directory and nnmail-procmail-directory two different
directories if nnmail-procmail-directory will contain the mail boxes
that procmail appends to and nnfolder-directory is supposed to be
"All the nnfolder mail boxes will be stored under this directory"?

Is the idea to have Gnus use nnmail-procmail-directory as a temporary
directory that it draws from to process and then deposit nnfolder
mailboxes in the nnfolder-directory?  In this case, I guess I would
have procmail filter like this (for example):

ding list --> ~/incoming/lists/ding
bind list --> ~/incoming/lists/bind

Then, if I define (setq nnfolder-directory "~/Mail/lists/") + 
(setq nnmail-procmail-directory "~/incoming/lists/") would Gnus
automagically know that the group 'nnfolder:lists.ding' should be
drawn from ~/incoming/lists/ding and stored as ~/Mail/lists/ding ?

Thanks,

-- 
Jason R. Mastaler                                 jason@mastaler.com

Re: procmail and nnfolder

[Jason L Tibbitts III]

>>>>> "JRM" == Jason R Mastaler <jason@mastaler.com> writes:

JRM> So does this mean I've been doing what the manual says not to?

Yes.

JRM> If so, why is this bad?  I've never noticed any side effects from this
JRM> combination.

You will eventually lose mail.  You may have already.  The problem is that
Gnus expects that it owns its folders and that nothing is messing with them
in the background.  If Procmail adds a message while Gnus is processing a
folder in memory, then Gnus saves its buffer out to disk, you have just
lost mail and you'll never know about it.

JRM> I'm also a bit confused with at the proposed solution of having
JRM> procmail filter incoming mail in a nnmail-procmail-directory instead.

You have Procmail stuff mail in spool files, pre-sorted and filtered.  Gnus
then picks these up and stuff the messages in the appropriate groups.  Gnus
uses movemail to actually move the mail out of the spool, and movemail uses
locking that Procmail understands, so there is no danger of mail loss.

JRM> Why are nnfolder-directory and nnmail-procmail-directory two different
JRM> directories if nnmail-procmail-directory will contain the mail boxes
JRM> that procmail appends to and nnfolder-directory is supposed to be
JRM> "All the nnfolder mail boxes will be stored under this directory"?

Because Procmail should stuff its mail in different folders, _not_ in the
ones that your regular mail is stored in.

JRM> Is the idea to have Gnus use nnmail-procmail-directory as a temporary
JRM> directory that it draws from to process and then deposit nnfolder
JRM> mailboxes in the nnfolder-directory?

Yep.

 - J<

Re: procmail and nnfolder

[Jason R. Mastaler]

Jason L Tibbitts III <tibbs@hpc.uh.edu> writes:

> You have Procmail stuff mail in spool files, pre-sorted and filtered.  Gnus
> then picks these up and stuff the messages in the appropriate groups.  Gnus
> uses movemail to actually move the mail out of the spool, and movemail uses
> locking that Procmail understands, so there is no danger of mail loss.

Ok, thanks for clearing this up.  However, this isn't working for
me.  When I fire up Gnus, it isn't moving any mail over to my 
'nnfolder-directory'.  Here is what I added to .gnus

  (setq nnmail-use-procmail t)
  (setq nnfolder-directory "~/gMail/")
  (setq nnmail-spool-file 'procmail)
  (setq nnmail-procmail-directory "~/incoming/lists/")
  (setq gnus-secondary-select-methods '((nnfolder "")))
  (setq nnmail-procmail-suffix "")

Procmail is adding incoming mail to ~/incoming/lists/listname.
The nnfolder groups I subscribed to are named "nnfolder:lists.listname"
Gnus does create the ~/gMail/lists directory with a zero lengh file
in this directory for each list, but doesn't move any mail over and
so it thinks I have "No more unread newsgroups".

I'm using Gnus v5.4.40/XEmacs 19.15.  What am I missing here?   

-- 
Jason R. Mastaler                                 jason@mastaler.com

Re: procmail and nnfolder

[Lars Magne Ingebrigtsen]

"Jason R. Mastaler" <jason@mastaler.com> writes:

> Gnus does create the ~/gMail/lists directory with a zero lengh file
> in this directory for each list, but doesn't move any mail over and
> so it thinks I have "No more unread newsgroups".

What does

(nnmail-get-spool-files)

and

(nnmail-get-spool-files "some.group")

where "some.group" is a valid group, say?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: procmail and nnfolder

[Jason R. Mastaler]

> What does
> 
> (nnmail-get-spool-files)
> 
> and
> 
> (nnmail-get-spool-files "some.group")
> 
> where "some.group" is a valid group, say?

I'm not sure what you mean.  Doing an M-x nnmail-get-spool-files
returns "no match", and I can't find mention of this anywhere
in the documentation.

Could someone clarify?

Re: procmail and nnfolder

[Lars Magne Ingebrigtsen]

"Jason R. Mastaler" <jason@mastaler.com> writes:

> > What does
> > 
> > (nnmail-get-spool-files)
> > 
> > and
> > 
> > (nnmail-get-spool-files "some.group")
> > 
> > where "some.group" is a valid group, say?
> 
> I'm not sure what you mean.  Doing an M-x nnmail-get-spool-files
> returns "no match", and I can't find mention of this anywhere
> in the documentation.

Eval the forms by typing `C-x C-e' after the closing paren.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: procmail and nnfolder

[Lars Magne Ingebrigtsen]

jason@mastaler.com (Jason R. Mastaler) writes:

> > I think nnmail expects the spool files to be called
> > "~/incoming/lists.whatever", not "~/incoming/lists/whatever".
> 
> But I set:
> 
> (setq nnmail-procmail-directory "~/incoming/lists/")

I thought you said the groups were called "lists.whatever"?  So the
spool files were called "~/incoming/lists/lists.whatever.spool", then?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: procmail and nnfolder

[Jason R. Mastaler]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> "Jason R. Mastaler" <jason@mastaler.com> writes:
> 
> > After much experimentation, I finally got movemail to work.  I changed
> > nnfolder-directory to "~/gMail/lists/" and Gnus now moves mail from
> > "~/incoming/lists/" to corresponding groups in "~/gMail/".  My problem
> > seems to be solved, but still these workings seem counter-intuitive 
> > to me.  By what the manual has to say about 'nnfolder-directory' I
> > would think Gnus should build the nnfolder groups in "~/gMail/lists/" 
> > instead given my definitions.
> 
> I think nnmail expects the spool files to be called
> "~/incoming/lists.whatever", not "~/incoming/lists/whatever".

But I set:

(setq nnmail-procmail-directory "~/incoming/lists/")

so why would nnmail be looking in "~/incoming/" for spool files?

-- 
Jason R. Mastaler                                 jason@mastaler.com

Re: procmail and nnfolder

[Lars Magne Ingebrigtsen]

"Jason R. Mastaler" <jason@mastaler.com> writes:

> After much experimentation, I finally got movemail to work.  I changed
> nnfolder-directory to "~/gMail/lists/" and Gnus now moves mail from
> "~/incoming/lists/" to corresponding groups in "~/gMail/".  My problem
> seems to be solved, but still these workings seem counter-intuitive 
> to me.  By what the manual has to say about 'nnfolder-directory' I
> would think Gnus should build the nnfolder groups in "~/gMail/lists/" 
> instead given my definitions.

I think nnmail expects the spool files to be called
"~/incoming/lists.whatever", not "~/incoming/lists/whatever".

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: procmail and nnfolder

[Jason R. Mastaler]

"Jason R. Mastaler" <jason@mastaler.com> writes:

> > Ok, thanks for clearing this up.  However, this isn't working for
> > me.  When I fire up Gnus, it isn't moving any mail over to my 
> > 'nnfolder-directory'.  Here is what I added to .gnus
> > 
> >   (setq nnmail-use-procmail t)
> >   (setq nnfolder-directory "~/gMail/")
> >   (setq nnmail-spool-file 'procmail)
> >   (setq nnmail-procmail-directory "~/incoming/lists/")
> >   (setq gnus-secondary-select-methods '((nnfolder "")))
> >   (setq nnmail-procmail-suffix "")
> > 
> > Procmail is adding incoming mail to ~/incoming/lists/listname.
> > The nnfolder groups I subscribed to are named "nnfolder:lists.listname"
> > Gnus does create the ~/gMail/lists directory with a zero lengh file
> > in this directory for each list, but doesn't move any mail over and
> > so it thinks I have "No more unread newsgroups". 
> 
> After much experimentation, I finally got movemail to work.  I changed
> nnfolder-directory to "~/gMail/lists/" and Gnus now moves mail from
> "~/incoming/lists/" to corresponding groups in "~/gMail/".  My problem
> seems to be solved, but still these workings seem counter-intuitive 
> to me.  By what the manual has to say about 'nnfolder-directory' I
> would think Gnus should build the nnfolder groups in "~/gMail/lists/" 
> instead given my definitions.

I just upgraded to Gnus 5.4.42 from 5.4.40, and without having changed
anything, Gnus now *does* moves mail to "~/gMail/lists/" instead of
"~/gMail/", but *also* creates a "~/gMail/lists/lists/" directory in
addition containing one zero length file for each nnfolder group.

Now, I'm quite confused.

--
Jason R. Mastaler                                 jason@mastaler.com

Re: procmail and nnfolder

[Jason R. Mastaler]

> Ok, thanks for clearing this up.  However, this isn't working for
> me.  When I fire up Gnus, it isn't moving any mail over to my 
> 'nnfolder-directory'.  Here is what I added to .gnus
> 
>   (setq nnmail-use-procmail t)
>   (setq nnfolder-directory "~/gMail/")
>   (setq nnmail-spool-file 'procmail)
>   (setq nnmail-procmail-directory "~/incoming/lists/")
>   (setq gnus-secondary-select-methods '((nnfolder "")))
>   (setq nnmail-procmail-suffix "")
> 
> Procmail is adding incoming mail to ~/incoming/lists/listname.
> The nnfolder groups I subscribed to are named "nnfolder:lists.listname"
> Gnus does create the ~/gMail/lists directory with a zero lengh file
> in this directory for each list, but doesn't move any mail over and
> so it thinks I have "No more unread newsgroups". 

After much experimentation, I finally got movemail to work.  I changed
nnfolder-directory to "~/gMail/lists/" and Gnus now moves mail from
"~/incoming/lists/" to corresponding groups in "~/gMail/".  My problem
seems to be solved, but still these workings seem counter-intuitive 
to me.  By what the manual has to say about 'nnfolder-directory' I
would think Gnus should build the nnfolder groups in "~/gMail/lists/" 
instead given my definitions.

-- 
Jason R. Mastaler                                 jason@mastaler.com