And now: mail spool fetching makeover

And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Well, since nobody seems to have comments on the disappearance of
gnus-article-display-hook, I take it that that particular makeover
went without any problems.  :-)

The major important outstanding issue now is how to edit MIME
messages.  (I.e., `e' in the summary buffer.)  But since I haven't
really had any ideas how to do that, I'm thinking of doing the mail
spool makeover thing.

But I forget what it was that we planned, so let me ruminate a bit
while I try to recall what it was all about.

1) We don't like the fact that the first mail backend that asks for
mail gets all the mail.  Instead of the mail backends requesting mail, 
we want the mail to be despatched to the backends that wants it.

Er.

2) We want a more flexible way of specifying mail sources.

(pop :server "some.mail.server" :user "name" :password "geheimnis"
     :port 666 :gobbledygook t)

That is, a "mail source" is a list where the car is the mail source
type, and then a bunch of source type-specific keywords follow.
Defaults will exist for non-specified keywords.

Source types would include `file', `directory', `pop', `qmail' and
`group'.

Hm.  This is all about how to find the mail; it says nothing about the 
format of the mail.  Now the format is auto-recognized, but is that
sloppy?  (Unix mbox, babyl and MMDF are recognized.)  For instance, a
directory may contain files of many different formats...

Examples of mail source specifiers:

(file :path "/usr/spool/larsi")

(directory :path "/home/larsi/procmailspool" :matching "\\.spool$")

(group :name "nnmh:incoming-mail")

Er.  How are users supposed to specify which mail sources should be
despatched to which mail backends?  Perhaps the mail sources should be 
specified in the virtual servers?

(nnml ""
 (sources ((file :path "/usr/spool/larsi")
	   (directory :path "/home/larsi/procmailspool"
		       :matching "\\.spool$"))))
Hm.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@ifi.uio.no * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Frank D. Cringle]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:
> Source types would include `file', `directory', `pop', `qmail' and
> `group'.

Minor point: I would suggest using the keyword `maildir' rather than
`qmail'.  Although maildirs are a qmail innovation, qmail can also
deliver to mailboxes and in future other MTAs may offer maildirs.

-- 
Frank Cringle,      fdc@cliwe.ping.de
voice: (+49 2304) 467101; fax: 943357

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

fdc@cliwe.ping.de (Frank D. Cringle) writes:

> Minor point: I would suggest using the keyword `maildir' rather than
> `qmail'.

Fixed.

I've now written most of the code for doing this, er, thing.  (I
haven't added maildir support yet.  I think I'll let someone who uses
maildir do that.)

How should the switchover be done?  Er...  A clean break would be
nice, but on the other hand, annoying people who have a working setup
is, well, annoying.

But so is duplicated code.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[William M. Perry]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> fdc@cliwe.ping.de (Frank D. Cringle) writes:
> 
> > Minor point: I would suggest using the keyword `maildir' rather than
> > `qmail'.
> 
> Fixed.
> 
> I've now written most of the code for doing this, er, thing.  (I
> haven't added maildir support yet.  I think I'll let someone who uses
> maildir do that.)
> 
> How should the switchover be done?  Er...  A clean break would be
> nice, but on the other hand, annoying people who have a working setup
> is, well, annoying.
> 
> But so is duplicated code.

Could we have some way to split the mail to different backends as well?  I
would love to be able to split some of the mail off of an imap server into
local nnml groups (I like to keep all my emacs and gnu related stuff on my
personal machine) but leave the bulk on the imap server.

-Bill P.

Re: And now: mail spool fetching makeover

[Kai.Grossjohann]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

  > (nnml ""
  >  (sources ((file :path "/usr/spool/larsi")
  > 	   (directory :path "/home/larsi/procmailspool"
  > 		       :matching "\\.spool$"))))

Taking into account what Bill said, it seems that we want to have it
the other way round: for each source, specify the backends that mail
should go to, rather than vice versa.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: And now: mail spool fetching makeover

[Simon Josefsson]

Lars Magne Ingebrigtsen <larsi@ifi.uio.no> writes:

> Er.  How are users supposed to specify which mail sources should be
> despatched to which mail backends?  Perhaps the mail sources should be 
> specified in the virtual servers?
> 
> (nnml ""
>  (sources ((file :path "/usr/spool/larsi")
> 	   (directory :path "/home/larsi/procmailspool"
> 		       :matching "\\.spool$"))))

Couldn't this dispatching be done by means of the splitting code?  Ie,
all mail sources plonk there mail into the splitting code where it is
decided in which backend it should end up in, and to what group in
that backend. This would also give us a backend-independent
mail-splitting code.

Something like:

(setq gnus-mail-sources '((pop :server "some.pop.server")
                          (group :name "nnimap+pdc:INBOX")))

(setq gnus-split-rules '(("nnml:private"         "To: jas@pdc.kth.se")
			 ("nnml:sendfail"        "MAILER-DAEMON")
			 ("nnfolder:sent-mail"   "From: jas@pdc.kth.se")
			 ("nnimap+pdc:lists.\1"  "owner-([a-z]+)")))

I see some obstacles, but the scheme as such doesn't feel extremely
complex which could be a good thing.

	* what to do if nothing matches?
	* how to handle crossposting?
	* what about fancy splitting?  (haven't used it)

Sorry if I'm completely missing the point and overlooking some major
issue..

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

wmperry@aventail.com (William M. Perry) writes:

> Could we have some way to split the mail to different backends as well?  I
> would love to be able to split some of the mail off of an imap server into
> local nnml groups (I like to keep all my emacs and gnu related stuff on my
> personal machine) but leave the bulk on the imap server.

Yes, that should be possible.  Although I haven't written the code to
use groups as the basis of splitting yet.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Simon Josefsson <jas@pdc.kth.se> writes:

> (setq gnus-split-rules '(("nnml:private"         "To: jas@pdc.kth.se")
> 			 ("nnml:sendfail"        "MAILER-DAEMON")
> 			 ("nnfolder:sent-mail"   "From: jas@pdc.kth.se")
> 			 ("nnimap+pdc:lists.\1"  "owner-([a-z]+)")))

Hm.  Hm.

Well, implementing this would require lots of work, but it does have a 
certain jennesequa.

But it's a bit vague to me what people would want to have done.  Is it 
likely that they want to say

1) I want all the mail from this source to be split onto this mail
server, and all mail from that sourcce to be split onto that mail
server, using the same split methods, or

2) I want all mail from all source to be split onto all these mail
servers.

In the latter scheme, there is no way to say "I want all mail from
MAILER-DAEMON that comes from pop to be split into nnml:sendfail,
while all mail from MAILER-DAEMON that comes from imap to be split
into nnfolder:sendfail".  While in the former it's impossible to say
"I want all mail from MAILER-DAEMON from all sources to be split into
nnml:sendfail and all mail from larsi from all source to be split into 
nnfolder:sendfail".

Hm.  Perhaps the split source specifiers should be named so that they
can be used in the split rules?

(setq gnus-split-rules
      '(("nnml:private"         "To: jas@pdc.kth.se" "imap-source")
	("nnfolder:private"     "To: jas@pdc.kth.se" "pop-source")
	("nnml:sendfail"        "MAILER-DAEMON")
	("nnfolder:sent-mail"   "From: jas@pdc.kth.se")
	("nnimap+pdc:lists.\1"  "owner-([a-z]+)")))

> I see some obstacles, but the scheme as such doesn't feel extremely
> complex which could be a good thing.
> 
> 	* what to do if nothing matches?

Hm.

> 	* how to handle crossposting?

Double hm.

> 	* what about fancy splitting?  (haven't used it)

Fancy splitting could be handled the same way as simple splitting.

One thing I like about alternative 1 is that it doesn't require the
user to rewrite the split rules.  And I think that alternative 2 will
be significantly slower, since all the splitting will have to take the 
long way round through Gnus before reaching the mail backends.  And
it's more difficult to optimize this splitting method...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE writes:

> Taking into account what Bill said, it seems that we want to have it
> the other way round: for each source, specify the backends that mail
> should go to, rather than vice versa.

Yes -- that would make it unambiguous which backend should get the
mail.

Perhaps as a user interface, this can be done from the server buffer?
Or should there be a new mail source buffer?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Kai.Grossjohann]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

  > Perhaps as a user interface, this can be done from the server buffer?
  > Or should there be a new mail source buffer?

Considering that /var/spool/mail/jrl is a mail source but usually
there is no backend for this, it seems a new mail source buffer is
needed.  As a first implementation, just setting a variable ought to
be sufficient, though, I think.  Hm.  Coming to think of it, if the
variable is defcustom'd then that would do fine even for a final
implementation :)

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: And now: mail spool fetching makeover

[Kai.Grossjohann]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

  > 1) I want all the mail from this source to be split onto this mail
  > server, and all mail from that sourcce to be split onto that mail
  > server, using the same split methods, or

I have a hard time imagining a situation where this would be useful.
I think you're always going to see special cases.  I don't think that
a rule will ever apply to more than one source/destination pair,
except for a special case.

What you're saying amounts to having a table:

  regex1 -> group1
  regex2 -> group2
  ...

And then you have another table:

  source1 -> backend1
  source2 -> backend2

The only reason for having these two tables would be that you have
many cases where mail from source1 ends up in backend1:group42 whereas
mail from source2 ends up in backend2:group42 BECAUSE OF THE SAME
REGEX.  I don't think this is ever going to happen.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE writes:

> Considering that /var/spool/mail/jrl is a mail source but usually
> there is no backend for this, it seems a new mail source buffer is
> needed.

Yes, but how does one add a backend to a mail source?  Backends are
added to Agent categories in the server buffer, for instance...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE writes:

> The only reason for having these two tables would be that you have
> many cases where mail from source1 ends up in backend1:group42 whereas
> mail from source2 ends up in backend2:group42 BECAUSE OF THE SAME
> REGEX.  I don't think this is ever going to happen.

Ok.  Do we then need a split method tied to the mail source?  Er, or
perhaps a split method tied to the server?

If so, we then have three things:

1) Servers
2) Mail sources
3) Split methods

And these are tied together in various nefarious ways.

I'm beginning to think that this whole thing is much to complex to
ever be used.  The mail source specification thing works independently 
of "server despatch", so perhaps we should just forget about being
able to split to different servers...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Well, implementing this would require lots of work, but it does have
> a certain jennesequa.

...but I don't know what...

Regardless of the implementation, I'd like to see backend independent
mail splitting.

I would really like to be able to send some mail to an IMAP folder and 
som mail to a local nnml folder.

Re: And now: mail spool fetching makeover

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Ok.  Do we then need a split method tied to the mail source?  Er, or
> perhaps a split method tied to the server?

Split methods should be tied to the mail source, I think.

But if the mail source is an IMAP server, and a rule sends an artikle
to an nnimap folder on the same IMAP only the headers should have gone
over the wire to the machine the Gnus is running on.

Re: And now: mail spool fetching makeover

[Kai.Grossjohann]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

  > I'm beginning to think that this whole thing is much to complex to
  > ever be used.

Hm.  You could lump all incoming messages from all sources together,
for one thing.  Then, all you need is a mapping from a regex to a
(server, group) pair (or maybe backend instead of server?  I don't
grok the terminology).

The ability to distinguish where mail came from could be added as a
feature later, if people so desire.

kai
-- 
Abort this operation?   [Abort]  [Cancel]

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Kai.Grossjohann@CS.Uni-Dortmund.DE writes:

> Hm.  You could lump all incoming messages from all sources together,
> for one thing.

Well, that wouldn't really make much of a difference.  Whether we
iterate over sources and call the splitting function for each source,
or whether we iterate over source, add to a file, and then at the end
call the splitting function...  Erm.  The latter sounds like more
work.  :-)

> Then, all you need is a mapping from a regex to a
> (server, group) pair (or maybe backend instead of server?  I don't
> grok the terminology).

Well, we just need a mapping from a regexp to a Gnus group, really.

(setq split-methods
      '(("To: larsi" "nnfolder:junk")
        ("To: ding" "nnml:important")))

But, like I said, this will be quite inefficient and require a large
re-write of oodles of stuff.

> The ability to distinguish where mail came from could be added as a
> feature later, if people so desire.

Well -- probably not, since the syntax we choose may not be flexible
enough.  If we want to do this, we should supply the functionality
now. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Hans de Graaff]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> > The ability to distinguish where mail came from could be added as a
> > feature later, if people so desire.
> 
> Well -- probably not, since the syntax we choose may not be flexible
> enough.  If we want to do this, we should supply the functionality
> now. 

Could we (well, you :-) not add a header indicating the source? That
could then be used to split on. For example:

X-Gnus-Mailsource: pop3:some.server

Hans

Re: And now: mail spool fetching makeover

[Dan Christensen]

Hans de Graaff <graaff@xs4all.nl> writes:
> 
> Could we (well, you :-) not add a header indicating the source? That
> could then be used to split on. For example:
> 
> X-Gnus-Mailsource: pop3:some.server

This might have a couple of extra benefits:

1) In addition to splitting based on mail source, one could score on it
as well.  (Some users may use a different e-mail address for news, and
lower the score of messages coming from that source if it is likely to
be spam.)

2) An article that is respooled, Gcc'd, "B c"'d, etc. could have the
mailsource indicate this.  Is there a way to use Gcc to put a copy
of an outgoing message through the splitting procedure?  If so,
then some of the automated Gcc methods people propose could simply
be split methods.

3) Maybe the procmail stuff could just become a form of splitting too,
e.g. with fancy splitting something like

  ("mailsource" "/home/jdc/incoming/\\(w+\\)\\.spool" "mail.\\1")

might do the trick.  (Of course, such a rule could be activated using
the same procmail variables as are in place now, so the user interface
doesn't have to change;  but the implementation could be simplified.
I love it when different features become unified!)

Dan

-- 
Dan Christensen
jdc@math.jhu.edu

[0.73] Unable to send article written in iso-accents-mode.

[Lloyd Zusman]

I'm using pgnus-0.73 under XEmacs 21.2-beta4.

I routinely switch in and out of `iso-accents-mode' to compose Spanish
email.  Up until version 0.73, I have been able to send this email
without any problems.

Now, however, before sending the article I get prompted with
"Charset used in the article:", and no matter what I enter, including
"iso-8859-1", I 

-- 
 Lloyd Zusman
 ljz@asfast.com

Re: And now: mail spool fetching makeover

[Lars Magne Ingebrigtsen]

Hans de Graaff <graaff@xs4all.nl> writes:

> Could we (well, you :-) not add a header indicating the source? That
> could then be used to split on. For example:
> 
> X-Gnus-Mailsource: pop3:some.server

Yup.  Fix in Pterodactyl Gnus v0.74.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: And now: mail spool fetching makeover

[Kai.Grossjohann]

Dan Christensen <jdc@chow.mat.jhu.edu> writes:

  > [...] Is there a way to use Gcc to put a copy of an outgoing
  > message through the splitting procedure? [...]

This is completely unnecessary, you can just use Fcc to write outgoing
messages to a file, then tell Gnus to read incoming mail from that
file (used to be nnmail-spool-file).

kai
-- 
Abort this operation?   [Abort]  [Cancel]