Splitting, crossposts, duplicates, the universe, and everything.

Splitting, crossposts, duplicates, the universe, and everything.

[Lloyd Zusman]

[ This is an auto-generated, courtesy copy of a message that I've
  also posted to the gnu.emacs.gnus newsgroup. ]

I'm using pgnus-0.43, and I'm running into a situation involving mail
splitting that I don't know how to solve.

Suppose these are my split methods:

 (setq nnmail-split-methods 
        '(("one"      "^\\(To\\|Cc\\):.*one@asfast\\.com")
          ("two"      "^\\(To\\|Cc\\):.*two@asfast\\.com")
          ("three"    "^\\(To\\|Cc\\):.*three@asfast\\.com")
          ("four"     "^\\(To\\|Cc\\):.*four@asfast\\.com")
	  ("other"    "")))

Furthermore, suppose I have set `nnmail-crosspost' to nil.

I have all my incoming email to the various users on my system going
to a single mail spool, and I use Gnus to split it.

Now, suppose I receive some email with the following `To:' line:

 To: one@asfast.com, two@asfast.com, three@asfast.com, four@asfast.com

Once the mail is split, it only ends up in my `one' mailbox.  But what
I want is a single copy in each of the `one', `two', `three', and
`four' mailboxes.  OK.  So I figured that I'd just set
`nnmail-crosspost' to `t'.  Once I did that, an email with the same
`To:' line as above indeed ended up in each of those four mailboxes,
but it appeared four times in each one, for a total of 16 instances
altogether.

I sort of see why this happened, but I'm wondering if there's a way to
prevent this.  I'd like to cause one and only one copy to appear in
each of the mailboxes that appear on the `To:' line.

I looked at `nnmail-split-fancy', and it doesn't seem to handle this
case.

Any ideas?

Thanks in advance.

-- 
 Lloyd Zusman
 ljz@asfast.com

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Michael Welsh Duggan]

Lloyd Zusman <ljz@asfast.com> writes:

> Suppose these are my split methods:
> 
>  (setq nnmail-split-methods 
>         '(("one"      "^\\(To\\|Cc\\):.*one@asfast\\.com")
>           ("two"      "^\\(To\\|Cc\\):.*two@asfast\\.com")
>           ("three"    "^\\(To\\|Cc\\):.*three@asfast\\.com")
>           ("four"     "^\\(To\\|Cc\\):.*four@asfast\\.com")
>         ("other"    "")))
> 
> Furthermore, suppose I have set `nnmail-crosspost' to nil.
> 
> I have all my incoming email to the various users on my system going
> to a single mail spool, and I use Gnus to split it.
> 
> Now, suppose I receive some email with the following `To:' line:
> 
>  To: one@asfast.com, two@asfast.com, three@asfast.com, four@asfast.com
> 
> Once the mail is split, it only ends up in my `one' mailbox.  But what
> I want is a single copy in each of the `one', `two', `three', and
> `four' mailboxes.  OK.  So I figured that I'd just set
> `nnmail-crosspost' to `t'.  Once I did that, an email with the same
> `To:' line as above indeed ended up in each of those four mailboxes,
> but it appeared four times in each one, for a total of 16 instances
> altogether.
> 
> I sort of see why this happened, but I'm wondering if there's a way to
> prevent this.  I'd like to cause one and only one copy to appear in
> each of the mailboxes that appear on the `To:' line.
> 
> I looked at `nnmail-split-fancy', and it doesn't seem to handle this
> case.

Actually, `nnmail-split-fancy' should work just fine.  Try the
following for the above:

(setq nnmail-split-methods 'nnmail-split-fancy)
(setq nnmail-split-fancy
      '(&
        (to "one@asfast\\.com" "one")
        (to "two@asfast\\.com" "two")
        (to "three@asfast\\.com" "three")
        (to "four@asfast\\.com" "four")
        "other"))

Here is my `nnmail-split-fancy' for reference:

(setq nnmail-split-fancy
      '(| 
        ;; Personal Mail 
        ;; We don't want mailing list foo to appear here, even if from
        ;; the person in question.
        (any "m?duggan@\\(ramlink\\|wwd\\)\\.net" "mail.family") ;From home
        (any "mi-trans-command" "mail.mi-trans-admin")
        ("subject" "\\(cat\\|kantoo\\)/[0-9]+:.*" "mail.gnats") ;Gnats
        (&                              ;Could be crossposted
         ;; Put mail to me in misc
         (|
          ;; Unless it is from one of these guys.
          (from "\\(eric\\|svoboda\\|teruko\\|igo\\|klb\\).*cmu" "mail.job")
          (from "Kathryn_Baker" "mail.job")
          (from "[@.]cat\\.com" "mail.job")
          (any "catalyst-project" "mail.job")
          (any "pittcmulingannouncements" "mail.job")
          (any "lti-\\(seminar\\|staff\\)" "mail.job")
          (any "cyerthall" "mail.job")
          (to "chaos\\|md5i\\|duggan" "mail.misc"))
         ;; Mailing Lists
         (any "ding" "mail.Gnus")       ;Gnus mailing list
         (any "gwm" "mail.gwm")         ;GWM mailing list
         (any "mi@ml\\.usagi\\.org" "mail.mi") ;Maison Ikkoku mailing list
         (any "mi-list\\|mlist-mi" "mail.mi")
                                        ;MI Translation mailing list
         (any "mi-trans@lists\\.best\\.com" "mail.MI-Translation") 
         (any "mule" "mail.mule")       ;MULE mailing list
         (any "ntemacs-hackers" "mail.ntemacs-hackers")
         (any "ntemacs-users" "mail.ntemacs")   ;NTEmacs mailing list
         (any "custom" "mail.custom")   ;emacs-custom mailing list
         (any "w3" "mail.w3")           ;W3 mailing list
         (any "kodocha" "mail.kodocha") ;Kodomo no Omocha ML
         (any "patlabor" "mail.patlabor") ;Patlabor ML
         (any "emacs-unicode" "mail.unicode") ;The Emacs/Unicode Project
         (any "DILBERT_NEWS" "mail.misc")
         (any "anime-l" "mail.misc")    ;UCI anime report
         )
        (to "cmu" "mail.misc")
        ;; Catch all (Should be junk.)
        "mail.junk"))

-- 
Michael Duggan
(md5i@cs.cmu.edu)
.

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Lloyd Zusman]

Michael Welsh Duggan <md5i@cs.cmu.edu> writes:

> Lloyd Zusman <ljz@asfast.com> writes:
> 
> > [ ... ]
> > 
> > Once the mail is split, it only ends up in my `one' mailbox.  But what
> > I want is a single copy in each of the `one', `two', `three', and
> > `four' mailboxes.  OK.  So I figured that I'd just set
> > `nnmail-crosspost' to `t'.  Once I did that, an email with the same
> > `To:' line as above indeed ended up in each of those four mailboxes,
> > but it appeared four times in each one, for a total of 16 instances
> > altogether.
> > 
> > I sort of see why this happened, but I'm wondering if there's a way to
> > prevent this.  I'd like to cause one and only one copy to appear in
> > each of the mailboxes that appear on the `To:' line.
> > 
> > [ ... ]
> > case.
> 
> Actually, `nnmail-split-fancy' should work just fine.  Try the
> following for the above:
> 
> (setq nnmail-split-methods 'nnmail-split-fancy)
> (setq nnmail-split-fancy
>       '(&
>         (to "one@asfast\\.com" "one")
>         (to "two@asfast\\.com" "two")
>         (to "three@asfast\\.com" "three")
>         (to "four@asfast\\.com" "four")
>         "other"))

Thank you very much.

I did this and got the same results as before (16 instances of the
message).  And now I realize why:

Suppose the `To:' line of some email coming to my system looks like
this:

  To: one@asfast.com, two@asfast.com, three@asfast.com, four@asfast.com

This causes 4 separate messages to be sent out, and all four of these
come to my machine.  My local mail software puts all four of these
into a central mailbox, and then Gnus splits them by the above rules.
Each of these four messages will then get sent through my
`nnmail-split-fancy' methods, and each of them will then go to each of
four mailboxes ("one", "two", "three", "four").  In other words, now
there are 16 messages where orginally there were four.

I know that Gnus has a `gnus-suppress-duplicates' variable that can be
set.  Would the use of this variable help me in this case?  The goal
is to have one copy of the message appearing in each of the four
mailboxes.

-- 
 Lloyd Zusman
 ljz@asfast.com

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Alexandre Oliva]

On Nov 17, 1998, Lloyd Zusman <ljz@asfast.com> wrote:

> Each of these four messages will then get sent through my
> `nnmail-split-fancy' methods, and each of them will then go to each of
> four mailboxes ("one", "two", "three", "four").  In other words, now
> there are 16 messages where orginally there were four.

> I know that Gnus has a `gnus-suppress-duplicates' variable that can be
> set.  Would the use of this variable help me in this case?

Yes, unless the host that runs the mailing list software cheats on the 
Message-Ids, which it shouldn't.

-- 
Alexandre Oliva  http://www.dcc.unicamp.br/~oliva  aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org.au}
Universidade Estadual de Campinas, SP, Brasil

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Alexandre Oliva]

[-- Attachment #1: Type: text/plain, Size: 1208 bytes --]


On Nov 17, 1998, Michael Welsh Duggan <md5i@cs.cmu.edu> wrote:

> Actually, `nnmail-split-fancy' should work just fine.  Try the
> following for the above:

> (setq nnmail-split-methods 'nnmail-split-fancy)
> (setq nnmail-split-fancy
>       '(&
>         (to "one@asfast\\.com" "one")
>         (to "two@asfast\\.com" "two")
>         (to "three@asfast\\.com" "three")
>         (to "four@asfast\\.com" "four")
>         "other"))

Since the topic includes `the universe, and everything', I thought I
should post an alternate mail splitting method I developed, and that I 
find very useful.  I have already sent it to Lars, but he kept telling 
me I should send a copyright assignment to FSF, and I would tell him I 
had sent one already.  Unfortunately, I recently found out the
assignment I had sent did not apply to gnus, so I'll have to send a
new one.  I hope to post it soon, but here's the code anyway, if you
feel like giving it a try.  It should probably be renamed to comply
with gnus' file naming general guidelines.

-- 
Alexandre Oliva  http://www.dcc.unicamp.br/~oliva  aoliva@{acm.org}
oliva@{dcc.unicamp.br,gnu.org,egcs.cygnus.com,samba.org.au}
Universidade Estadual de Campinas, SP, Brasil

[-- Attachment #2: Type: application/emacs-lisp, Size: 6732 bytes --]

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Karl Kleinpaste]

Lloyd Zusman <ljz@asfast.com> writes:
>     To: one@asfast.com, two@asfast.com, three@asfast.com, four@asfast.com
> This causes 4 separate messages to be sent out

It certainly should not.  RFC 821, page 3:

   When the same message is sent to multiple recipients the SMTP
   encourages the transmission of only one copy of the data for all the
   recipients at the same destination host.

If you are suffering under an MTA which stupidly generates distinct
SMTP sessions for a single message addressed to multiple recipients at
a single destination, get a new one.  Sheesh, talk about trivial
optimizations..."Don't send it more times than absolutely necessary."

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Lloyd Zusman]

Alexandre Oliva <oliva@dcc.unicamp.br> writes:

> On Nov 17, 1998, Lloyd Zusman <ljz@asfast.com> wrote:
> 
> > Each of these four messages will then get sent through my
> > `nnmail-split-fancy' methods, and each of them will then go to each of
> > four mailboxes ("one", "two", "three", "four").  In other words, now
> > there are 16 messages where orginally there were four.
> 
> > I know that Gnus has a `gnus-suppress-duplicates' variable that can be
> > set.  Would the use of this variable help me in this case?
> 
> Yes, unless the host that runs the mailing list software cheats on the 
> Message-Ids, which it shouldn't.

Well, I tested `gnus-suppress-duplicates', and it does indeed work,
but the manner in which is works isn't what I want.  If I send one
message to the four addresses above and then split using the methods
discussed earlier, `gnus-suppress-duplicates' causes all four of them
to end up in the same mail group (the first one of the group).

I think what I really need to do here would be something like this:

(1) Do not cause all of the email that goes to the various addresses
    to end up in the same mail spool file;  rather, allow each piece
    of email to reside in the mail spool file corresponding to the
    user to which it is sent.  For example, in the case of my sample
    `To:' line ...

      To: one@asfast.com, two@asfast.com, three@asfast.com, four@asfast.com

    ... each of the following spool files will end up with one
    copy of the message:

        /var/spool/mail/one
        /var/spool/mail/two
        /var/spool/mail/three
        /var/spool/mail/four

    [ This part is trivially easy ... all I have to do is stop 
      invoking my program which merges my mail spools together. ]

(2) Cause Gnus to somehow (???) get its `nnml' email from each
    of these four spool files, each one feeding a different
    `nnml' mailbox.  I would also want to perform a separate
    splitting procedure on each of these four spool files, to
    do the higher-granularity splitting that will be necessary.

I think I could take care of the first part of this by making use of
multiple `nnml' backends, each working with a different
`nnmail-spool-file' (is this correct?).  However, I'm not sure if I
can also define different split methods for each backend via
backend-specific values of `nnmail-split-method'.  Does anyone know if
this would work? (I'd rather ask here first before experimenting and
possibly messing up my large, existing mail spools).

Thanks again, in advance.

-- 
 Lloyd Zusman
 ljz@asfast.com

Re: Splitting, crossposts, duplicates, the universe, and everything.

[Stephen Zander]

>>>>> "Lloyd" == Lloyd Zusman <ljz@asfast.com> writes:
    Lloyd> (1) Do not cause all of the email that goes to the various
    Lloyd> addresses to end up in the same mail spool file; rather,
    Lloyd> allow each piece of email to reside in the mail spool file
    Lloyd> corresponding to the user to which it is sent.  For
    Lloyd> example, in the case of my sample `To:' line ...

procmail, mailagent or (I think) deliver will do this for you.

    Lloyd> (2) Cause Gnus to somehow (???) get its `nnml' email from
    Lloyd> each of these four spool files, each one feeding a
    Lloyd> different `nnml' mailbox.  I would also want to perform a
    Lloyd> separate splitting procedure on each of these four spool
    Lloyd> files, to do the higher-granularity splitting that will be
    Lloyd> necessary.

Gnus will easily do this for you.  The following in my ~/.gnus.el

(setq nnml-directory "~/nnml/"
      nnmail-use-procmail t
      nnmail-spool-file 'procmail
      nnmail-procmail-directory "~/.incoming/")

makes things Just Work.

For the record, I don't do *any* splitting within Gnus.  It all
happens beforehand.

-- 
Stephen
---
Perl is really designed more for the guys that will hack Perl at least
20 minutes a day for the rest of their career.  TCL/Python is more a
"20 minutes a week", and VB is probably in that "20 minutes a month"
group. :) -- Randal Schwartz