Mailing list and Gcc header in nnimap groups

Mailing list and Gcc header in nnimap groups

[Kai Großjohann]

Suppose I've got an nnimap group for a mailing list and compose a
message to the list.  The message has a Gcc header pointing to that
group (via gcc-self).

After sending, I will have two messages in the nnimap group, one
message from Gcc and another message that was sent to me via the
mailing list (to which I'm subscribed, of course).

Until now, I've always tried to remember to delete the Gcc header
(even made a keybinding for that), but now I think maybe there's a
better way to prevent such duplicates.

Suggestions?

Using Cyrus 1.6.22.

kai
-- 
I like BOTH kinds of music.

Re: Mailing list and Gcc header in nnimap groups

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> Suppose I've got an nnimap group for a mailing list and compose a
> message to the list.  The message has a Gcc header pointing to that
> group (via gcc-self).

I assume the gcc-self is there to catch non-wide replies.  (If not,
what is it for?)  In that case, you could replace the gcc-self with a
Gcc posting style, where the value is a function that detects whether
the reply is wide, and returns the appropriate string.


paul

Re: Mailing list and Gcc header in nnimap groups

[Ted Zlatanov]

prj@po.cwru.edu (Paul Jarc) writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
>> Suppose I've got an nnimap group for a mailing list and compose a
>> message to the list.  The message has a Gcc header pointing to that
>> group (via gcc-self).
>
> I assume the gcc-self is there to catch non-wide replies.  (If not,
> what is it for?)  In that case, you could replace the gcc-self with a
> Gcc posting style, where the value is a function that detects whether
> the reply is wide, and returns the appropriate string.

What about comparing Message-ID headers of incoming messages to
existing messages, similar to the way nnmail-split-fancy-with-parent
works?  I don't mind duplicate messages so much if they end up in the
same folder, but I don't want to have to set up mailing lists for
everything.  Especially since I don't necessarily group messages by
mailing list.  For instance, my "ding" folder contains messages
forwarded from the newsgroup, mailing list messages, and personal
discussions about bugs.  So for me, the "mailing list == folder"
assumption doesn't work too well.

Maybe there could be an explicit comparison of messages with identical
message IDs.  If the message body is a duplicate (plus or minus some
user-defined variable fodder that some mailing lists inject) then the
user could choose to delete that message.

Thanks
Ted

Re: Mailing list and Gcc header in nnimap groups

[Paul Jarc]

Ted Zlatanov <tzz@northernlight.com> wrote:
> What about comparing Message-ID headers of incoming messages to
> existing messages, similar to the way nnmail-split-fancy-with-parent
> works?

Do you mean duplicate suppression?  (See the manual node of that
name.)

> I don't mind duplicate messages so much if they end up in the
> same folder, but I don't want to have to set up mailing lists for
> everything.  Especially since I don't necessarily group messages by
> mailing list.  For instance, my "ding" folder contains messages
> forwarded from the newsgroup, mailing list messages, and personal
> discussions about bugs.  So for me, the "mailing list == folder"
> assumption doesn't work too well.

Well, Kai's previous suggestion of using the MFT stuff should work
then.  Have a hook function called during sending that checks for
subscribed list addresses in To+Cc, and set Gcc accordingly.  Is there
any send-time hook that gets called before Gcc is processed?

Maybe the checking of addresses could be done only once, and then a
buffer-local variable could be set to record the result.

For Gcc, it'd be nice to also check Bcc for subscribed addresses.  But
MFT should not look at Bcc.  So maybe they shouldn't be combined much
after all.


paul

Re: Mailing list and Gcc header in nnimap groups

[Ted Zlatanov]

prj@po.cwru.edu (Paul Jarc) writes:

> Ted Zlatanov <tzz@northernlight.com> wrote:
>> What about comparing Message-ID headers of incoming messages to
>> existing messages, similar to the way nnmail-split-fancy-with-parent
>> works?
>
> Do you mean duplicate suppression?  (See the manual node of that
> name.)

Well, articles are only marked as read that way, I want to delete them
entirely, and I don't want to rely only on the message ID.  Seems like
it should be possible (or at least, should be possible to apply a
function to known articles).

> Well, Kai's previous suggestion of using the MFT stuff should work
> then.  Have a hook function called during sending that checks for
> subscribed list addresses in To+Cc, and set Gcc accordingly.  Is there
> any send-time hook that gets called before Gcc is processed?

I'll try that suggestion if deleting known articles is not workable.

> Maybe the checking of addresses could be done only once, and then a
> buffer-local variable could be set to record the result.
>
> For Gcc, it'd be nice to also check Bcc for subscribed addresses.  But
> MFT should not look at Bcc.  So maybe they shouldn't be combined much
> after all.

What about people that send a private reply and a reply to the list as
well?  Junking duplicate message IDs would solve that in a way that
*I* find acceptable :)  Or am I wrong to want to delete duplicate
messages in general?

Thanks
Ted

Re: Mailing list and Gcc header in nnimap groups

[Paul Jarc]

Ted Zlatanov <tzz@northernlight.com> wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> Well, Kai's previous suggestion of using the MFT stuff should work
>> then.  Have a hook function called during sending that checks for
>> subscribed list addresses in To+Cc, and set Gcc accordingly.  Is there
>> any send-time hook that gets called before Gcc is processed?
>
> I'll try that suggestion if deleting known articles is not workable.

Setting Gcc intelligently is better: before the extra copies even
exist, it identifies them accurately and "deletes" them.  We avoid the
work of saving an extra copy and of comparing potential duplicates.

> What about people that send a private reply and a reply to the list as
> well?

We put Mail-Followup-To in our outgoing message to instruct them not
to do that.

> Junking duplicate message IDs would solve that in a way that
> *I* find acceptable :)  Or am I wrong to want to delete duplicate
> messages in general?

<URL:http://cr.yp.to/proto/replyto.html>


paul

Re: Mailing list and Gcc header in nnimap groups

[Amos Gouaux]

>>>>> On Fri, 09 Nov 2001 17:50:46 +0100,
>>>>> Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> (kg) writes:

kg> Suppose I've got an nnimap group for a mailing list and compose a
kg> message to the list.  The message has a Gcc header pointing to that
kg> group (via gcc-self).

[...]

kg> Using Cyrus 1.6.22.

Hmmm... is duplicate delivery check enabled on the Cyrus server?  I
think it uses the Message-ID for this check.  Perhaps this would
take care of the dups for you??


-- 
Amos

Re: Mailing list and Gcc header in nnimap groups

[Matt Armstrong]

Amos Gouaux <amos+lists.ding@utdallas.edu> writes:

>>>>>> On Fri, 09 Nov 2001 17:50:46 +0100,
>>>>>> Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> (kg) writes:
>
> kg> Suppose I've got an nnimap group for a mailing list and compose a
> kg> message to the list.  The message has a Gcc header pointing to that
> kg> group (via gcc-self).
>
> [...]
>
> kg> Using Cyrus 1.6.22.
>
> Hmmm... is duplicate delivery check enabled on the Cyrus server?  I
> think it uses the Message-ID for this check.  Perhaps this would
> take care of the dups for you??

Duplicate suppression by Message-ID is insecure:

    - You send a mail to Bob and Patty

    - Patty gets your mail, manages to disrupt Bob's mail delivery and
      sends a message to Bob with the same Message-ID.

    - Bob gets Patty's mail first.

    - Bob's Message-ID duplicate suppression system deletes your mail
      when it finally arrives.

Of course, I don't think this is a problem in practice.  But it is
safer to suppress duplicates based on checksums of the message
contents.

-- 
matt

Re: Mailing list and Gcc header in nnimap groups

[Matt Armstrong]

prj@po.cwru.edu (Paul Jarc) writes:

> Ted Zlatanov <tzz@northernlight.com> wrote:
>> What about comparing Message-ID headers of incoming messages to
>> existing messages, similar to the way nnmail-split-fancy-with-parent
>> works?
>
> Do you mean duplicate suppression?  (See the manual node of that
> name.)
>
>> I don't mind duplicate messages so much if they end up in the
>> same folder, but I don't want to have to set up mailing lists for
>> everything.  Especially since I don't necessarily group messages by
>> mailing list.  For instance, my "ding" folder contains messages
>> forwarded from the newsgroup, mailing list messages, and personal
>> discussions about bugs.  So for me, the "mailing list == folder"
>> assumption doesn't work too well.
>
> Well, Kai's previous suggestion of using the MFT stuff should work
> then.  Have a hook function called during sending that checks for
> subscribed list addresses in To+Cc, and set Gcc accordingly.  Is there
> any send-time hook that gets called before Gcc is processed?
>
> Maybe the checking of addresses could be done only once, and then a
> buffer-local variable could be set to record the result.
>
> For Gcc, it'd be nice to also check Bcc for subscribed addresses.
> But MFT should not look at Bcc.  So maybe they shouldn't be combined
> much after all.

I know from tracing through it that the Gcc: header is gone by the
time message.el sees it -- which makes some sense since message.el
wouldn't know what to do with it.

The MFT code could be re-factored such that it is easier for Gnus to
use it for this purpose though.


-- 
matt

Re: Mailing list and Gcc header in nnimap groups

[Paul Jarc]

"Matt Armstrong" <matt+dated+1007925301.f1c199@lickey.com> wrote:
> But it is safer to suppress duplicates based on checksums of the
> message contents.

Safer, but effective less often, due to mailing list trailers, etc.


paul

Re: Mailing list and Gcc header in nnimap groups

[Paul Jarc]

"Matt Armstrong" <matt+dated+1007925409.d21349@lickey.com> wrote:
> prj@po.cwru.edu (Paul Jarc) writes:
>> Well, Kai's previous suggestion of using the MFT stuff should work
>> then.  Have a hook function called during sending that checks for
>> subscribed list addresses in To+Cc, and set Gcc accordingly.  Is there
>> any send-time hook that gets called before Gcc is processed?
>
> I know from tracing through it that the Gcc: header is gone by the
> time message.el sees it -- which makes some sense since message.el
> wouldn't know what to do with it.

Everything starts with message-send; there's no opportunity for Gcc to
be processed earlier than that.  So message-send sees Gcc.  (What
tracing did you do?)  It would suffice to add a message-*-hook that
gets called before Gcc is processed, if there isn't one already.


paul

Re: Mailing list and Gcc header in nnimap groups

[Ted Zlatanov]

prj@po.cwru.edu (Paul Jarc) writes:

> Everything starts with message-send; there's no opportunity for Gcc to
> be processed earlier than that.  So message-send sees Gcc.  (What
> tracing did you do?)  It would suffice to add a message-*-hook that
> gets called before Gcc is processed, if there isn't one already.

Could Paul or someone else give an example?  Sounds like what I should
do, but my Lisp is not up to the task.

Thanks
Ted

Re: Mailing list and Gcc header in nnimap groups

[Kai Großjohann]

prj@po.cwru.edu (Paul Jarc) writes:

> I assume the gcc-self is there to catch non-wide replies.  (If not,
> what is it for?)  In that case, you could replace the gcc-self with a
> Gcc posting style, where the value is a function that detects whether
> the reply is wide, and returns the appropriate string.

Well, err.  I think I did that for a while, and I wasn't happy with
it.  Hm.  What might have been the reason?  Well, it's annoying to
miss a Gcc header just because you get a message addressed to 3
people and the list, and you want to reply to the 3 people but not
the list.  In that case, the Gcc header will be gone.

Hm.  But we have this nifty MFT logic -- the same logic could be used
for removing the Gcc header if appropriate.  What do you guys think?

kai
-- 
I like BOTH kinds of music.

Re: Mailing list and Gcc header in nnimap groups

[Kai Großjohann]

Amos Gouaux <amos+lists.ding@utdallas.edu> writes:

> Hmmm... is duplicate delivery check enabled on the Cyrus server?  I
> think it uses the Message-ID for this check.  Perhaps this would
> take care of the dups for you??

AFAIK, duplicate suppression happens in `deliver'.  But `deliver'
isn't called for the Gcc'd copy.

kai
-- 
I like BOTH kinds of music.

Re: Mailing list and Gcc header in nnimap groups

[Amos Gouaux]

>>>>> On Fri, 09 Nov 2001 22:53:47 +0100,
>>>>> Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> (kg) writes:

kg> AFAIK, duplicate suppression happens in `deliver'.  But `deliver'
kg> isn't called for the Gcc'd copy.

Oh yeah.  A BCC might do it, but not GCC.  Oh well.  Seemed too easy.

-- 
Amos

Re: Mailing list and Gcc header in nnimap groups

[Matt Armstrong]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Hm.  But we have this nifty MFT logic -- the same logic could be used
> for removing the Gcc header if appropriate.  What do you guys think?

I like the idea.

The same logic could also be used to generate a "Mail-Copies-To:
never" header when appropriate as well.


-- 
matt

Re: Mailing list and Gcc header in nnimap groups

[Kai Großjohann]

Amos Gouaux <amos+lists.ding@utdallas.edu> writes:

>>>>>> On Fri, 09 Nov 2001 22:53:47 +0100,
>>>>>> Kai Großjohann <Kai.Grossjohann@CS.Uni-Dortmund.DE> (kg) writes:
>
> kg> AFAIK, duplicate suppression happens in `deliver'.  But `deliver'
> kg> isn't called for the Gcc'd copy.
>
> Oh yeah.  A BCC might do it, but not GCC.  Oh well.  Seemed too easy.

Oh, boy.  Whee.  Yes, I could indeed use Bcc rather than Gcc, if we
had plus addressing.  Right now, I've got a kludge to circumvent the
need for plus addressing, but it's too ugly.

(Gcc can put the message into the right group right away.)

kai
-- 
I like BOTH kinds of music.

Re: Mailing list and Gcc header in nnimap groups

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) wrote:
> Amos Gouaux <amos+lists.ding@utdallas.edu> writes:
>> Hmmm... is duplicate delivery check enabled on the Cyrus server?  I
>> think it uses the Message-ID for this check.  Perhaps this would
>> take care of the dups for you??
>
> AFAIK, duplicate suppression happens in `deliver'.  But `deliver'
> isn't called for the Gcc'd copy.

Does it need to be?  By the time the list copy comes in, the Gcc'ed
copy will already be there; won't the list copy be dropped then, since
it does come through "deliver"?


paul

Re: Mailing list and Gcc header in nnimap groups

[Amos Gouaux]

>>>>> On Sat, 10 Nov 2001 22:05:56 -0500,
>>>>> Paul Jarc <prj@po.cwru.edu> (pj) writes:

pj> Does it need to be?  By the time the list copy comes in, the Gcc'ed
pj> copy will already be there; won't the list copy be dropped then, since
pj> it does come through "deliver"?

The deliver process stores the Message-ID's of mail that it has
handled into a db.  Since, as Kai reminded me, a GCC just does a
save to that folder, it doesn't go through the steps to stash the
Message-ID into the duplicate delivery db.  So as far as deliver is
concerned, it hasn't seen this message before.

-- 
Amos

Dynamic stuff (was: Mailing list and Gcc header in nnimap groups)

[Lars Magne Ingebrigtsen]

prj@po.cwru.edu (Paul Jarc) writes:

> Safer, but effective less often, due to mailing list trailers, etc.

Which reminds me of an idea I've had now for, er, 30 seconds.  Which
probably means that it's somebody else's idea I've just remembered.

In the Gnus universe, there are two kinds of information.  There's
static information (protocols, etc) and dynamic information (the
layout of slashdot).  These two kinds of information are squished
together in the Gnus source, requiring using the CVS Gnus if you want
thinks like nnslashdot to work more than a couple of weeks to work.

Wouldn't it be nice if it was possible to put the nnslashdot parsing
thingie somewhere outside the source so that it could be downloaded
separately?

(Now I remember.  This really is somebody else's idea.)

However, parsing slashdot is more or less bound to involve Lisp code,
which means that you can't fetch it from an untrusted source, which
(again) means that it might as well just remain in CVS.

Mailing list trailers, on the other hand, don't have to remain there.
Wouldn't it be nice if there was a collaborative source of mailing
list trailers that told mail- and news readers how to filter them
away?

Just imagine.  A world without mailing list trailers.

Sounds beautiful, doesn't it?

And since that would be just a file full of regexps -- "if the headers
match this regexp, then filter out this regexp from the body" -- then
it would be safe to update that list (semi-)automatically for users.
wget -o /usr/share/gnus/trailers.txt http://www.gnus.org/trailers.txt
in cron.

-- 
(domestic pets only, the antidote for overdose, milk.)
   larsi@gnus.org * Lars Magne Ingebrigtsen