resending bounces: DKIM and Message-ID:

resending bounces: DKIM and Message-ID:

[Greg Troxel]

I found a behavior problem that I think is a bug in both gnus and
opendkim.  Headers that are supposed to be added by the originating MTA
are not removed, so the resent message has odd content: duplicate DKIM
header, and the same Message-ID:

My system in NetBSD 9 amd64 with emacs 28.2, but I don't think that's
particularly important.

Reproduction Recipe:

  - send a message to a mailing list that you are not subscribed to, so
    that it will bounce, and CC: an actual person
  - subscribe to the list and confirm
  - In the summary at the bounce, S D b (gnus-summary-resend-bounced-mail)
  - In the resulting message buffer, C-c C-c (realizing or not realizing
    that the actual person will get a second copy)

This seems to me to be a normal sequence of events, and I'm here because
it actually happened.  It was a discussion of DKIM/DMARC configuration
for mailman, so the person in the CC: got two copies and actually
noticed that there were two DKIM headers from my domain.


The problem is that the message that is sent has two DKIM headers.  This
is because the one that was added by the MTA on the way to the
mailinglist is present because it was in the bounce, and then opendkim
added a new one, because it just adds.

But also, the Message-ID: was generated for the first message, and the
that is reused.  That seems also like a bug.


The proposal is to fix this by:

  * in gnus
    - adding a list of headers which should be removed from bounce
      messages when doing gnus-summary-resend-bounced-mail
    - set the list to Message-ID and DKIM-Signature to start with

  * in opendkim
    - adjust to remove DKIM-Signature from matching d=
      See https://sourceforge.net/p/opendkim/bugs/283/

To make this report properly lispy, I have intentionally typod the list
by sending to ding2@ instead, and will include the headers from the
bounce resend buffer in the body.

Greg
(end of first message)

Content of headers in *unsent-bounce* after S D b:

  DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lexort.com; s=mail;
          t=1699014219; bh=eeB1OL0c0u2+SOWzRtNCVlH+oCKEc2vYsqRGp4ZsUk0=;
          h=From:To:Subject:Date;
          b=TtTS6faBNmfjVMi3VmqAOjsnMjefzbuHrU2yJFTEYuaHMMDQr9lY8j2uAKQHn1bYR
           g3VmgRP/gkkm0cYU5cGj/Oazk+ooWC+qxghvnXo+0/iDxNdhfgSVQi0k79VSvNlbLP
           S/l11R1Yf2mk2Ga02Nx/+K3AVxATm17MKh8AcLHc=
  From: Greg Troxel <gdt@lexort.com>
  To: ding2@gnus.org
  Subject: resending bounces: DKIM and Message-ID:
  OpenPGP: id=098ED60E
  Date: Fri, 03 Nov 2023 08:23:39 -0400
  Message-ID: <rmi34xn2fg4.fsf@s1.lexort.com>
  User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix)
  Gcc: nnfolder+archive:sent.2023-11

showing both DKIM-Signature and Message-ID.  Because I have changed the
To: the original signature will be invalid.

I have edited the To: and am about to type C-c C-c.

(end of second message)

Re: resending bounces: DKIM and Message-ID:

[Emanuel Berg]

Greg Troxel wrote:

> I found a behavior problem that I think is a bug in both
> gnus and opendkim. Headers that are supposed to be added by
> the originating MTA are not removed, so the resent message
> has odd content: duplicate DKIM header, and the same
> Message-ID

Not an answer to your situation but FWIW I had problems with
DKIM in the past, I don't remember how I solved it but here
are a bunch of commands and URLs related to that whole thing:

#! /bin/zsh
#
# this file:
#   https://dataswamp.org/~incal/conf/.zsh/mail
#
# DMARC issue:
#   https://en.wikipedia.org/wiki/DMARC
#   https://mailarchive.ietf.org/arch/msg/ietf/kL24edUthAOuHuwK3ZnpFkCXduI
#
# discussion and commands:
#   http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00476.html
#   http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00477.html
#   http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00481.html
#
# (rfc 7489)
#   "Domain-based Message Authentication, Reporting, and Conformance (DMARC)"
#   <https://datatracker.ietf.org/doc/html/rfc7489>
#
# (rfc 8616)
#   "Email Authentication for Internationalized Mail"
#   <https://datatracker.ietf.org/doc/html/rfc8616>

dig-srv () {
    local srv=$1
    nslookup -type=mx $srv
    dig $srv mx
}

DMARK_ZOHO=_dmarc.zoho.eu

dig-zoho () {
    dig $DMARK_ZOHO txt +short
}

host-zoho () {
    host -t txt $DMARK_ZOHO
}

test-zoho () {
    local server=pop.zoho.com
    local port=995
    local server_port=${server}:${port}

    echo "socat"
    socat - OPENSSL:${server_port}

    echo "\nopenssl"
    openssl s_client -quiet -connect $server_port

    echo "\ngutls-cli"
    gnutls-cli --port $port $server
}

-- 
underground experts united
https://dataswamp.org/~incal

Re: resending bounces: DKIM and Message-ID:

[Eric Abrahamsen]

Greg Troxel <gdt@lexort.com> writes:

> I found a behavior problem that I think is a bug in both gnus and
> opendkim.  Headers that are supposed to be added by the originating MTA
> are not removed, so the resent message has odd content: duplicate DKIM
> header, and the same Message-ID:
>
> My system in NetBSD 9 amd64 with emacs 28.2, but I don't think that's
> particularly important.
>
> Reproduction Recipe:
>
>   - send a message to a mailing list that you are not subscribed to, so
>     that it will bounce, and CC: an actual person
>   - subscribe to the list and confirm
>   - In the summary at the bounce, S D b (gnus-summary-resend-bounced-mail)
>   - In the resulting message buffer, C-c C-c (realizing or not realizing
>     that the actual person will get a second copy)
>
> This seems to me to be a normal sequence of events, and I'm here because
> it actually happened.  It was a discussion of DKIM/DMARC configuration
> for mailman, so the person in the CC: got two copies and actually
> noticed that there were two DKIM headers from my domain.
>
>
> The problem is that the message that is sent has two DKIM headers.  This
> is because the one that was added by the MTA on the way to the
> mailinglist is present because it was in the bounce, and then opendkim
> added a new one, because it just adds.
>
> But also, the Message-ID: was generated for the first message, and the
> that is reused.  That seems also like a bug.
> 
> The proposal is to fix this by:
>
>   * in gnus
>     - adding a list of headers which should be removed from bounce
>       messages when doing gnus-summary-resend-bounced-mail

I believe this is called `message-ignored-bounced-headers'! It does not
currently include DKIM-Signature, but maybe it should. I haven't
actually tested this, but looking at the code I believe it should take
effect in this situation.

>     - set the list to Message-ID and DKIM-Signature to start with

This is deeper in the protocol weeds than I usually go, so this is a bit
tentative, but I will note that there's a separate
`gnus-summary-resend-message-edit' command, which will not preserve the
Message-ID header. I think there's an argument to be made that, if
you're re-sending the bounced the message, it is in fact the same
message over again. If you're changing the "To:" or other headers, on
the other hand, that sounds like a new message, and you should be using
"S D e" instead.

What do you think?

Eric

Re: resending bounces: DKIM and Message-ID:

[Greg Troxel]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

>> The proposal is to fix this by:
>>
>>   * in gnus
>>     - adding a list of headers which should be removed from bounce
>>       messages when doing gnus-summary-resend-bounced-mail
>
> I believe this is called `message-ignored-bounced-headers'! It does not
> currently include DKIM-Signature, but maybe it should. I haven't
> actually tested this, but looking at the code I believe it should take
> effect in this situation.

Wow, exactly the right thing.   Yes, I added DKIM-Signature in .emacs:

      (setq message-ignored-bounced-headers
	    "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):")

and now SDb omits it.

I would say that the default value should be changed to include
DKIM-Signature.   If the message is altered at all it becomes invalid,
and the new message should get signed the same way, so having two valid
signatures one of which has an incorrect time is just not helpful.  I
cannot think of ever wanting to keep it.  It's not quite like Received
but it's close.

>>     - set the list to Message-ID and DKIM-Signature to start with
>
> This is deeper in the protocol weeds than I usually go, so this is a bit
> tentative, but I will note that there's a separate
> `gnus-summary-resend-message-edit' command, which will not preserve the
> Message-ID header. I think there's an argument to be made that, if
> you're re-sending the bounced the message, it is in fact the same
> message over again. If you're changing the "To:" or other headers, on
> the other hand, that sounds like a new message, and you should be using
> "S D e" instead.

Interesting point.  When I run that on the bounce, I am queued up to
re-send the bounce message, not the original message that was included
in the bounce.

> What do you think?

I would say that 99% of the time, when you are resending a bounce, you
are fixing what you did wrong that caused it to bounce, like mistyping a
To/CC.  In which case the new message is..  a new message.  In rare
cases, you have fixed something else (like the receiving system such as
subscribing or your MTA) and are sending the exact same message.

I think:

  Some people treat messages with the same message-id as dups.  I am not
  sure that's 100% legit but I know it happens.

  If nobody got the message, it doesn't really matter if the new
  message-id is the same or not.

  If the message is identical, and somebody got it twice (because their
  addr was right the first time), it might be marginally better to have
  the same message-id.  But doing this right requires keeping track of
  the original buffer contents and later dropping message-id if there is
  even 1 byte different, and that just feels too hard/fragile to let
  somebody filter a dup in such an edge case.

  If some people got the message and some didn't, because the original
  sender typoed one of the addresses, then the question is if the
  original successful recipients should get
    A) a 2nd copy with the same message-id and updated headers
    B) a 2nd copy with a new message-id and updated headers
    C) nothing
    D) something else
  I lean to B, because they should realize a typo was fixed, especially
  if the person added [resending due to address typo], and should thus
  delete the first and keep the second for replies.
  

Re: resending bounces: DKIM and Message-ID:

[Eric Abrahamsen]

Greg Troxel <gdt@lexort.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>>> The proposal is to fix this by:
>>>
>>>   * in gnus
>>>     - adding a list of headers which should be removed from bounce
>>>       messages when doing gnus-summary-resend-bounced-mail
>>
>> I believe this is called `message-ignored-bounced-headers'! It does not
>> currently include DKIM-Signature, but maybe it should. I haven't
>> actually tested this, but looking at the code I believe it should take
>> effect in this situation.
>
> Wow, exactly the right thing.   Yes, I added DKIM-Signature in .emacs:
>
>       (setq message-ignored-bounced-headers
> 	    "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):")
>
> and now SDb omits it.
>
> I would say that the default value should be changed to include
> DKIM-Signature.   If the message is altered at all it becomes invalid,
> and the new message should get signed the same way, so having two valid
> signatures one of which has an incorrect time is just not helpful.  I
> cannot think of ever wanting to keep it.  It's not quite like Received
> but it's close.

Great! And yes, let's add it to the default. I poked around briefly for
other signing-type headers that we might want to delete, but haven't
come up with anything that puts an actual signature in the headers.
Hashcash, maybe?

>>>     - set the list to Message-ID and DKIM-Signature to start with
>>
>> This is deeper in the protocol weeds than I usually go, so this is a bit
>> tentative, but I will note that there's a separate
>> `gnus-summary-resend-message-edit' command, which will not preserve the
>> Message-ID header. I think there's an argument to be made that, if
>> you're re-sending the bounced the message, it is in fact the same
>> message over again. If you're changing the "To:" or other headers, on
>> the other hand, that sounds like a new message, and you should be using
>> "S D e" instead.
>
> Interesting point.  When I run that on the bounce, I am queued up to
> re-send the bounce message, not the original message that was included
> in the bounce.

The command is supposed to be run on your own original message (in Sent
or wherever), not the bounce that you got back. I think if you do that
it should behave correctly.

>> What do you think?
>
> I would say that 99% of the time, when you are resending a bounce, you
> are fixing what you did wrong that caused it to bounce, like mistyping a
> To/CC.  In which case the new message is..  a new message.  In rare
> cases, you have fixed something else (like the receiving system such as
> subscribing or your MTA) and are sending the exact same message.

Okay, I'd been assuming the most common case was the not-yet-subscribed
case. Or some other transient system-level failure, ie not an actual
change to the message itself.

> I think:
>
>   Some people treat messages with the same message-id as dups.  I am not
>   sure that's 100% legit but I know it happens.

Yeah, I don't think we can assume/enforce one interpretation or the
other.

>   If nobody got the message, it doesn't really matter if the new
>   message-id is the same or not.
>
>   If the message is identical, and somebody got it twice (because their
>   addr was right the first time), it might be marginally better to have
>   the same message-id.  But doing this right requires keeping track of
>   the original buffer contents and later dropping message-id if there is
>   even 1 byte different, and that just feels too hard/fragile to let
>   somebody filter a dup in such an edge case.

I agree we shouldn't try a DWIM thing here.

>   If some people got the message and some didn't, because the original
>   sender typoed one of the addresses, then the question is if the
>   original successful recipients should get
>     A) a 2nd copy with the same message-id and updated headers
>     B) a 2nd copy with a new message-id and updated headers
>     C) nothing
>     D) something else
>   I lean to B, because they should realize a typo was fixed, especially
>   if the person added [resending due to address typo], and should thus
>   delete the first and keep the second for replies.

I'm still coming back to the distinction between "resend-edit" and
"resend-bounced". In terms of the commands themselves, I think they do
the right thing: The first essentially creates a new message from an
earlier template (which fits with the fact that it's called from your
"Sent" folder/group), meaning that it regenerates most of the headers.
The second command just repeats an earlier message, and preserves all
the headers it can. To me it feels right that the first case gets a new
Message-ID, and the second case doesn't.

The problem is that, in your example situation, the message has been
bounced, but the appropriate command to deal with it is actually
"resend-edit".

In a sense, it's a documentation issue: "Use `resend-bounced' unless you
need to edit anything in the message, in that case use `resend-edit'".
But it's unreasonable to expect anyone to keep all that in mind.

That's as far as I've gotten...

Re: resending bounces: DKIM and Message-ID:

[Greg Troxel]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

>> Wow, exactly the right thing.   Yes, I added DKIM-Signature in .emacs:
>>
>>       (setq message-ignored-bounced-headers
>> 	    "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):")
>>
>> and now SDb omits it.
>>
>> I would say that the default value should be changed to include
>> DKIM-Signature.   If the message is altered at all it becomes invalid,
>> and the new message should get signed the same way, so having two valid
>> signatures one of which has an incorrect time is just not helpful.  I
>> cannot think of ever wanting to keep it.  It's not quite like Received
>> but it's close.
>
> Great! And yes, let's add it to the default. I poked around briefly for
> other signing-type headers that we might want to delete, but haven't
> come up with anything that puts an actual signature in the headers.
> Hashcash, maybe?

I suppose you could add hashcash, but I perceive hashcash to be
historical.  Is anybody still using it?  (I used to have it configured
both generated and scored on receipt.)

>>> This is deeper in the protocol weeds than I usually go, so this is a bit
>>> tentative, but I will note that there's a separate
>>> `gnus-summary-resend-message-edit' command, which will not preserve the
>>> Message-ID header. I think there's an argument to be made that, if
>>> you're re-sending the bounced the message, it is in fact the same
>>> message over again. If you're changing the "To:" or other headers, on
>>> the other hand, that sounds like a new message, and you should be using
>>> "S D e" instead.
>>
>> Interesting point.  When I run that on the bounce, I am queued up to
>> re-send the bounce message, not the original message that was included
>> in the bounce.
>
> The command is supposed to be run on your own original message (in Sent
> or wherever), not the bounce that you got back. I think if you do that
> it should behave correctly.

Sure, and it should probably also remove DKIM, but because it's more
likely from Sent it probably won't be there.

>> I would say that 99% of the time, when you are resending a bounce, you
>> are fixing what you did wrong that caused it to bounce, like mistyping a
>> To/CC.  In which case the new message is..  a new message.  In rare
>> cases, you have fixed something else (like the receiving system such as
>> subscribing or your MTA) and are sending the exact same message.
>
> Okay, I'd been assuming the most common case was the not-yet-subscribed
> case. Or some other transient system-level failure, ie not an actual
> change to the message itself.

My experience is that almost all bounces are because I typed something
wrong.  Transient failures are once in a blue moon.  Not being
subscribed, subscribing and resending (vs just giving up) is rare for
me.  But I believe this is different for different PEBKAC specimens.

>>   If some people got the message and some didn't, because the original
>>   sender typoed one of the addresses, then the question is if the
>>   original successful recipients should get
>>     A) a 2nd copy with the same message-id and updated headers
>>     B) a 2nd copy with a new message-id and updated headers
>>     C) nothing
>>     D) something else
>>   I lean to B, because they should realize a typo was fixed, especially
>>   if the person added [resending due to address typo], and should thus
>>   delete the first and keep the second for replies.
>
> I'm still coming back to the distinction between "resend-edit" and
> "resend-bounced". In terms of the commands themselves, I think they do
> the right thing: The first essentially creates a new message from an
> earlier template (which fits with the fact that it's called from your
> "Sent" folder/group), meaning that it regenerates most of the headers.

But still, the question is: Is that new message the same or not?

> The second command just repeats an earlier message, and preserves all
> the headers it can. To me it feels right that the first case gets a new
> Message-ID, and the second case doesn't.
>
> The problem is that, in your example situation, the message has been
> bounced, but the appropriate command to deal with it is actually
> "resend-edit".

It's really not that, but gnus-summary-resend-bounced-mail-edit, a
variant where you can change it, vs a variant where you can't
(readonly).  The current code lets you edit it, and I 99% do edit it, so
that seems normal.

> In a sense, it's a documentation issue: "Use `resend-bounced' unless you
> need to edit anything in the message, in that case use `resend-edit'".
> But it's unreasonable to expect anyone to keep all that in mind.

Yes, and it's further unreasoable that there are really 4 options as a
product of two choices
  modify vs not
  extract from bounce vs operate on self-bcc or Sent

I'm basically saying that wanting to resend a bounce exactly feels rare,
and that even if it is the same, you want a fresh message-id and a fresh
dkim signature so that you don't have problems from state created
perhaps wrongly by the original message.

Re: resending bounces: DKIM and Message-ID:

[Emanuel Berg]

Greg Troxel wrote:

> My experience is that almost all bounces are because I typed
> something wrong.

Do you have a ~/.mailrc file with aliases?

If so, you can type the alias and see if it expands, if it
does it should be a valid e-mail.

Unless you have a typo the file, that is :P

-- 
underground experts united
https://dataswamp.org/~incal

Re: resending bounces: DKIM and Message-ID:

[Emanuel Berg]

>> My experience is that almost all bounces are because
>> I typed something wrong.
>
> Do you have a ~/.mailrc file with aliases?

alias kate "Katherine Moss <enquiries@katemossagency.com>"

-- 
underground experts united
https://dataswamp.org/~incal

Re: resending bounces: DKIM and Message-ID:

[Greg Troxel]

Emanuel Berg <incal@dataswamp.org> writes:

> Greg Troxel wrote:
>
>> My experience is that almost all bounces are because I typed
>> something wrong.
>
> Do you have a ~/.mailrc file with aliases?

No, I use bbdb, and not everything is there.

> If so, you can type the alias and see if it expands, if it
> does it should be a valid e-mail.

Sure, but it is not mandatory to use aliases, and someone typing an
email address in To: or CC: is not contrary to documentation.  So I
don't think we can say "the bounce resend function should never allow
editing because the user should never had made a mistake".

For me, bounces because I mistype are not that common.  It is just that
of the bounces where I want to resend it, they are the majority.

Re: resending bounces: DKIM and Message-ID:

[Emanuel Berg]

Greg Troxel wrote:

> Sure, but it is not mandatory to use aliases, and someone
> typing an email address in To: or CC: is not contrary to
> documentation. So I don't think we can say "the bounce
> resend function should never allow editing because the user
> should never had made a mistake".

That would be an interesting software philosophy.

> For me, bounces because I mistype are not that common.

Okay, glad to hear it!

But yes, actually it sounded that way.

-- 
underground experts united
https://dataswamp.org/~incal

Re: resending bounces: DKIM and Message-ID:

[Eric Abrahamsen]

Greg Troxel <gdt@lexort.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>>> Wow, exactly the right thing.   Yes, I added DKIM-Signature in .emacs:
>>>
>>>       (setq message-ignored-bounced-headers
>>> 	    "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):")
>>>
>>> and now SDb omits it.
>>>
>>> I would say that the default value should be changed to include
>>> DKIM-Signature.   If the message is altered at all it becomes invalid,
>>> and the new message should get signed the same way, so having two valid
>>> signatures one of which has an incorrect time is just not helpful.  I
>>> cannot think of ever wanting to keep it.  It's not quite like Received
>>> but it's close.
>>
>> Great! And yes, let's add it to the default. I poked around briefly for
>> other signing-type headers that we might want to delete, but haven't
>> come up with anything that puts an actual signature in the headers.
>> Hashcash, maybe?
>
> I suppose you could add hashcash, but I perceive hashcash to be
> historical.  Is anybody still using it?  (I used to have it configured
> both generated and scored on receipt.)

I feel like Gnus is willing to be the last one to turn off the lights on
a lot of network technologies.

>>>> This is deeper in the protocol weeds than I usually go, so this is a bit
>>>> tentative, but I will note that there's a separate
>>>> `gnus-summary-resend-message-edit' command, which will not preserve the
>>>> Message-ID header. I think there's an argument to be made that, if
>>>> you're re-sending the bounced the message, it is in fact the same
>>>> message over again. If you're changing the "To:" or other headers, on
>>>> the other hand, that sounds like a new message, and you should be using
>>>> "S D e" instead.
>>>
>>> Interesting point.  When I run that on the bounce, I am queued up to
>>> re-send the bounce message, not the original message that was included
>>> in the bounce.
>>
>> The command is supposed to be run on your own original message (in Sent
>> or wherever), not the bounce that you got back. I think if you do that
>> it should behave correctly.
>
> Sure, and it should probably also remove DKIM, but because it's more
> likely from Sent it probably won't be there.

Also, SDe's behavior is to obey a (short) _preserve header_ list, in
contrast to SDb, which has a (equally short) _remove header_ list. So in
the SDe case, most headers will be dropped.

>>> I would say that 99% of the time, when you are resending a bounce, you
>>> are fixing what you did wrong that caused it to bounce, like mistyping a
>>> To/CC.  In which case the new message is..  a new message.  In rare
>>> cases, you have fixed something else (like the receiving system such as
>>> subscribing or your MTA) and are sending the exact same message.
>>
>> Okay, I'd been assuming the most common case was the not-yet-subscribed
>> case. Or some other transient system-level failure, ie not an actual
>> change to the message itself.
>
> My experience is that almost all bounces are because I typed something
> wrong.  Transient failures are once in a blue moon.  Not being
> subscribed, subscribing and resending (vs just giving up) is rare for
> me.  But I believe this is different for different PEBKAC specimens.
>
>>>   If some people got the message and some didn't, because the original
>>>   sender typoed one of the addresses, then the question is if the
>>>   original successful recipients should get
>>>     A) a 2nd copy with the same message-id and updated headers
>>>     B) a 2nd copy with a new message-id and updated headers
>>>     C) nothing
>>>     D) something else
>>>   I lean to B, because they should realize a typo was fixed, especially
>>>   if the person added [resending due to address typo], and should thus
>>>   delete the first and keep the second for replies.
>>
>> I'm still coming back to the distinction between "resend-edit" and
>> "resend-bounced". In terms of the commands themselves, I think they do
>> the right thing: The first essentially creates a new message from an
>> earlier template (which fits with the fact that it's called from your
>> "Sent" folder/group), meaning that it regenerates most of the headers.
>
> But still, the question is: Is that new message the same or not?

What I was saying is: in the SDe case it isn't, but in the SDb case it
is. But! See below.

>> The second command just repeats an earlier message, and preserves all
>> the headers it can. To me it feels right that the first case gets a new
>> Message-ID, and the second case doesn't.
>>
>> The problem is that, in your example situation, the message has been
>> bounced, but the appropriate command to deal with it is actually
>> "resend-edit".
>
> It's really not that, but gnus-summary-resend-bounced-mail-edit, a
> variant where you can change it, vs a variant where you can't
> (readonly).  The current code lets you edit it, and I 99% do edit it, so
> that seems normal.
>
>> In a sense, it's a documentation issue: "Use `resend-bounced' unless you
>> need to edit anything in the message, in that case use `resend-edit'".
>> But it's unreasonable to expect anyone to keep all that in mind.
>
> Yes, and it's further unreasoable that there are really 4 options as a
> product of two choices
>   modify vs not
>   extract from bounce vs operate on self-bcc or Sent
>
> I'm basically saying that wanting to resend a bounce exactly feels rare,
> and that even if it is the same, you want a fresh message-id and a fresh
> dkim signature so that you don't have problems from state created
> perhaps wrongly by the original message.

I have a solution! Potentially. When a user runs SDb, we first prompt
"Edit original (bounced) message before sending?"

If the user chooses "no", we resend immediately using `message-bounce',
including the original Message-ID and all that jazz (though still
respecting `message-ignore-bounced-headers'). If the user chooses "yes",
we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer.

This way the user is the one deciding "is this a new message or not?" If
they decide to edit, they're essentially sending a new message.

WDYT?

Re: resending bounces: DKIM and Message-ID:

[Greg Troxel]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Greg Troxel <gdt@lexort.com> writes:

>> I'm basically saying that wanting to resend a bounce exactly feels rare,
>> and that even if it is the same, you want a fresh message-id and a fresh
>> dkim signature so that you don't have problems from state created
>> perhaps wrongly by the original message.
>
> I have a solution! Potentially. When a user runs SDb, we first prompt
> "Edit original (bounced) message before sending?"
>
> If the user chooses "no", we resend immediately using `message-bounce',
> including the original Message-ID and all that jazz (though still
> respecting `message-ignore-bounced-headers'). If the user chooses "yes",
> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer.

> This way the user is the one deciding "is this a new message or not?" If
> they decide to edit, they're essentially sending a new message.
>
> WDYT?

That could be ok.   I personally feel that SDb without editing is so
rare as to not be worth accomodating.   And, the headers that are
included are potentially added by other systems.   I would like to see
only headers that were present when the message was sent from gnus (or
whatever MUA) to the originating MTA.

Which makes me think that all of this should tend to keep fewer headers,
and not keep those added by intermedaite systems.

However this is getting unreasonably complicated and the idea of
explicitly dropping troublesome headers and hoping that the residual
issues are less than the effort it would take to resolve this 100%
perfectly.

Re: resending bounces: DKIM and Message-ID:

[Eric Abrahamsen]

Greg Troxel <gdt@lexort.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Greg Troxel <gdt@lexort.com> writes:
>
>>> I'm basically saying that wanting to resend a bounce exactly feels rare,
>>> and that even if it is the same, you want a fresh message-id and a fresh
>>> dkim signature so that you don't have problems from state created
>>> perhaps wrongly by the original message.
>>
>> I have a solution! Potentially. When a user runs SDb, we first prompt
>> "Edit original (bounced) message before sending?"
>>
>> If the user chooses "no", we resend immediately using `message-bounce',
>> including the original Message-ID and all that jazz (though still
>> respecting `message-ignore-bounced-headers'). If the user chooses "yes",
>> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer.
>
>> This way the user is the one deciding "is this a new message or not?" If
>> they decide to edit, they're essentially sending a new message.
>>
>> WDYT?
>
> That could be ok.   I personally feel that SDb without editing is so
> rare as to not be worth accomodating.   And, the headers that are
> included are potentially added by other systems.   I would like to see
> only headers that were present when the message was sent from gnus (or
> whatever MUA) to the originating MTA.
>
> Which makes me think that all of this should tend to keep fewer headers,
> and not keep those added by intermedaite systems.

At this point it feels like we're just turning SDb into SDe, except an
SDe that works on a received bounce rather than the user's original sent
message. I understand that all this is annoying, and points to a real
issue, but it also doesn't quite seem worth disturbing
compatibility/expected behavior. WDYT?

Re: resending bounces: DKIM and Message-ID:

[Dan Christensen]

Eric wrote:

>>> I have a solution! Potentially. When a user runs SDb, we first prompt
>>> "Edit original (bounced) message before sending?"
>>>
>>> If the user chooses "no", we resend immediately using `message-bounce',
>>> including the original Message-ID and all that jazz (though still
>>> respecting `message-ignore-bounced-headers'). If the user chooses "yes",
>>> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer.

I like that solution.

I don't like the status quo, since when you do `S D b', it drops you
in a message buffer, inviting you to edit it, but then makes the
assumption that you won't edit it (by preserving the Message-ID, etc).

My second choice is to just assume that it may be edited, and
remove the appropriate headers.

Dan

Re: resending bounces: DKIM and Message-ID:

[Greg Troxel]

Dan Christensen <jdc@uwo.ca> writes:

> Eric wrote:
>
>>>> I have a solution! Potentially. When a user runs SDb, we first prompt
>>>> "Edit original (bounced) message before sending?"
>>>>
>>>> If the user chooses "no", we resend immediately using `message-bounce',
>>>> including the original Message-ID and all that jazz (though still
>>>> respecting `message-ignore-bounced-headers'). If the user chooses "yes",
>>>> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer.
>
> I like that solution.
>
> I don't like the status quo, since when you do `S D b', it drops you
> in a message buffer, inviting you to edit it, but then makes the
> assumption that you won't edit it (by preserving the Message-ID, etc).
>
> My second choice is to just assume that it may be edited, and
> remove the appropriate headers.

I agree mostly except I lean to "assume edit".



Really there are 2 orthogonal choices:

  "extract a bounce and prepare to resend" vs "resend a message"

and

  "don't edit" vs "edit"

except that "resend" more or less is always edit in that you choose a
new destination and you add Resent-From: and Resent-To: headers.  (If
these are covered by DKIM you need to remove DKIM_Signature.)


This argument leads to "offer choice to edit or not and prune headers on
the edit branch", for both SDb and SDe.

Maybe edit by default and ^U prefix arg to not edit?  (Only an emacs user
could think that's reasonable!)

Re: resending bounces: DKIM and Message-ID:

[Eric Abrahamsen]

Greg Troxel <gdt@lexort.com> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>>> Wow, exactly the right thing.   Yes, I added DKIM-Signature in .emacs:
>>>
>>>       (setq message-ignored-bounced-headers
>>> 	    "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):")
>>>
>>> and now SDb omits it.
>>>
>>> I would say that the default value should be changed to include
>>> DKIM-Signature.   If the message is altered at all it becomes invalid,
>>> and the new message should get signed the same way, so having two valid
>>> signatures one of which has an incorrect time is just not helpful.  I
>>> cannot think of ever wanting to keep it.  It's not quite like Received
>>> but it's close.
>>
>> Great! And yes, let's add it to the default. I poked around briefly for
>> other signing-type headers that we might want to delete, but haven't
>> come up with anything that puts an actual signature in the headers.
>> Hashcash, maybe?
>
> I suppose you could add hashcash, but I perceive hashcash to be
> historical.  Is anybody still using it?  (I used to have it configured
> both generated and scored on receipt.)

(FWIW I've added both to the default and pushed to master.)