* resending bounces: DKIM and Message-ID: @ 2023-11-03 12:23 Greg Troxel 2023-11-03 16:14 ` Emanuel Berg 2023-11-03 23:14 ` Eric Abrahamsen 0 siblings, 2 replies; 16+ messages in thread From: Greg Troxel @ 2023-11-03 12:23 UTC (permalink / raw) To: ding I found a behavior problem that I think is a bug in both gnus and opendkim. Headers that are supposed to be added by the originating MTA are not removed, so the resent message has odd content: duplicate DKIM header, and the same Message-ID: My system in NetBSD 9 amd64 with emacs 28.2, but I don't think that's particularly important. Reproduction Recipe: - send a message to a mailing list that you are not subscribed to, so that it will bounce, and CC: an actual person - subscribe to the list and confirm - In the summary at the bounce, S D b (gnus-summary-resend-bounced-mail) - In the resulting message buffer, C-c C-c (realizing or not realizing that the actual person will get a second copy) This seems to me to be a normal sequence of events, and I'm here because it actually happened. It was a discussion of DKIM/DMARC configuration for mailman, so the person in the CC: got two copies and actually noticed that there were two DKIM headers from my domain. The problem is that the message that is sent has two DKIM headers. This is because the one that was added by the MTA on the way to the mailinglist is present because it was in the bounce, and then opendkim added a new one, because it just adds. But also, the Message-ID: was generated for the first message, and the that is reused. That seems also like a bug. The proposal is to fix this by: * in gnus - adding a list of headers which should be removed from bounce messages when doing gnus-summary-resend-bounced-mail - set the list to Message-ID and DKIM-Signature to start with * in opendkim - adjust to remove DKIM-Signature from matching d= See https://sourceforge.net/p/opendkim/bugs/283/ To make this report properly lispy, I have intentionally typod the list by sending to ding2@ instead, and will include the headers from the bounce resend buffer in the body. Greg (end of first message) Content of headers in *unsent-bounce* after S D b: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lexort.com; s=mail; t=1699014219; bh=eeB1OL0c0u2+SOWzRtNCVlH+oCKEc2vYsqRGp4ZsUk0=; h=From:To:Subject:Date; b=TtTS6faBNmfjVMi3VmqAOjsnMjefzbuHrU2yJFTEYuaHMMDQr9lY8j2uAKQHn1bYR g3VmgRP/gkkm0cYU5cGj/Oazk+ooWC+qxghvnXo+0/iDxNdhfgSVQi0k79VSvNlbLP S/l11R1Yf2mk2Ga02Nx/+K3AVxATm17MKh8AcLHc= From: Greg Troxel <gdt@lexort.com> To: ding2@gnus.org Subject: resending bounces: DKIM and Message-ID: OpenPGP: id=098ED60E Date: Fri, 03 Nov 2023 08:23:39 -0400 Message-ID: <rmi34xn2fg4.fsf@s1.lexort.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix) Gcc: nnfolder+archive:sent.2023-11 showing both DKIM-Signature and Message-ID. Because I have changed the To: the original signature will be invalid. I have edited the To: and am about to type C-c C-c. (end of second message) ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-03 12:23 resending bounces: DKIM and Message-ID: Greg Troxel @ 2023-11-03 16:14 ` Emanuel Berg 2023-11-03 23:14 ` Eric Abrahamsen 1 sibling, 0 replies; 16+ messages in thread From: Emanuel Berg @ 2023-11-03 16:14 UTC (permalink / raw) To: ding Greg Troxel wrote: > I found a behavior problem that I think is a bug in both > gnus and opendkim. Headers that are supposed to be added by > the originating MTA are not removed, so the resent message > has odd content: duplicate DKIM header, and the same > Message-ID Not an answer to your situation but FWIW I had problems with DKIM in the past, I don't remember how I solved it but here are a bunch of commands and URLs related to that whole thing: #! /bin/zsh # # this file: # https://dataswamp.org/~incal/conf/.zsh/mail # # DMARC issue: # https://en.wikipedia.org/wiki/DMARC # https://mailarchive.ietf.org/arch/msg/ietf/kL24edUthAOuHuwK3ZnpFkCXduI # # discussion and commands: # http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00476.html # http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00477.html # http://lists.gnu.org/archive/html/help-gnu-emacs/2019-05/msg00481.html # # (rfc 7489) # "Domain-based Message Authentication, Reporting, and Conformance (DMARC)" # <https://datatracker.ietf.org/doc/html/rfc7489> # # (rfc 8616) # "Email Authentication for Internationalized Mail" # <https://datatracker.ietf.org/doc/html/rfc8616> dig-srv () { local srv=$1 nslookup -type=mx $srv dig $srv mx } DMARK_ZOHO=_dmarc.zoho.eu dig-zoho () { dig $DMARK_ZOHO txt +short } host-zoho () { host -t txt $DMARK_ZOHO } test-zoho () { local server=pop.zoho.com local port=995 local server_port=${server}:${port} echo "socat" socat - OPENSSL:${server_port} echo "\nopenssl" openssl s_client -quiet -connect $server_port echo "\ngutls-cli" gnutls-cli --port $port $server } -- underground experts united https://dataswamp.org/~incal ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-03 12:23 resending bounces: DKIM and Message-ID: Greg Troxel 2023-11-03 16:14 ` Emanuel Berg @ 2023-11-03 23:14 ` Eric Abrahamsen 2023-11-05 15:55 ` Greg Troxel 1 sibling, 1 reply; 16+ messages in thread From: Eric Abrahamsen @ 2023-11-03 23:14 UTC (permalink / raw) To: ding Greg Troxel <gdt@lexort.com> writes: > I found a behavior problem that I think is a bug in both gnus and > opendkim. Headers that are supposed to be added by the originating MTA > are not removed, so the resent message has odd content: duplicate DKIM > header, and the same Message-ID: > > My system in NetBSD 9 amd64 with emacs 28.2, but I don't think that's > particularly important. > > Reproduction Recipe: > > - send a message to a mailing list that you are not subscribed to, so > that it will bounce, and CC: an actual person > - subscribe to the list and confirm > - In the summary at the bounce, S D b (gnus-summary-resend-bounced-mail) > - In the resulting message buffer, C-c C-c (realizing or not realizing > that the actual person will get a second copy) > > This seems to me to be a normal sequence of events, and I'm here because > it actually happened. It was a discussion of DKIM/DMARC configuration > for mailman, so the person in the CC: got two copies and actually > noticed that there were two DKIM headers from my domain. > > > The problem is that the message that is sent has two DKIM headers. This > is because the one that was added by the MTA on the way to the > mailinglist is present because it was in the bounce, and then opendkim > added a new one, because it just adds. > > But also, the Message-ID: was generated for the first message, and the > that is reused. That seems also like a bug. > > The proposal is to fix this by: > > * in gnus > - adding a list of headers which should be removed from bounce > messages when doing gnus-summary-resend-bounced-mail I believe this is called `message-ignored-bounced-headers'! It does not currently include DKIM-Signature, but maybe it should. I haven't actually tested this, but looking at the code I believe it should take effect in this situation. > - set the list to Message-ID and DKIM-Signature to start with This is deeper in the protocol weeds than I usually go, so this is a bit tentative, but I will note that there's a separate `gnus-summary-resend-message-edit' command, which will not preserve the Message-ID header. I think there's an argument to be made that, if you're re-sending the bounced the message, it is in fact the same message over again. If you're changing the "To:" or other headers, on the other hand, that sounds like a new message, and you should be using "S D e" instead. What do you think? Eric ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-03 23:14 ` Eric Abrahamsen @ 2023-11-05 15:55 ` Greg Troxel 2023-11-05 19:30 ` Eric Abrahamsen 0 siblings, 1 reply; 16+ messages in thread From: Greg Troxel @ 2023-11-05 15:55 UTC (permalink / raw) To: Eric Abrahamsen; +Cc: ding Eric Abrahamsen <eric@ericabrahamsen.net> writes: >> The proposal is to fix this by: >> >> * in gnus >> - adding a list of headers which should be removed from bounce >> messages when doing gnus-summary-resend-bounced-mail > > I believe this is called `message-ignored-bounced-headers'! It does not > currently include DKIM-Signature, but maybe it should. I haven't > actually tested this, but looking at the code I believe it should take > effect in this situation. Wow, exactly the right thing. Yes, I added DKIM-Signature in .emacs: (setq message-ignored-bounced-headers "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):") and now SDb omits it. I would say that the default value should be changed to include DKIM-Signature. If the message is altered at all it becomes invalid, and the new message should get signed the same way, so having two valid signatures one of which has an incorrect time is just not helpful. I cannot think of ever wanting to keep it. It's not quite like Received but it's close. >> - set the list to Message-ID and DKIM-Signature to start with > > This is deeper in the protocol weeds than I usually go, so this is a bit > tentative, but I will note that there's a separate > `gnus-summary-resend-message-edit' command, which will not preserve the > Message-ID header. I think there's an argument to be made that, if > you're re-sending the bounced the message, it is in fact the same > message over again. If you're changing the "To:" or other headers, on > the other hand, that sounds like a new message, and you should be using > "S D e" instead. Interesting point. When I run that on the bounce, I am queued up to re-send the bounce message, not the original message that was included in the bounce. > What do you think? I would say that 99% of the time, when you are resending a bounce, you are fixing what you did wrong that caused it to bounce, like mistyping a To/CC. In which case the new message is.. a new message. In rare cases, you have fixed something else (like the receiving system such as subscribing or your MTA) and are sending the exact same message. I think: Some people treat messages with the same message-id as dups. I am not sure that's 100% legit but I know it happens. If nobody got the message, it doesn't really matter if the new message-id is the same or not. If the message is identical, and somebody got it twice (because their addr was right the first time), it might be marginally better to have the same message-id. But doing this right requires keeping track of the original buffer contents and later dropping message-id if there is even 1 byte different, and that just feels too hard/fragile to let somebody filter a dup in such an edge case. If some people got the message and some didn't, because the original sender typoed one of the addresses, then the question is if the original successful recipients should get A) a 2nd copy with the same message-id and updated headers B) a 2nd copy with a new message-id and updated headers C) nothing D) something else I lean to B, because they should realize a typo was fixed, especially if the person added [resending due to address typo], and should thus delete the first and keep the second for replies. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-05 15:55 ` Greg Troxel @ 2023-11-05 19:30 ` Eric Abrahamsen 2023-11-07 0:53 ` Greg Troxel 0 siblings, 1 reply; 16+ messages in thread From: Eric Abrahamsen @ 2023-11-05 19:30 UTC (permalink / raw) To: Greg Troxel; +Cc: ding Greg Troxel <gdt@lexort.com> writes: > Eric Abrahamsen <eric@ericabrahamsen.net> writes: > >>> The proposal is to fix this by: >>> >>> * in gnus >>> - adding a list of headers which should be removed from bounce >>> messages when doing gnus-summary-resend-bounced-mail >> >> I believe this is called `message-ignored-bounced-headers'! It does not >> currently include DKIM-Signature, but maybe it should. I haven't >> actually tested this, but looking at the code I believe it should take >> effect in this situation. > > Wow, exactly the right thing. Yes, I added DKIM-Signature in .emacs: > > (setq message-ignored-bounced-headers > "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):") > > and now SDb omits it. > > I would say that the default value should be changed to include > DKIM-Signature. If the message is altered at all it becomes invalid, > and the new message should get signed the same way, so having two valid > signatures one of which has an incorrect time is just not helpful. I > cannot think of ever wanting to keep it. It's not quite like Received > but it's close. Great! And yes, let's add it to the default. I poked around briefly for other signing-type headers that we might want to delete, but haven't come up with anything that puts an actual signature in the headers. Hashcash, maybe? >>> - set the list to Message-ID and DKIM-Signature to start with >> >> This is deeper in the protocol weeds than I usually go, so this is a bit >> tentative, but I will note that there's a separate >> `gnus-summary-resend-message-edit' command, which will not preserve the >> Message-ID header. I think there's an argument to be made that, if >> you're re-sending the bounced the message, it is in fact the same >> message over again. If you're changing the "To:" or other headers, on >> the other hand, that sounds like a new message, and you should be using >> "S D e" instead. > > Interesting point. When I run that on the bounce, I am queued up to > re-send the bounce message, not the original message that was included > in the bounce. The command is supposed to be run on your own original message (in Sent or wherever), not the bounce that you got back. I think if you do that it should behave correctly. >> What do you think? > > I would say that 99% of the time, when you are resending a bounce, you > are fixing what you did wrong that caused it to bounce, like mistyping a > To/CC. In which case the new message is.. a new message. In rare > cases, you have fixed something else (like the receiving system such as > subscribing or your MTA) and are sending the exact same message. Okay, I'd been assuming the most common case was the not-yet-subscribed case. Or some other transient system-level failure, ie not an actual change to the message itself. > I think: > > Some people treat messages with the same message-id as dups. I am not > sure that's 100% legit but I know it happens. Yeah, I don't think we can assume/enforce one interpretation or the other. > If nobody got the message, it doesn't really matter if the new > message-id is the same or not. > > If the message is identical, and somebody got it twice (because their > addr was right the first time), it might be marginally better to have > the same message-id. But doing this right requires keeping track of > the original buffer contents and later dropping message-id if there is > even 1 byte different, and that just feels too hard/fragile to let > somebody filter a dup in such an edge case. I agree we shouldn't try a DWIM thing here. > If some people got the message and some didn't, because the original > sender typoed one of the addresses, then the question is if the > original successful recipients should get > A) a 2nd copy with the same message-id and updated headers > B) a 2nd copy with a new message-id and updated headers > C) nothing > D) something else > I lean to B, because they should realize a typo was fixed, especially > if the person added [resending due to address typo], and should thus > delete the first and keep the second for replies. I'm still coming back to the distinction between "resend-edit" and "resend-bounced". In terms of the commands themselves, I think they do the right thing: The first essentially creates a new message from an earlier template (which fits with the fact that it's called from your "Sent" folder/group), meaning that it regenerates most of the headers. The second command just repeats an earlier message, and preserves all the headers it can. To me it feels right that the first case gets a new Message-ID, and the second case doesn't. The problem is that, in your example situation, the message has been bounced, but the appropriate command to deal with it is actually "resend-edit". In a sense, it's a documentation issue: "Use `resend-bounced' unless you need to edit anything in the message, in that case use `resend-edit'". But it's unreasonable to expect anyone to keep all that in mind. That's as far as I've gotten... ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-05 19:30 ` Eric Abrahamsen @ 2023-11-07 0:53 ` Greg Troxel 2023-11-07 1:57 ` Emanuel Berg ` (2 more replies) 0 siblings, 3 replies; 16+ messages in thread From: Greg Troxel @ 2023-11-07 0:53 UTC (permalink / raw) To: Eric Abrahamsen; +Cc: ding Eric Abrahamsen <eric@ericabrahamsen.net> writes: >> Wow, exactly the right thing. Yes, I added DKIM-Signature in .emacs: >> >> (setq message-ignored-bounced-headers >> "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):") >> >> and now SDb omits it. >> >> I would say that the default value should be changed to include >> DKIM-Signature. If the message is altered at all it becomes invalid, >> and the new message should get signed the same way, so having two valid >> signatures one of which has an incorrect time is just not helpful. I >> cannot think of ever wanting to keep it. It's not quite like Received >> but it's close. > > Great! And yes, let's add it to the default. I poked around briefly for > other signing-type headers that we might want to delete, but haven't > come up with anything that puts an actual signature in the headers. > Hashcash, maybe? I suppose you could add hashcash, but I perceive hashcash to be historical. Is anybody still using it? (I used to have it configured both generated and scored on receipt.) >>> This is deeper in the protocol weeds than I usually go, so this is a bit >>> tentative, but I will note that there's a separate >>> `gnus-summary-resend-message-edit' command, which will not preserve the >>> Message-ID header. I think there's an argument to be made that, if >>> you're re-sending the bounced the message, it is in fact the same >>> message over again. If you're changing the "To:" or other headers, on >>> the other hand, that sounds like a new message, and you should be using >>> "S D e" instead. >> >> Interesting point. When I run that on the bounce, I am queued up to >> re-send the bounce message, not the original message that was included >> in the bounce. > > The command is supposed to be run on your own original message (in Sent > or wherever), not the bounce that you got back. I think if you do that > it should behave correctly. Sure, and it should probably also remove DKIM, but because it's more likely from Sent it probably won't be there. >> I would say that 99% of the time, when you are resending a bounce, you >> are fixing what you did wrong that caused it to bounce, like mistyping a >> To/CC. In which case the new message is.. a new message. In rare >> cases, you have fixed something else (like the receiving system such as >> subscribing or your MTA) and are sending the exact same message. > > Okay, I'd been assuming the most common case was the not-yet-subscribed > case. Or some other transient system-level failure, ie not an actual > change to the message itself. My experience is that almost all bounces are because I typed something wrong. Transient failures are once in a blue moon. Not being subscribed, subscribing and resending (vs just giving up) is rare for me. But I believe this is different for different PEBKAC specimens. >> If some people got the message and some didn't, because the original >> sender typoed one of the addresses, then the question is if the >> original successful recipients should get >> A) a 2nd copy with the same message-id and updated headers >> B) a 2nd copy with a new message-id and updated headers >> C) nothing >> D) something else >> I lean to B, because they should realize a typo was fixed, especially >> if the person added [resending due to address typo], and should thus >> delete the first and keep the second for replies. > > I'm still coming back to the distinction between "resend-edit" and > "resend-bounced". In terms of the commands themselves, I think they do > the right thing: The first essentially creates a new message from an > earlier template (which fits with the fact that it's called from your > "Sent" folder/group), meaning that it regenerates most of the headers. But still, the question is: Is that new message the same or not? > The second command just repeats an earlier message, and preserves all > the headers it can. To me it feels right that the first case gets a new > Message-ID, and the second case doesn't. > > The problem is that, in your example situation, the message has been > bounced, but the appropriate command to deal with it is actually > "resend-edit". It's really not that, but gnus-summary-resend-bounced-mail-edit, a variant where you can change it, vs a variant where you can't (readonly). The current code lets you edit it, and I 99% do edit it, so that seems normal. > In a sense, it's a documentation issue: "Use `resend-bounced' unless you > need to edit anything in the message, in that case use `resend-edit'". > But it's unreasonable to expect anyone to keep all that in mind. Yes, and it's further unreasoable that there are really 4 options as a product of two choices modify vs not extract from bounce vs operate on self-bcc or Sent I'm basically saying that wanting to resend a bounce exactly feels rare, and that even if it is the same, you want a fresh message-id and a fresh dkim signature so that you don't have problems from state created perhaps wrongly by the original message. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 0:53 ` Greg Troxel @ 2023-11-07 1:57 ` Emanuel Berg 2023-11-07 3:15 ` Emanuel Berg 2023-11-07 13:49 ` Greg Troxel 2023-11-07 3:38 ` Eric Abrahamsen 2023-11-10 16:38 ` Eric Abrahamsen 2 siblings, 2 replies; 16+ messages in thread From: Emanuel Berg @ 2023-11-07 1:57 UTC (permalink / raw) To: ding Greg Troxel wrote: > My experience is that almost all bounces are because I typed > something wrong. Do you have a ~/.mailrc file with aliases? If so, you can type the alias and see if it expands, if it does it should be a valid e-mail. Unless you have a typo the file, that is :P -- underground experts united https://dataswamp.org/~incal ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 1:57 ` Emanuel Berg @ 2023-11-07 3:15 ` Emanuel Berg 2023-11-07 13:49 ` Greg Troxel 1 sibling, 0 replies; 16+ messages in thread From: Emanuel Berg @ 2023-11-07 3:15 UTC (permalink / raw) To: ding >> My experience is that almost all bounces are because >> I typed something wrong. > > Do you have a ~/.mailrc file with aliases? alias kate "Katherine Moss <enquiries@katemossagency.com>" -- underground experts united https://dataswamp.org/~incal ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 1:57 ` Emanuel Berg 2023-11-07 3:15 ` Emanuel Berg @ 2023-11-07 13:49 ` Greg Troxel 2023-11-07 13:59 ` Emanuel Berg 1 sibling, 1 reply; 16+ messages in thread From: Greg Troxel @ 2023-11-07 13:49 UTC (permalink / raw) To: ding Emanuel Berg <incal@dataswamp.org> writes: > Greg Troxel wrote: > >> My experience is that almost all bounces are because I typed >> something wrong. > > Do you have a ~/.mailrc file with aliases? No, I use bbdb, and not everything is there. > If so, you can type the alias and see if it expands, if it > does it should be a valid e-mail. Sure, but it is not mandatory to use aliases, and someone typing an email address in To: or CC: is not contrary to documentation. So I don't think we can say "the bounce resend function should never allow editing because the user should never had made a mistake". For me, bounces because I mistype are not that common. It is just that of the bounces where I want to resend it, they are the majority. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 13:49 ` Greg Troxel @ 2023-11-07 13:59 ` Emanuel Berg 0 siblings, 0 replies; 16+ messages in thread From: Emanuel Berg @ 2023-11-07 13:59 UTC (permalink / raw) To: ding Greg Troxel wrote: > Sure, but it is not mandatory to use aliases, and someone > typing an email address in To: or CC: is not contrary to > documentation. So I don't think we can say "the bounce > resend function should never allow editing because the user > should never had made a mistake". That would be an interesting software philosophy. > For me, bounces because I mistype are not that common. Okay, glad to hear it! But yes, actually it sounded that way. -- underground experts united https://dataswamp.org/~incal ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 0:53 ` Greg Troxel 2023-11-07 1:57 ` Emanuel Berg @ 2023-11-07 3:38 ` Eric Abrahamsen 2023-11-11 16:58 ` Greg Troxel 2023-11-10 16:38 ` Eric Abrahamsen 2 siblings, 1 reply; 16+ messages in thread From: Eric Abrahamsen @ 2023-11-07 3:38 UTC (permalink / raw) To: ding Greg Troxel <gdt@lexort.com> writes: > Eric Abrahamsen <eric@ericabrahamsen.net> writes: > >>> Wow, exactly the right thing. Yes, I added DKIM-Signature in .emacs: >>> >>> (setq message-ignored-bounced-headers >>> "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):") >>> >>> and now SDb omits it. >>> >>> I would say that the default value should be changed to include >>> DKIM-Signature. If the message is altered at all it becomes invalid, >>> and the new message should get signed the same way, so having two valid >>> signatures one of which has an incorrect time is just not helpful. I >>> cannot think of ever wanting to keep it. It's not quite like Received >>> but it's close. >> >> Great! And yes, let's add it to the default. I poked around briefly for >> other signing-type headers that we might want to delete, but haven't >> come up with anything that puts an actual signature in the headers. >> Hashcash, maybe? > > I suppose you could add hashcash, but I perceive hashcash to be > historical. Is anybody still using it? (I used to have it configured > both generated and scored on receipt.) I feel like Gnus is willing to be the last one to turn off the lights on a lot of network technologies. >>>> This is deeper in the protocol weeds than I usually go, so this is a bit >>>> tentative, but I will note that there's a separate >>>> `gnus-summary-resend-message-edit' command, which will not preserve the >>>> Message-ID header. I think there's an argument to be made that, if >>>> you're re-sending the bounced the message, it is in fact the same >>>> message over again. If you're changing the "To:" or other headers, on >>>> the other hand, that sounds like a new message, and you should be using >>>> "S D e" instead. >>> >>> Interesting point. When I run that on the bounce, I am queued up to >>> re-send the bounce message, not the original message that was included >>> in the bounce. >> >> The command is supposed to be run on your own original message (in Sent >> or wherever), not the bounce that you got back. I think if you do that >> it should behave correctly. > > Sure, and it should probably also remove DKIM, but because it's more > likely from Sent it probably won't be there. Also, SDe's behavior is to obey a (short) _preserve header_ list, in contrast to SDb, which has a (equally short) _remove header_ list. So in the SDe case, most headers will be dropped. >>> I would say that 99% of the time, when you are resending a bounce, you >>> are fixing what you did wrong that caused it to bounce, like mistyping a >>> To/CC. In which case the new message is.. a new message. In rare >>> cases, you have fixed something else (like the receiving system such as >>> subscribing or your MTA) and are sending the exact same message. >> >> Okay, I'd been assuming the most common case was the not-yet-subscribed >> case. Or some other transient system-level failure, ie not an actual >> change to the message itself. > > My experience is that almost all bounces are because I typed something > wrong. Transient failures are once in a blue moon. Not being > subscribed, subscribing and resending (vs just giving up) is rare for > me. But I believe this is different for different PEBKAC specimens. > >>> If some people got the message and some didn't, because the original >>> sender typoed one of the addresses, then the question is if the >>> original successful recipients should get >>> A) a 2nd copy with the same message-id and updated headers >>> B) a 2nd copy with a new message-id and updated headers >>> C) nothing >>> D) something else >>> I lean to B, because they should realize a typo was fixed, especially >>> if the person added [resending due to address typo], and should thus >>> delete the first and keep the second for replies. >> >> I'm still coming back to the distinction between "resend-edit" and >> "resend-bounced". In terms of the commands themselves, I think they do >> the right thing: The first essentially creates a new message from an >> earlier template (which fits with the fact that it's called from your >> "Sent" folder/group), meaning that it regenerates most of the headers. > > But still, the question is: Is that new message the same or not? What I was saying is: in the SDe case it isn't, but in the SDb case it is. But! See below. >> The second command just repeats an earlier message, and preserves all >> the headers it can. To me it feels right that the first case gets a new >> Message-ID, and the second case doesn't. >> >> The problem is that, in your example situation, the message has been >> bounced, but the appropriate command to deal with it is actually >> "resend-edit". > > It's really not that, but gnus-summary-resend-bounced-mail-edit, a > variant where you can change it, vs a variant where you can't > (readonly). The current code lets you edit it, and I 99% do edit it, so > that seems normal. > >> In a sense, it's a documentation issue: "Use `resend-bounced' unless you >> need to edit anything in the message, in that case use `resend-edit'". >> But it's unreasonable to expect anyone to keep all that in mind. > > Yes, and it's further unreasoable that there are really 4 options as a > product of two choices > modify vs not > extract from bounce vs operate on self-bcc or Sent > > I'm basically saying that wanting to resend a bounce exactly feels rare, > and that even if it is the same, you want a fresh message-id and a fresh > dkim signature so that you don't have problems from state created > perhaps wrongly by the original message. I have a solution! Potentially. When a user runs SDb, we first prompt "Edit original (bounced) message before sending?" If the user chooses "no", we resend immediately using `message-bounce', including the original Message-ID and all that jazz (though still respecting `message-ignore-bounced-headers'). If the user chooses "yes", we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer. This way the user is the one deciding "is this a new message or not?" If they decide to edit, they're essentially sending a new message. WDYT? ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 3:38 ` Eric Abrahamsen @ 2023-11-11 16:58 ` Greg Troxel 2023-11-19 19:44 ` Eric Abrahamsen 0 siblings, 1 reply; 16+ messages in thread From: Greg Troxel @ 2023-11-11 16:58 UTC (permalink / raw) To: Eric Abrahamsen; +Cc: ding Eric Abrahamsen <eric@ericabrahamsen.net> writes: > Greg Troxel <gdt@lexort.com> writes: >> I'm basically saying that wanting to resend a bounce exactly feels rare, >> and that even if it is the same, you want a fresh message-id and a fresh >> dkim signature so that you don't have problems from state created >> perhaps wrongly by the original message. > > I have a solution! Potentially. When a user runs SDb, we first prompt > "Edit original (bounced) message before sending?" > > If the user chooses "no", we resend immediately using `message-bounce', > including the original Message-ID and all that jazz (though still > respecting `message-ignore-bounced-headers'). If the user chooses "yes", > we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer. > This way the user is the one deciding "is this a new message or not?" If > they decide to edit, they're essentially sending a new message. > > WDYT? That could be ok. I personally feel that SDb without editing is so rare as to not be worth accomodating. And, the headers that are included are potentially added by other systems. I would like to see only headers that were present when the message was sent from gnus (or whatever MUA) to the originating MTA. Which makes me think that all of this should tend to keep fewer headers, and not keep those added by intermedaite systems. However this is getting unreasonably complicated and the idea of explicitly dropping troublesome headers and hoping that the residual issues are less than the effort it would take to resolve this 100% perfectly. ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-11 16:58 ` Greg Troxel @ 2023-11-19 19:44 ` Eric Abrahamsen 2023-11-19 21:36 ` Dan Christensen 0 siblings, 1 reply; 16+ messages in thread From: Eric Abrahamsen @ 2023-11-19 19:44 UTC (permalink / raw) To: ding Greg Troxel <gdt@lexort.com> writes: > Eric Abrahamsen <eric@ericabrahamsen.net> writes: > >> Greg Troxel <gdt@lexort.com> writes: > >>> I'm basically saying that wanting to resend a bounce exactly feels rare, >>> and that even if it is the same, you want a fresh message-id and a fresh >>> dkim signature so that you don't have problems from state created >>> perhaps wrongly by the original message. >> >> I have a solution! Potentially. When a user runs SDb, we first prompt >> "Edit original (bounced) message before sending?" >> >> If the user chooses "no", we resend immediately using `message-bounce', >> including the original Message-ID and all that jazz (though still >> respecting `message-ignore-bounced-headers'). If the user chooses "yes", >> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer. > >> This way the user is the one deciding "is this a new message or not?" If >> they decide to edit, they're essentially sending a new message. >> >> WDYT? > > That could be ok. I personally feel that SDb without editing is so > rare as to not be worth accomodating. And, the headers that are > included are potentially added by other systems. I would like to see > only headers that were present when the message was sent from gnus (or > whatever MUA) to the originating MTA. > > Which makes me think that all of this should tend to keep fewer headers, > and not keep those added by intermedaite systems. At this point it feels like we're just turning SDb into SDe, except an SDe that works on a received bounce rather than the user's original sent message. I understand that all this is annoying, and points to a real issue, but it also doesn't quite seem worth disturbing compatibility/expected behavior. WDYT? ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-19 19:44 ` Eric Abrahamsen @ 2023-11-19 21:36 ` Dan Christensen 2023-11-20 0:01 ` Greg Troxel 0 siblings, 1 reply; 16+ messages in thread From: Dan Christensen @ 2023-11-19 21:36 UTC (permalink / raw) To: ding Eric wrote: >>> I have a solution! Potentially. When a user runs SDb, we first prompt >>> "Edit original (bounced) message before sending?" >>> >>> If the user chooses "no", we resend immediately using `message-bounce', >>> including the original Message-ID and all that jazz (though still >>> respecting `message-ignore-bounced-headers'). If the user chooses "yes", >>> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer. I like that solution. I don't like the status quo, since when you do `S D b', it drops you in a message buffer, inviting you to edit it, but then makes the assumption that you won't edit it (by preserving the Message-ID, etc). My second choice is to just assume that it may be edited, and remove the appropriate headers. Dan ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-19 21:36 ` Dan Christensen @ 2023-11-20 0:01 ` Greg Troxel 0 siblings, 0 replies; 16+ messages in thread From: Greg Troxel @ 2023-11-20 0:01 UTC (permalink / raw) To: ding Dan Christensen <jdc@uwo.ca> writes: > Eric wrote: > >>>> I have a solution! Potentially. When a user runs SDb, we first prompt >>>> "Edit original (bounced) message before sending?" >>>> >>>> If the user chooses "no", we resend immediately using `message-bounce', >>>> including the original Message-ID and all that jazz (though still >>>> respecting `message-ignore-bounced-headers'). If the user chooses "yes", >>>> we sneakily delegate to SDe, dumping them in a *Unsent Message* buffer. > > I like that solution. > > I don't like the status quo, since when you do `S D b', it drops you > in a message buffer, inviting you to edit it, but then makes the > assumption that you won't edit it (by preserving the Message-ID, etc). > > My second choice is to just assume that it may be edited, and > remove the appropriate headers. I agree mostly except I lean to "assume edit". Really there are 2 orthogonal choices: "extract a bounce and prepare to resend" vs "resend a message" and "don't edit" vs "edit" except that "resend" more or less is always edit in that you choose a new destination and you add Resent-From: and Resent-To: headers. (If these are covered by DKIM you need to remove DKIM_Signature.) This argument leads to "offer choice to edit or not and prune headers on the edit branch", for both SDb and SDe. Maybe edit by default and ^U prefix arg to not edit? (Only an emacs user could think that's reasonable!) ^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: resending bounces: DKIM and Message-ID: 2023-11-07 0:53 ` Greg Troxel 2023-11-07 1:57 ` Emanuel Berg 2023-11-07 3:38 ` Eric Abrahamsen @ 2023-11-10 16:38 ` Eric Abrahamsen 2 siblings, 0 replies; 16+ messages in thread From: Eric Abrahamsen @ 2023-11-10 16:38 UTC (permalink / raw) To: Greg Troxel; +Cc: ding Greg Troxel <gdt@lexort.com> writes: > Eric Abrahamsen <eric@ericabrahamsen.net> writes: > >>> Wow, exactly the right thing. Yes, I added DKIM-Signature in .emacs: >>> >>> (setq message-ignored-bounced-headers >>> "^\\(Received\\|Return-Path\\|Delivered-To\\|DKIM-Signature\\):") >>> >>> and now SDb omits it. >>> >>> I would say that the default value should be changed to include >>> DKIM-Signature. If the message is altered at all it becomes invalid, >>> and the new message should get signed the same way, so having two valid >>> signatures one of which has an incorrect time is just not helpful. I >>> cannot think of ever wanting to keep it. It's not quite like Received >>> but it's close. >> >> Great! And yes, let's add it to the default. I poked around briefly for >> other signing-type headers that we might want to delete, but haven't >> come up with anything that puts an actual signature in the headers. >> Hashcash, maybe? > > I suppose you could add hashcash, but I perceive hashcash to be > historical. Is anybody still using it? (I used to have it configured > both generated and scored on receipt.) (FWIW I've added both to the default and pushed to master.) ^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2023-11-20 0:01 UTC | newest] Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2023-11-03 12:23 resending bounces: DKIM and Message-ID: Greg Troxel 2023-11-03 16:14 ` Emanuel Berg 2023-11-03 23:14 ` Eric Abrahamsen 2023-11-05 15:55 ` Greg Troxel 2023-11-05 19:30 ` Eric Abrahamsen 2023-11-07 0:53 ` Greg Troxel 2023-11-07 1:57 ` Emanuel Berg 2023-11-07 3:15 ` Emanuel Berg 2023-11-07 13:49 ` Greg Troxel 2023-11-07 13:59 ` Emanuel Berg 2023-11-07 3:38 ` Eric Abrahamsen 2023-11-11 16:58 ` Greg Troxel 2023-11-19 19:44 ` Eric Abrahamsen 2023-11-19 21:36 ` Dan Christensen 2023-11-20 0:01 ` Greg Troxel 2023-11-10 16:38 ` Eric Abrahamsen
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).