9front - general discussion about 9front
 help / color / mirror / Atom feed
* test
@ 2020-09-21 15:40 kvik
  2020-09-21 17:12 ` [9front] test Stanley Lieber
  2020-09-21 17:54 ` *****SPAM***** " Stefan Hertenberger
  0 siblings, 2 replies; 26+ messages in thread
From: kvik @ 2020-09-21 15:40 UTC (permalink / raw)
  To: 9front

I've changed DMARC policy to 'none'.

Let's see if this mail still gets to spam.

P.S. Sorry for spamming the list.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 15:40 test kvik
@ 2020-09-21 17:12 ` Stanley Lieber
  2020-09-21 17:16   ` hiro
  2020-09-21 17:54 ` *****SPAM***** " Stefan Hertenberger
  1 sibling, 1 reply; 26+ messages in thread
From: Stanley Lieber @ 2020-09-21 17:12 UTC (permalink / raw)
  To: 9front

On September 21, 2020 11:40:20 AM EDT, kvik@a-b.xyz wrote:
>I've changed DMARC policy to 'none'.
>
>Let's see if this mail still gets to spam.
>
>P.S. Sorry for spamming the list.

message received.

sl


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:12 ` [9front] test Stanley Lieber
@ 2020-09-21 17:16   ` hiro
  2020-09-21 17:36     ` ori
  0 siblings, 1 reply; 26+ messages in thread
From: hiro @ 2020-09-21 17:16 UTC (permalink / raw)
  To: 9front

this example has cleared up for me why gmail does such stupidity,
sadly they were misleading and claimed the spam classification was due
to "similar to other spam".

i can confirm that most of the incorrectly classified spam messages i
get in gmail seem to have dmarc not set to none.
i checked a few exemplary senders, one of which being a web forum with
quite a few messages every day.

gmail itself also has it's dmarc set to none.

so while gmail wants to force you to use dmarc (unverified, somebody
claimed that on irc once) they don't seem to think anybody should have
to act upon it in any way ;)


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:16   ` hiro
@ 2020-09-21 17:36     ` ori
  2020-09-21 17:40       ` Stanley Lieber
  0 siblings, 1 reply; 26+ messages in thread
From: ori @ 2020-09-21 17:36 UTC (permalink / raw)
  To: 23hiro, 9front

> this example has cleared up for me why gmail does such stupidity,
> sadly they were misleading and claimed the spam classification was due
> to "similar to other spam".
> 
> i can confirm that most of the incorrectly classified spam messages i
> get in gmail seem to have dmarc not set to none.
> i checked a few exemplary senders, one of which being a web forum with
> quite a few messages every day.
> 
> gmail itself also has it's dmarc set to none.
> 
> so while gmail wants to force you to use dmarc (unverified, somebody
> claimed that on irc once) they don't seem to think anybody should have
> to act upon it in any way ;)

I'll pile on and summarize the discussion from cat-v
earlier today:

Kvik's email was being shitcanned on gmail because
of his dkim conflicting with our munging. Dkim signs
headers that it expects anyone forwarding mail to
leave untouched. Kvik's list was excessive, but most
dkim configs will be unhappy with our rewrites, since
they want the subject left untouched.

That leaves us 3 options:

	1. Strip out DKIM entirely from forwarded emails.
	2. Don't mess with any headers we don't need to
	3. Implement DKIM, munge to our heart's content,
	   and re-sign.

My vote is in favor of 2 right now, and maybe 3 if we
ever implement dkim in upas.

This seems like a good summary of the situation.

	https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html

For reference, the headers gmail doesn't want us to touch are:

	mime-version:in-reply-to:references:from:date:message-id:subject:to;

Not sure what the other big providers want.



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:36     ` ori
@ 2020-09-21 17:40       ` Stanley Lieber
  2020-09-21 17:51         ` ori
                           ` (2 more replies)
  0 siblings, 3 replies; 26+ messages in thread
From: Stanley Lieber @ 2020-09-21 17:40 UTC (permalink / raw)
  To: 9front

On September 21, 2020 1:36:11 PM EDT, ori@eigenstate.org wrote:
>> this example has cleared up for me why gmail does such stupidity,
>> sadly they were misleading and claimed the spam classification was
>due
>> to "similar to other spam".
>> 
>> i can confirm that most of the incorrectly classified spam messages i
>> get in gmail seem to have dmarc not set to none.
>> i checked a few exemplary senders, one of which being a web forum
>with
>> quite a few messages every day.
>> 
>> gmail itself also has it's dmarc set to none.
>> 
>> so while gmail wants to force you to use dmarc (unverified, somebody
>> claimed that on irc once) they don't seem to think anybody should
>have
>> to act upon it in any way ;)
>
>I'll pile on and summarize the discussion from cat-v
>earlier today:
>
>Kvik's email was being shitcanned on gmail because
>of his dkim conflicting with our munging. Dkim signs
>headers that it expects anyone forwarding mail to
>leave untouched. Kvik's list was excessive, but most
>dkim configs will be unhappy with our rewrites, since
>they want the subject left untouched.
>
>That leaves us 3 options:
>
>	1. Strip out DKIM entirely from forwarded emails.
>	2. Don't mess with any headers we don't need to
>	3. Implement DKIM, munge to our heart's content,
>	   and re-sign.
>
>My vote is in favor of 2 right now, and maybe 3 if we
>ever implement dkim in upas.
>
>This seems like a good summary of the situation.
>
>	https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html
>
>For reference, the headers gmail doesn't want us to touch are:
>
>	mime-version:in-reply-to:references:from:date:message-id:subject:to;
>
>Not sure what the other big providers want.

how does every other mailing list in the world manage modifying the subject line?

sl


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:40       ` Stanley Lieber
@ 2020-09-21 17:51         ` ori
  2020-09-21 17:59           ` ori
  2020-09-21 18:27           ` Kurt H Maier
  2020-09-21 17:58         ` hiro
  2020-09-21 17:59         ` Lyndon Nerenberg
  2 siblings, 2 replies; 26+ messages in thread
From: ori @ 2020-09-21 17:51 UTC (permalink / raw)
  To: 9front

>>
>>That leaves us 3 options:
>>
>>	1. Strip out DKIM entirely from forwarded emails.
>>	2. Don't mess with any headers we don't need to
>>	3. Implement DKIM, munge to our heart's content,
>>	   and re-sign.
>>
> 
> how does every other mailing list in the world manage modifying the subject line?
> 
> sl

 From a non-comprehensive survey (some google groups,
9fans): Option 3: they rewrite stuff and sign it again
with their own key.

OpenBSD mailing lists forward without munging.



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 15:40 test kvik
  2020-09-21 17:12 ` [9front] test Stanley Lieber
@ 2020-09-21 17:54 ` Stefan Hertenberger
  2020-09-21 20:33   ` Kurt H Maier
  1 sibling, 1 reply; 26+ messages in thread
From: Stefan Hertenberger @ 2020-09-21 17:54 UTC (permalink / raw)
  To: 9front

Am Mon, 21 Sep 2020 17:40:20 +0200
schrieb kvik@a-b.xyz:

> Spam detection software, running on the system "vbsd.alarum.de",
> has identified this incoming email as possible spam.  The original
> message has been attached to this so you can view it or label
> similar future email.  If you have any questions, see
> postmaster for details.
> 
> Content preview:  I've changed DMARC policy to 'none'. Let's see if
> this mail still gets to spam. P.S. Sorry for spamming the list. 
> 
> Content analysis details:   (5.1 points, 5.0 required)
> 
>  pts rule name              description
> ---- ----------------------
> -------------------------------------------------- -0.0
> RCVD_IN_MSPIKE_H4      RBL: Very Good reputation (+4) [94.23.1.103
> listed in wl.mailspike.net] 0.0 URIBL_BLOCKED          ADMINISTRATOR
> NOTICE: The query to URIBL was blocked.  See
>                             http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
>                              for more information.
>                             [URIs: a-b.xyz]
>  0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level
>                             mail domains are different
>  0.0 T_PDS_OTHER_BAD_TLD    Untrustworthy TLDs
>                             [URI: a-b.xyz (xyz)]
>  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
>  0.9 SPF_FAIL               SPF: sender does not match SPF record
> (fail) [SPF failed: Please see
> http://www.openspf.org/Why?s=mfrom;id=9front-bounces%40ewsd.inri.net;ip=94.23.1.103;r=vbsd.alarum.de]
> 0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not
> necessarily valid 0.1 DKIM_INVALID           DKIM or DK signature
> exists, but is not valid 0.0 UNPARSEABLE_RELAY      Informational:
> message has unparseable relay lines
> -0.0 RCVD_IN_MSPIKE_WL      Mailspike good senders
>  2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD
>  1.3 RDNS_NONE              Delivered to internal network by a host
> with no rDNS 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
> 
> 

hi,

SpamAssassin still thinks this is spam.

Stefan


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:40       ` Stanley Lieber
  2020-09-21 17:51         ` ori
@ 2020-09-21 17:58         ` hiro
  2020-09-21 18:42           ` Stanley Lieber
  2020-09-21 17:59         ` Lyndon Nerenberg
  2 siblings, 1 reply; 26+ messages in thread
From: hiro @ 2020-09-21 17:58 UTC (permalink / raw)
  To: 9front

option 1 and 3 would cover that

On 9/21/20, Stanley Lieber <sl@stanleylieber.com> wrote:
> On September 21, 2020 1:36:11 PM EDT, ori@eigenstate.org wrote:
>>> this example has cleared up for me why gmail does such stupidity,
>>> sadly they were misleading and claimed the spam classification was
>>due
>>> to "similar to other spam".
>>>
>>> i can confirm that most of the incorrectly classified spam messages i
>>> get in gmail seem to have dmarc not set to none.
>>> i checked a few exemplary senders, one of which being a web forum
>>with
>>> quite a few messages every day.
>>>
>>> gmail itself also has it's dmarc set to none.
>>>
>>> so while gmail wants to force you to use dmarc (unverified, somebody
>>> claimed that on irc once) they don't seem to think anybody should
>>have
>>> to act upon it in any way ;)
>>
>>I'll pile on and summarize the discussion from cat-v
>>earlier today:
>>
>>Kvik's email was being shitcanned on gmail because
>>of his dkim conflicting with our munging. Dkim signs
>>headers that it expects anyone forwarding mail to
>>leave untouched. Kvik's list was excessive, but most
>>dkim configs will be unhappy with our rewrites, since
>>they want the subject left untouched.
>>
>>That leaves us 3 options:
>>
>>	1. Strip out DKIM entirely from forwarded emails.
>>	2. Don't mess with any headers we don't need to
>>	3. Implement DKIM, munge to our heart's content,
>>	   and re-sign.
>>
>>My vote is in favor of 2 right now, and maybe 3 if we
>>ever implement dkim in upas.
>>
>>This seems like a good summary of the situation.
>>
>>	https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html
>>
>>For reference, the headers gmail doesn't want us to touch are:
>>
>>	mime-version:in-reply-to:references:from:date:message-id:subject:to;
>>
>>Not sure what the other big providers want.
>
> how does every other mailing list in the world manage modifying the subject
> line?
>
> sl
>


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:40       ` Stanley Lieber
  2020-09-21 17:51         ` ori
  2020-09-21 17:58         ` hiro
@ 2020-09-21 17:59         ` Lyndon Nerenberg
  2 siblings, 0 replies; 26+ messages in thread
From: Lyndon Nerenberg @ 2020-09-21 17:59 UTC (permalink / raw)
  To: 9front

Stanley Lieber writes:

> how does every other mailing list in the world manage modifying the subjec=
> t line?

Poorly.  They rewrite the From: and MAIL FROM addresses to match
the mailing list exploder address, thus completely destroying the
functionality of the From: header.

DMARC is a cluster fuck that must be made to die.  Just don't do it.

--lyndon


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:51         ` ori
@ 2020-09-21 17:59           ` ori
  2020-09-21 18:27           ` Kurt H Maier
  1 sibling, 0 replies; 26+ messages in thread
From: ori @ 2020-09-21 17:59 UTC (permalink / raw)
  To: ori, 9front


> 
> OpenBSD mailing lists forward without munging.

On closer inspection, they also strip DKIM headers.



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:51         ` ori
  2020-09-21 17:59           ` ori
@ 2020-09-21 18:27           ` Kurt H Maier
  1 sibling, 0 replies; 26+ messages in thread
From: Kurt H Maier @ 2020-09-21 18:27 UTC (permalink / raw)
  To: 9front

On Mon, Sep 21, 2020 at 10:51:46AM -0700, ori@eigenstate.org wrote:
> 
>  From a non-comprehensive survey (some google groups,
> 9fans): Option 3: they rewrite stuff and sign it again
> with their own key.
> 
> OpenBSD mailing lists forward without munging.
> 

This is why on some lists (I think TUHS, maybe others) you get some
messages from the recipient (these are from non-DKIM senders) and some
messages from "Original Sender via TUHS" with a TUHS reply-to and TUHS
DKIM signature.

khm


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 17:58         ` hiro
@ 2020-09-21 18:42           ` Stanley Lieber
  2020-09-21 21:40             ` hiro
  2020-09-22 12:57             ` Ethan Gardener
  0 siblings, 2 replies; 26+ messages in thread
From: Stanley Lieber @ 2020-09-21 18:42 UTC (permalink / raw)
  To: 9front

On September 21, 2020 1:58:17 PM EDT, hiro <23hiro@gmail.com> wrote:
>option 1 and 3 would cover that
>
>On 9/21/20, Stanley Lieber <sl@stanleylieber.com> wrote:
>> On September 21, 2020 1:36:11 PM EDT, ori@eigenstate.org wrote:
>>>> this example has cleared up for me why gmail does such stupidity,
>>>> sadly they were misleading and claimed the spam classification was
>>>due
>>>> to "similar to other spam".
>>>>
>>>> i can confirm that most of the incorrectly classified spam messages
>i
>>>> get in gmail seem to have dmarc not set to none.
>>>> i checked a few exemplary senders, one of which being a web forum
>>>with
>>>> quite a few messages every day.
>>>>
>>>> gmail itself also has it's dmarc set to none.
>>>>
>>>> so while gmail wants to force you to use dmarc (unverified,
>somebody
>>>> claimed that on irc once) they don't seem to think anybody should
>>>have
>>>> to act upon it in any way ;)
>>>
>>>I'll pile on and summarize the discussion from cat-v
>>>earlier today:
>>>
>>>Kvik's email was being shitcanned on gmail because
>>>of his dkim conflicting with our munging. Dkim signs
>>>headers that it expects anyone forwarding mail to
>>>leave untouched. Kvik's list was excessive, but most
>>>dkim configs will be unhappy with our rewrites, since
>>>they want the subject left untouched.
>>>
>>>That leaves us 3 options:
>>>
>>>	1. Strip out DKIM entirely from forwarded emails.
>>>	2. Don't mess with any headers we don't need to
>>>	3. Implement DKIM, munge to our heart's content,
>>>	   and re-sign.
>>>
>>>My vote is in favor of 2 right now, and maybe 3 if we
>>>ever implement dkim in upas.
>>>
>>>This seems like a good summary of the situation.
>>>
>>>	https://begriffs.com/posts/2018-09-18-dmarc-mailing-list.html
>>>
>>>For reference, the headers gmail doesn't want us to touch are:
>>>
>>>	mime-version:in-reply-to:references:from:date:message-id:subject:to;
>>>
>>>Not sure what the other big providers want.
>>
>> how does every other mailing list in the world manage modifying the
>subject
>> line?
>>
>> sl
>>

my point is, the implication is that dkim is both universally expected and desirable. is that really the case? it seems like just glossing over the question is silly. we need to make a decision (and modify it later, if necessary). if we need dkim, we should implement it. if not, not. right?

sl


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 17:54 ` *****SPAM***** " Stefan Hertenberger
@ 2020-09-21 20:33   ` Kurt H Maier
  2020-09-21 21:23     ` kvik
  2020-09-22 18:57     ` Stefan Hertenberger
  0 siblings, 2 replies; 26+ messages in thread
From: Kurt H Maier @ 2020-09-21 20:33 UTC (permalink / raw)
  To: 9front

Your SpamAssassin installation is misconfigured.  Notes inline.


On Mon, Sep 21, 2020 at 07:54:29PM +0200, Stefan Hertenberger wrote:
> > 0.9 SPF_FAIL               SPF: sender does not match SPF record
> > 0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not necessarily valid 
> > 0.1 DKIM_INVALID           DKIM or DK signature exists, but is not valid 

These are caused by invalid forwarding confix on a-b.xyz.  It needs to
strip DKIM and rewrite From: if it's going to behave like this.  I
suspect but cannot prove that migadu is adding the dkim signatures,
which then don't match the Fron: line since 9front.org mail doesn't come
from migadu.

> >  2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD
> >  1.3 RDNS_NONE              Delivered to internal network by a host with no rDNS 
> >  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD

These are the majority of the weights causing the SPAM tag from SA, and
again it's because xyz is a spam hotbed and also because a-b.xyz's IP
address reverse-resolves to ten.a-b.xyz.  

If you intend to continue receiving 9front mail through this domain,
it's probably simplest to whitelist the domain in your sa-spamd rules,
since nothing sl does to the 9front list will change any of these
things.  Dropping a 'whitelist_from_rcvd *@9front.org a-b.xyz' into your
spamassassin rules may do it, but without seeing the full headers of the
as-delivered message I can't be sure.

khm


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 20:33   ` Kurt H Maier
@ 2020-09-21 21:23     ` kvik
  2020-09-21 21:36       ` Kurt H Maier
  2020-09-22 18:57     ` Stefan Hertenberger
  1 sibling, 1 reply; 26+ messages in thread
From: kvik @ 2020-09-21 21:23 UTC (permalink / raw)
  To: 9front

Khm,

I'm very confused with this email.

I don't run SpamAssassin.  Maybe my provider (Migadu) does but
I don't know about it.  More so, the problem here wasn't me getting
mail from @9front.org but my mail sent to the list being sorted into
Spam for (at least) the Gmail users.

There were two things at play here:

1) @9front.org mailing list mangles the headers, like the subject line,
without taking ownership of the mail. This causes my DKIM-signed
mail to fail to verify at the receiver's end.
2) I had my DMARC policy set to 'reject', which tells the receiver
to drop the faulty mail. Gmail chose to mark it as spam instead.

Now that I have disabled this DMARC policy my mails through the
list seem to arrive fine to everyone.

I can't do anything about SUSPICIOUS NTLD, but I'll check what's
up with the rDNS.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 21:23     ` kvik
@ 2020-09-21 21:36       ` Kurt H Maier
  2020-09-21 21:42         ` kvik
  0 siblings, 1 reply; 26+ messages in thread
From: Kurt H Maier @ 2020-09-21 21:36 UTC (permalink / raw)
  To: 9front

On Mon, Sep 21, 2020 at 11:23:12PM +0200, kvik@a-b.xyz wrote:
> I can't do anything about SUSPICIOUS NTLD, but I'll check what's
> up with the rDNS.

This is causing more of the spam-weighting than the dkim issue was.  For
some reason SpamAssassin is being run *after* an a-b.xyz MTA is touching
things.  As a result a lot of the received-from stuff is mismatched
against the origin domain, and SA isn't smart enough to reverse-engineer
the delivery chain.

khm


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 18:42           ` Stanley Lieber
@ 2020-09-21 21:40             ` hiro
  2020-09-21 22:20               ` hiro
  2020-09-22 12:57             ` Ethan Gardener
  1 sibling, 1 reply; 26+ messages in thread
From: hiro @ 2020-09-21 21:40 UTC (permalink / raw)
  To: 9front

best to ask these people: https://www.mail-archive.com/mailop@mailop.org

imo if dkim+dmarc becomes popular enough I do expect that we (mailing
list operators) should change our convention and stop mangling headers
like the subject, from, or body of the mail.

but this is also only viable if people don't continue to use spf +
dmarc (!=none) without valid dkim, in the longer term. I'd say then
it's their own fault and there should be no way out provided by the
mailing list operator.

but i'd certainly wish that NONE of any above mentioned technologies
were used by anybody lest enforced. so definitely not desirable.

at this point i don't think there's a need for action. the early
adopters of strict dmarc setups deserve whatever punishment they are
experiencing for the upcoming while...


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 21:36       ` Kurt H Maier
@ 2020-09-21 21:42         ` kvik
  2020-09-21 21:57           ` Kurt H Maier
  2020-09-21 22:16           ` hiro
  0 siblings, 2 replies; 26+ messages in thread
From: kvik @ 2020-09-21 21:42 UTC (permalink / raw)
  To: 9front

I still don't understand whose SA you are talking about.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 21:42         ` kvik
@ 2020-09-21 21:57           ` Kurt H Maier
  2020-09-21 22:32             ` kvik
  2020-09-21 22:16           ` hiro
  1 sibling, 1 reply; 26+ messages in thread
From: Kurt H Maier @ 2020-09-21 21:57 UTC (permalink / raw)
  To: 9front

On Mon, Sep 21, 2020 at 11:42:50PM +0200, kvik@a-b.xyz wrote:
> I still don't understand whose SA you are talking about.

>> Spam detection software, running on the system "vbsd.alarum.de"

this one.  It's far angrier about receiving mail (any mail) from a-b.xyz
than any dkim-related problem.  The DKIM errors contribute 0.2, the SPF
failure contributes 0.9, the .xyz domain contributes 2.5, and the lack of
accurate RDNS contributes 1.3.  The default SA spam threshold is 5, so
you're already at 3.8 just for any message at all from any xyz domain
with bad RDNS.  

IMO the 'suspicious ntld' rule is egregious, and the only thing that you
(kvik) should care about is fixing the server ident -- I think your
system is identifying itself during SMTP handshake as 'a-b.xyz' when it
should be identifying as 'ten.a-b.xyz' and nothing else should need to
change.

It's also possible that the mailserver at vbsd.alarum.de is failing to
resolve ANY rdns, but I can't tell from the outside.

If you want to get fancy you could add the 9front.org sending host to
your SPF permissions and that would cure that, but I don't really think
that's your problem.

khm


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 21:42         ` kvik
  2020-09-21 21:57           ` Kurt H Maier
@ 2020-09-21 22:16           ` hiro
  1 sibling, 0 replies; 26+ messages in thread
From: hiro @ 2020-09-21 22:16 UTC (permalink / raw)
  To: 9front

is khm conflating kvik with stefan?


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 21:40             ` hiro
@ 2020-09-21 22:20               ` hiro
  0 siblings, 0 replies; 26+ messages in thread
From: hiro @ 2020-09-21 22:20 UTC (permalink / raw)
  To: 9front

here's when it all started ;)
https://thehackernews.com/2014/04/yahoos-new-dmarc-policy-destroys-every.html


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 21:57           ` Kurt H Maier
@ 2020-09-21 22:32             ` kvik
  2020-09-22  1:04               ` Kurt H Maier
  0 siblings, 1 reply; 26+ messages in thread
From: kvik @ 2020-09-21 22:32 UTC (permalink / raw)
  To: 9front

> IMO the 'suspicious ntld' rule is egregious, and the only thing that you
> (kvik) should care about is fixing the server ident -- I think your
> system is identifying itself during SMTP handshake as 'a-b.xyz' when it
> should be identifying as 'ten.a-b.xyz' and nothing else should need to
> change.

Thanks, very valuable info.

What's confusing to me is how any of my systems, ten.a-b.xyz in
particular, come into this picture -- they have no part in the mail
business.

I'm paying Migadu to host all my mail, so I set up my clients with their
SMTP servers, and all the mail-related DNS records are pointed at their
servers exactly as instructed.  So, why does SA go check for a-b.xyz and
its rDNS?  Shouldn't it be doing that dance with Migadu's outgoing SMTP
server, and only getting the SPF info out of a-b.xyz to confirm?

I know nothing about mail anymore.


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 22:32             ` kvik
@ 2020-09-22  1:04               ` Kurt H Maier
  0 siblings, 0 replies; 26+ messages in thread
From: Kurt H Maier @ 2020-09-22  1:04 UTC (permalink / raw)
  To: 9front

On Tue, Sep 22, 2020 at 12:32:38AM +0200, kvik@a-b.xyz wrote:
> 
> What's confusing to me is how any of my systems, ten.a-b.xyz in
> particular, come into this picture -- they have no part in the mail
> business.

Some server, I don't know which, is identifying to some other server as
'a-b.xyz' and this is where the problem starts.  I suspect but cannot
know that this may be migadu's service masquerading as your domain for
purposes of mail deliverability, but it might just be some server along
the chain setting a fucked-up header and subsequent systems getting mad
at that header.

Anyway, once Mystery Asshole identifies as a-b.xyz, the receiving
server, in this case vbsd.alarum.de, looks at the IP making this claim.
We don't know what IP that might be because we don't have access to
headers or logs.  Regardless, it wants to check that the host sending
from this IP and claiming to be a-b.xyz is in fact a-b.xyz, so it does a
reverse DNS lookup on that address, which comes up empty.

SpamAssassin grumply assigns the message an RDNS_NONE label with a spam
weight of 1.3 and continues processing.

We cannot know from available data whether this SpamAssassin evaluation
is occurring during the SMTP handshake, or if that handshake is
happening and then headers are being set for SpamAssassin to get mad at.
If it's the latter, it's possible that something is munging those
headers before SpamAssassin gets its hands on the message.

However:  even if it had looked up rDNS of an ACTUAL connection from a
machine that REALLY IS a-b.xyz, it still would have failed.  a-b.xyz
resolves to 178.63.42.176.  rDNS for that address is 'ten.a-b.xyz' --
which means even if everything else were on the level, this test would
still have failed (albeit with a different rule violation).

Generally speaking, a lot of these kinds of errors occur when someone
has put SpamAssassin too late in the incoming-mail processing pipeline,
which can cause it to process all kinds of headers that MDAs should care
about and SpamAssassin should not.  For instance, if an MTA receives a
message, and hands it off to the MDA, and the MDA is configured to
bounce that mail to another MTA, you wind up with really weird headers
that occur twice or don't match up with other headers, sometimes leading
to RFC violations, which the secondary MTA runs through SpamAssassin,
marks as spam, then hands off to an MDA.

In other words,  that SpamAssassin instance is misconfigured.  Exactly
how is a fun detective puzzle where we have to extrapolate from absent
logs and headers.

khm


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-21 18:42           ` Stanley Lieber
  2020-09-21 21:40             ` hiro
@ 2020-09-22 12:57             ` Ethan Gardener
  2020-09-22 13:12               ` hiro
  1 sibling, 1 reply; 26+ messages in thread
From: Ethan Gardener @ 2020-09-22 12:57 UTC (permalink / raw)
  To: 9front

FYI, Gnu.org changed its mailing lists to not munge in October last year. (The explanation was approximately 10 times as long as this entire thread but not as clear, IMO. ;) The default for Mailman was also changed at that time.

I'm a little bit in favour of just not munging, but that's not a deeply considered opinion. I think it would only impact people who don't filter their mail (understandable?) and people who filter on subject (bad practice?).


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: [9front] test
  2020-09-22 12:57             ` Ethan Gardener
@ 2020-09-22 13:12               ` hiro
  0 siblings, 0 replies; 26+ messages in thread
From: hiro @ 2020-09-22 13:12 UTC (permalink / raw)
  To: 9front

thankfully we need not decide (yet) :)


^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-21 20:33   ` Kurt H Maier
  2020-09-21 21:23     ` kvik
@ 2020-09-22 18:57     ` Stefan Hertenberger
  2020-09-22 19:38       ` Kurt H Maier
  1 sibling, 1 reply; 26+ messages in thread
From: Stefan Hertenberger @ 2020-09-22 18:57 UTC (permalink / raw)
  To: 9front

Am Mon, 21 Sep 2020 13:33:00 -0700
schrieb Kurt H Maier <khm@sciops.net>:

> Your SpamAssassin installation is misconfigured.  Notes inline.
> 
> 
> On Mon, Sep 21, 2020 at 07:54:29PM +0200, Stefan Hertenberger wrote:
> > > 0.9 SPF_FAIL               SPF: sender does not match SPF record
> > > 0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
> > > not necessarily valid 0.1 DKIM_INVALID           DKIM or DK
> > > signature exists, but is not valid 
> 
> These are caused by invalid forwarding confix on a-b.xyz.  It needs to
> strip DKIM and rewrite From: if it's going to behave like this.  I
> suspect but cannot prove that migadu is adding the dkim signatures,
> which then don't match the Fron: line since 9front.org mail doesn't
> come from migadu.
> 
> > >  2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD
> > >  1.3 RDNS_NONE              Delivered to internal network by a
> > > host with no rDNS 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
> 
> These are the majority of the weights causing the SPAM tag from SA,
> and again it's because xyz is a spam hotbed and also because
> a-b.xyz's IP address reverse-resolves to ten.a-b.xyz.  
> 
> If you intend to continue receiving 9front mail through this domain,
> it's probably simplest to whitelist the domain in your sa-spamd rules,
> since nothing sl does to the 9front list will change any of these
> things.  Dropping a 'whitelist_from_rcvd *@9front.org a-b.xyz' into
> your spamassassin rules may do it, but without seeing the full
> headers of the as-delivered message I can't be sure.
> 
> khm

Hello,

sorry for the late reply! alarum.de is my personal playground, so a
misconfiguration is possible.

Here is the complete source for the email.


Return-Path: <9front-bounces@ewsd.inri.net>
X-Original-To: stefan@alarum.de
Delivered-To: stefan@alarum.de
Received: from ewsd.inri.net (ewsd.inri.net [107.191.116.128])
	by alarum.de (Postfix) with ESMTP id 4FFC71B8D67
	for <stefan@alarum.de>; Mon, 21 Sep 2020 17:41:37 +0200 (CEST)
Received: from out0.migadu.com ([94.23.1.103]) by ewsd; Mon Sep 21
11:40:32 EDT 2020 Message-ID: <1E3083B627E4EDD1DF08AA61E95E4DA2@a-b.xyz>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=a-b.xyz; s=key1;
	t=1600702823;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=InrW//k9HspT0nLKA/dQBT2lattjYFTpecCGszIrIiY=;
	b=couXgEp+X7wJBDGXNfssrxirjlVxqS5+SFgUFvN47oWZZKlkw6M4iZl1pkh42DE+T/DkgS
	AfbdwbJKX88GnoJcQGgwnb+JMcq+WjTznq/guqIvl3mGx4t8QIu+2KJQQAPegF7P9eZIAW
	e4cTtZQoA2VpyX1v4SO5OuWX22vnHPE60TwBUcNtOGoJPAAEJMbF0LXcGt5dU1/3Txl54g
	xMfYxrxkZtbs8shWEzqw+Fr6wNM39K4E8SVX2YXPlgHONj32rkqvbea7SBzOEA7TiDS0vf
	X+qWVL+97pLy9Vo1SivCeOqR519dU9tWY+qcEaUg62MKTmYPAPE4Bzr3oRNYnw==
To: 9front@9front.org
Date: Mon, 21 Sep 2020 17:40:20 +0200
X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and
include these headers. From: kvik@a-b.xyz
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="----------=_5F68C9B3.6CD1B719"
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.10
List-ID: <9front.9front.org>
List-Help: <http://lists.9front.org>
X-Glyph: ➈
X-Bullshit: virtualized metadata metadata-based dependency-aware
backend Subject: *****SPAM***** [9front] test
Reply-To: 9front@9front.org
Precedence: bulk
X-Spam-Flag: YES
X-Spam-Status: Yes, score=5.1 required=5.0
tests=DKIM_INVALID,DKIM_SIGNED,
FROM_SUSPICIOUS_NTLD,FROM_SUSPICIOUS_NTLD_FP,
HEADER_FROM_DIFFERENT_DOMAINS,RCVD_IN_MSPIKE_H4,RCVD_IN_MSPIKE_WL,
RDNS_NONE,SPF_FAIL,SPF_HELO_NONE,T_PDS_OTHER_BAD_TLD,UNPARSEABLE_RELAY,
URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.4
X-Spam-Level: ***** X-Spam-Checker-Version: SpamAssassin 3.4.4
(2020-01-24) on vbsd.alarum.de

This is a multi-part message in MIME format.

------------=_5F68C9B3.6CD1B719
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

Spam detection software, running on the system "vbsd.alarum.de",
has identified this incoming email as possible spam.  The original
message has been attached to this so you can view it or label
similar future email.  If you have any questions, see
postmaster for details.

Content preview:  I've changed DMARC policy to 'none'. Let's see if
this mail still gets to spam. P.S. Sorry for spamming the list. 

Content analysis details:   (5.1 points, 5.0 required)

 pts rule name              description
---- ----------------------
 -------------------------------------------------- -0.0
 RCVD_IN_MSPIKE_H4      RBL: Very Good reputation (+4) [94.23.1.103
 listed in wl.mailspike.net] 0.0 URIBL_BLOCKED          ADMINISTRATOR
 NOTICE: The query to URIBL was blocked.  See
                            http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
                             for more information.
                            [URIs: a-b.xyz]
 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level
                            mail domains are different
 0.0 T_PDS_OTHER_BAD_TLD    Untrustworthy TLDs
                            [URI: a-b.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.9 SPF_FAIL               SPF: sender does not match SPF record (fail)
[SPF failed: Please see
 http://www.openspf.org/Why?s=mfrom;id=9front-bounces%40ewsd.inri.net;ip=94.23.1.103;r=vbsd.alarum.de]
 0.1 DKIM_SIGNED            Message has a DKIM or DK signature, not
 necessarily valid 0.1 DKIM_INVALID           DKIM or DK signature
 exists, but is not valid 0.0 UNPARSEABLE_RELAY      Informational:
 message has unparseable relay lines
-0.0 RCVD_IN_MSPIKE_WL      Mailspike good senders
 2.0 FROM_SUSPICIOUS_NTLD_FP From abused NTLD
 1.3 RDNS_NONE              Delivered to internal network by a host
 with no rDNS 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD



------------=_5F68C9B3.6CD1B719
Content-Type: message/rfc822; x-spam-type=original
Content-Description: original message before SpamAssassin
Content-Disposition: inline
Content-Transfer-Encoding: 8bit

X-Envelope-From: <9front-bounces@ewsd.inri.net>
X-Envelope-To: <stefan@alarum.de>
Received: from ewsd.inri.net (unknown)
	by alarum.de(Postfix 3.1.4/8.13.0) with SMTP id unknown;
	Mon, 21 Sep 2020 17:41:37 +0200
	(envelope-from <9front-bounces@ewsd.inri.net>)
Received: from out0.migadu.com ([94.23.1.103]) by ewsd; Mon Sep 21
11:40:32 EDT 2020 Message-ID: <1E3083B627E4EDD1DF08AA61E95E4DA2@a-b.xyz>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=a-b.xyz; s=key1;
	t=1600702823;
	h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
	 to:to:cc:mime-version:mime-version:content-type:content-type:
	 content-transfer-encoding:content-transfer-encoding;
	bh=InrW//k9HspT0nLKA/dQBT2lattjYFTpecCGszIrIiY=;
	b=couXgEp+X7wJBDGXNfssrxirjlVxqS5+SFgUFvN47oWZZKlkw6M4iZl1pkh42DE+T/DkgS
	AfbdwbJKX88GnoJcQGgwnb+JMcq+WjTznq/guqIvl3mGx4t8QIu+2KJQQAPegF7P9eZIAW
	e4cTtZQoA2VpyX1v4SO5OuWX22vnHPE60TwBUcNtOGoJPAAEJMbF0LXcGt5dU1/3Txl54g
	xMfYxrxkZtbs8shWEzqw+Fr6wNM39K4E8SVX2YXPlgHONj32rkqvbea7SBzOEA7TiDS0vf
	X+qWVL+97pLy9Vo1SivCeOqR519dU9tWY+qcEaUg62MKTmYPAPE4Bzr3oRNYnw==
To: 9front@9front.org
Date: Mon, 21 Sep 2020 17:40:20 +0200
X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and
include these headers. From: kvik@a-b.xyz
MIME-Version: 1.0
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.10
List-ID: <9front.9front.org>
List-Help: <http://lists.9front.org>
X-Glyph: ➈
X-Bullshit: virtualized metadata metadata-based dependency-aware
backend Subject: [9front] test
Reply-To: 9front@9front.org
Precedence: bulk

I've changed DMARC policy to 'none'.

Let's see if this mail still gets to spam.

P.S. Sorry for spamming the list.

------------=_5F68C9B3.6CD1B719--



^ permalink raw reply	[flat|nested] 26+ messages in thread

* Re: *****SPAM***** [9front] test
  2020-09-22 18:57     ` Stefan Hertenberger
@ 2020-09-22 19:38       ` Kurt H Maier
  0 siblings, 0 replies; 26+ messages in thread
From: Kurt H Maier @ 2020-09-22 19:38 UTC (permalink / raw)
  To: 9front

This is where the confusion is coming from.  Notes inline.

On Tue, Sep 22, 2020 at 08:57:21PM +0200, Stefan Hertenberger wrote:
> Return-Path: <9front-bounces@ewsd.inri.net>
> X-Original-To: stefan@alarum.de
> Delivered-To: stefan@alarum.de
> Received: from ewsd.inri.net (ewsd.inri.net [107.191.116.128])
> 	by alarum.de (Postfix) with ESMTP id 4FFC71B8D67
> 	for <stefan@alarum.de>; Mon, 21 Sep 2020 17:41:37 +0200 (CEST)

This is the Received header that alarum SpamAssassin should be looking
at.

> Received: from out0.migadu.com ([94.23.1.103]) by ewsd; Mon Sep 21
> 11:40:32 EDT 2020 Message-ID: <1E3083B627E4EDD1DF08AA61E95E4DA2@a-b.xyz>

This is the one it's actually looking at, which is why it's blaming
migadu for all kinds of shenanigans.  I don't know why it's choosing the
wrong one, but the fact that the second line is incorrectly folded may
be causing the system to read the header as ending after "Mon Sep 21".
Proper folding would have immediately followed the linebreak with a
folding-whitespace character (ascii space or tab). 

> X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and
> include these headers. From: kvik@a-b.xyz

Same thing here.  Note the embedded header in this header.  Valid, but
could confuse software in this case, especially as I see no sign of an
actual From: header in the rest of the message.  The improper folding
might be causing SA to identify this as the From: header.

> X-Bullshit: virtualized metadata metadata-based dependency-aware
> backend Subject: *****SPAM***** [9front] test

More improper folding, leading to the Subject: line being munged.  

Something is mangling messages before they're getting handed to
SpamAssassin.  The copy I received on my mailserver was not mangled, so
I must presume it's something on vbsd.alarum.de.  The fact that the SPF
testing is in the SA report but not set as headers tells me it's likel
that the SA run was done after receipt of the message instead of during
the SMTP handshake, so the problem is somewhere between your MTA and
your SpamAssassin run.

The absent From: header combined with the mangled X-Abuse: header 
probably tricked SA into applying the NTLD rules for .xyz.  That
problem would be fixed alongside the other mangling-based problems, but
I really do recommend turning those rules off anyway, or at least
whitelisting TLDs you know people are legitimately using.

Hope this helps,
khm


^ permalink raw reply	[flat|nested] 26+ messages in thread

end of thread, other threads:[~2020-09-22 19:38 UTC | newest]

Thread overview: 26+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-21 15:40 test kvik
2020-09-21 17:12 ` [9front] test Stanley Lieber
2020-09-21 17:16   ` hiro
2020-09-21 17:36     ` ori
2020-09-21 17:40       ` Stanley Lieber
2020-09-21 17:51         ` ori
2020-09-21 17:59           ` ori
2020-09-21 18:27           ` Kurt H Maier
2020-09-21 17:58         ` hiro
2020-09-21 18:42           ` Stanley Lieber
2020-09-21 21:40             ` hiro
2020-09-21 22:20               ` hiro
2020-09-22 12:57             ` Ethan Gardener
2020-09-22 13:12               ` hiro
2020-09-21 17:59         ` Lyndon Nerenberg
2020-09-21 17:54 ` *****SPAM***** " Stefan Hertenberger
2020-09-21 20:33   ` Kurt H Maier
2020-09-21 21:23     ` kvik
2020-09-21 21:36       ` Kurt H Maier
2020-09-21 21:42         ` kvik
2020-09-21 21:57           ` Kurt H Maier
2020-09-21 22:32             ` kvik
2020-09-22  1:04               ` Kurt H Maier
2020-09-21 22:16           ` hiro
2020-09-22 18:57     ` Stefan Hertenberger
2020-09-22 19:38       ` Kurt H Maier

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).