patch smtp: ignore unrecognized certificates

patch smtp: ignore unrecognized certificates

[Joel Fridolin Meyer]

[-- Attachment #1: Type: text/plain, Size: 1533 bytes --]

(This is a duplicate mail, because I'm not sure if the original mail from
my mail server is delivered correctly.)

Hello devs,

this patch adds a new flag -c to upas/smtp command ( smtp(8) ). This flag
removes the need to manually add thumbprints for tls certificates to
/sys/lib/tls/smtp, ignoring unrecognized certificates.

Thumbprints for unrecognized certificates will still be printed to
/sys/log/smtp, the message will still be sent.

If you want to add this to 9front, I'll be happily adjust the usage message
as well as the man page.

diff -r f8f63e944375 sys/src/cmd/upas/smtp/smtp.c
--- a/sys/src/cmd/upas/smtp/smtp.c  Fri Jul 17 16:53:20 2020 +0200
+++ b/sys/src/cmd/upas/smtp/smtp.c  Sun Jul 19 11:00:15 2020 +0200
@@ -46,6 +46,7 @@
int  quitting; /* when error occurs in quit */
int  tryauth;  /* Try to authenticate, if supported */
int  trysecure;  /* Try to use TLS if the other side supports it */
+int nocertcheck; /* ignore unrecognized certs. Still logged */

char *quitrv;  /* deferred return value when in quit */
char ddomain[1024];  /* domain name of destination machine */
@@ -185,6 +186,9 @@
case 'u':
user = EARGF(usage());
break;
+ case 'c':
+   nocertcheck = 1;
+   break;
default:
usage();
break;
@@ -420,7 +424,9 @@
}
if (!okCertificate(c->cert, c->certlen, goodcerts)) {
syslog(0, "smtp", "cert for %s not recognized: %r", ddomain);
-   goto Out;
+   if (!nocertcheck)
+     goto Out;
+   syslog(0, "smtp", "ignoring unrecognized cert");
}
syslog(0, "smtp", "started TLS to %q", ddomain);
err = nil;

[-- Attachment #2: Type: text/html, Size: 2036 bytes --]

Re: [9front] patch smtp: ignore unrecognized certificates

[ori]

> this patch adds a new flag -c to upas/smtp command ( smtp(8) ). This flag
> removes the need to manually add thumbprints for tls certificates to
> /sys/lib/tls/smtp, ignoring unrecognized certificates.

I think this is a bad idea -- I'd prefer to make it more obvious how
to add the certificate in the first place. Possibly a upas/configmail
command that prompts for the server/login/..., and then gets the
thumbnail.

Ignoring the thumbprint entirely opens you up to MITM attacks.

Re: [9front] patch smtp: ignore unrecognized certificates

[hiro]

isn't the issue that the cert keeps on randomly changing on server
side and thus making this "security" completely unachievable?

i maintain a similar patch for dillo so that i can visit https-only
websites that aren't worth protecting with SSL anyway.

whoever manually adds the certificates sometimes (Ori, do you?), do
you really check their validity?

once CAs are involved there's a lot of problems, and I doubt 9front is
in the position to change anything about that. that's why we have
dp9ik at least for our protocols and services :)

On 7/20/20, ori@eigenstate.org <ori@eigenstate.org> wrote:
>> this patch adds a new flag -c to upas/smtp command ( smtp(8) ). This flag
>> removes the need to manually add thumbprints for tls certificates to
>> /sys/lib/tls/smtp, ignoring unrecognized certificates.
>
> I think this is a bad idea -- I'd prefer to make it more obvious how
> to add the certificate in the first place. Possibly a upas/configmail
> command that prompts for the server/login/..., and then gets the
> thumbnail.
>
> Ignoring the thumbprint entirely opens you up to MITM attacks.
>
>

Re: [9front] patch smtp: ignore unrecognized certificates

[ori]

> whoever manually adds the certificates sometimes (Ori, do you?), do
> you really check their validity?

I run my own mail server, so the certificates don't change. I assume
that the first time I connect, it's valid.

How often do these certificates change?

Re: [9front] patch smtp: ignore unrecognized certificates

[tlaronde]

On Mon, Jul 20, 2020 at 07:26:08AM -0700, ori@eigenstate.org wrote:
> > whoever manually adds the certificates sometimes (Ori, do you?), do
> > you really check their validity?
> 
> I run my own mail server, so the certificates don't change. I assume
> that the first time I connect, it's valid.
> 
> How often do these certificates change?

It can be quite often. I have a certificate made via cacert.org for
example. And I have to renew it regularily. I think that
certificates with too long TTL are considered weak. So it's an
annoyance.
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9front] patch smtp: ignore unrecognized certificates

[ori]

> It can be quite often. I have a certificate made via cacert.org for
> example. And I have to renew it regularily. I think that
> certificates with too long TTL are considered weak. So it's an
> annoyance.

Hm. I see. Without some sort of check, anyone with a certificate
can MITM. I wonder how crazy it'd be to implement certificate
chain verification.

Re: [9front] patch smtp: ignore unrecognized certificates

[tlaronde]

On Mon, Jul 20, 2020 at 07:52:04AM -0700, ori@eigenstate.org wrote:
> > It can be quite often. I have a certificate made via cacert.org for
> > example. And I have to renew it regularily. I think that
> > certificates with too long TTL are considered weak. So it's an
> > annoyance.
> 
> Hm. I see. Without some sort of check, anyone with a certificate
> can MITM. I wonder how crazy it'd be to implement certificate
> chain verification.

The problem is that if there is a chain, you need to trust the root you
finally end with. So whether this is a volunteers based C.A. or a
professional paying one. It's kind of squaring the circle. The solution
is to maintain one's network with an iron fist in a lead glove (because
lead is malleable so you can hit heavily while protecting the iron) and
to not trust anything outside and to not have to care about how much
untrustable is outside... Well, this generally means kicking out
users...
-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C

Re: [9front] patch smtp: ignore unrecognized certificates

[ori]

> The problem is that if there is a chain, you need to trust the root you
> finally end with. So whether this is a volunteers based C.A. or a
> professional paying one.

Yes -- it'd mean distributing the usual set of trusted root certs
with the OS. Probably lifting /etc/ssl/cert.pem from a Unix box.

It also sucks, but it's a problem everyone else seems to have solved.

Re: [9front] patch smtp: ignore unrecognized certificates

[hiro]

> a problem everyone else seems to have solved.
you call that solved?!

i think it's the opposite, as it makes it worse giving people false
sense of security.

On 7/20/20, ori@eigenstate.org <ori@eigenstate.org> wrote:
>> The problem is that if there is a chain, you need to trust the root you
>> finally end with. So whether this is a volunteers based C.A. or a
>> professional paying one.
>
> Yes -- it'd mean distributing the usual set of trusted root certs
> with the OS. Probably lifting /etc/ssl/cert.pem from a Unix box.
>
> It also sucks, but it's a problem everyone else seems to have solved.
>
>

Re: [9front] patch smtp: ignore unrecognized certificates

[cinap_lenrek]

thats why you really want to pin the key, not the cert. this
is what we do at the moment. the thumbprint is a hash of the
public key, not the certificate. so the cert can be renewed
without changing the key.

	if(X509digestSPKI(cert, len, sha2_256, hash) < 0)
		return 0;
	if(okThumbprint(hash, SHA2_256dlen, table))
		return 1;

--
cinap

Re: [9front] patch smtp: ignore unrecognized certificates

[cinap_lenrek]

you need to differentiate some use cases.

for servers like your isp's or your networks forwarding mailserver
(machines that you have direct relation or control over), it makes
sense to pin certificates or public key. this is where you use
smtps, instead of this insecure STARTTLS contraption.

but on the internet, nobody is really checking the cert as theres no
root of trust. last time i checked, you could put a self signed cert
in your smtp server and you'd still get mail as theres not really
a common root of trust. this is different from webbrowsers where
the root of trust is controlled by the browser vendo^Wgoogle.

smtp is just opportunistic encrytion. hell, starttls can be attacked
by mitm by just stripping the command with a proxy to sabotage the
tls negotiation.

it is just best effort to at least encrypt the data, but it will not
hold against an adversary that has active mitm capability. to it
protects from passive sniffing...

mayyyybe, if we had dnssec and put the public key hash in the dns
then we could have mitm protected email transport. but its not
working very well. people whine that they dont want to trust the
dns (fair enougth) and then use letsencrypt instead which completely
100% relies on dns for the renewal...

in conclusion, it makes sense to me to add a flag to smtp client
for opportunistic encryption that skips certificate verification.
(you'd control it in your remotemail script depending on the
destination, so you can still pin the keys for sites that you
trust).

tho please dont use -c flag. its a negative, so should have
a capital letter. and we should also skip loading the tumbprint
at all, as it makes no sense to try to check anything in this case.

--
cinap

Re: [9front] patch smtp: ignore unrecognized certificates

[Joel Fridolin Meyer]

Hello again.

I read through your responses and will adjust the patch (use -C instead of -c).

About no thumbprint loading: I left it in on purpose, so that admins
still get the notice and can track connections etc. For example admins
can see what connections were made and manually add them to the file
if they want to enable security checks at some point.

I can remove loading of thumbprints at all, I just wanted to give you
my thoughts.

Also we could remove the goto here, but I guess I need some hint,
because somehow my brain didn't want to think properly when I looked
at the code.

sirjofri

2020-07-20 18:18 GMT+02:00, cinap_lenrek@felloff.net <cinap_lenrek@felloff.net>:
> you need to differentiate some use cases.
>
> for servers like your isp's or your networks forwarding mailserver
> (machines that you have direct relation or control over), it makes
> sense to pin certificates or public key. this is where you use
> smtps, instead of this insecure STARTTLS contraption.
>
> but on the internet, nobody is really checking the cert as theres no
> root of trust. last time i checked, you could put a self signed cert
> in your smtp server and you'd still get mail as theres not really
> a common root of trust. this is different from webbrowsers where
> the root of trust is controlled by the browser vendo^Wgoogle.
>
> smtp is just opportunistic encrytion. hell, starttls can be attacked
> by mitm by just stripping the command with a proxy to sabotage the
> tls negotiation.
>
> it is just best effort to at least encrypt the data, but it will not
> hold against an adversary that has active mitm capability. to it
> protects from passive sniffing...
>
> mayyyybe, if we had dnssec and put the public key hash in the dns
> then we could have mitm protected email transport. but its not
> working very well. people whine that they dont want to trust the
> dns (fair enougth) and then use letsencrypt instead which completely
> 100% relies on dns for the renewal...
>
> in conclusion, it makes sense to me to add a flag to smtp client
> for opportunistic encryption that skips certificate verification.
> (you'd control it in your remotemail script depending on the
> destination, so you can still pin the keys for sites that you
> trust).
>
> tho please dont use -c flag. its a negative, so should have
> a capital letter. and we should also skip loading the tumbprint
> at all, as it makes no sense to try to check anything in this case.
>
> --
> cinap
>

Re: [9front] patch smtp: ignore unrecognized certificates

[Steffen Nurpmeso]

cinap_lenrek@felloff.net wrote in
<BC75BC74DEF132FCD6A0F1C7E3D9FCEC@felloff.net>:
 |
 |for servers like your isp's or your networks forwarding mailserver
 |(machines that you have direct relation or control over), it makes
 |sense to pin certificates or public key. this is where you use
 |smtps, instead of this insecure STARTTLS contraption.

In November 2019 they waved through RFC 8689:

                        SMTP Require TLS Option

  Abstract

   The SMTP STARTTLS option, used in negotiating transport-level
   encryption of SMTP connections, is not as useful from a security
   standpoint as it might be because of its opportunistic nature;
   message delivery is, by default, prioritized over security.  This
   document describes an SMTP service extension, REQUIRETLS, and a
   message header field, TLS-Required.  If the REQUIRETLS option or TLS-
   Required message header field is used when sending a message, it
   asserts a request on the part of the message sender to override the
   default negotiation of TLS, either by requiring that TLS be
   negotiated when the message is relayed or by requesting that
   recipient-side policy mechanisms such as MTA-STS and DNS-Based
   Authentication of Named Entities (DANE) be ignored when relaying a
   message for which security is unimportant.

P.S.: several times i tried to contact Stanley Lieber in the past,
but never got a response.  If this mail gets through (quite some
did not in the past), would you mind changing my address to
steffen@@sdaoden.eu, please?  Thank you!!

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [9front] patch smtp: ignore unrecognized certificates

[ori]

> In November 2019 they waved through RFC 8689:

Nice, with luck we'll be able to turn it on by 2029 :)

Re: [9front] patch smtp: ignore unrecognized certificates

[Kurt H Maier]

On Mon, Jul 20, 2020 at 03:05:15PM -0700, ori@eigenstate.org wrote:
> > In November 2019 they waved through RFC 8689:
> 
> Nice, with luck we'll be able to turn it on by 2029 :)
> 

It is a failure of the IETF process that this sort of crap (which
recommends MTA-STS) is shoved through while taking extremely simple
steps like "standardizing an SMTP+TLS port" is regarded as impossible.
I suggest the current approach (manually confirming fingerprints or else
declaring them irrelevant) is superior in just about every way.

khm

Re: [9front] patch smtp: ignore unrecognized certificates

[Steffen Nurpmeso]

Kurt H Maier wrote in
<20200720221909.GB55786@wopr>:
 |On Mon, Jul 20, 2020 at 03:05:15PM -0700, ori@eigenstate.org wrote:
 |>> In November 2019 they waved through RFC 8689:
 |> 
 |> Nice, with luck we'll be able to turn it on by 2029 :)
 |
 |It is a failure of the IETF process that this sort of crap (which
 |recommends MTA-STS) is shoved through while taking extremely simple
 |steps like "standardizing an SMTP+TLS port" is regarded as impossible.
 |I suggest the current approach (manually confirming fingerprints or else
 |declaring them irrelevant) is superior in just about every way.

Yes.  Luckily "SUBMISSIONS" on 465 is finally also standardized.

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [9front] patch smtp: ignore unrecognized certificates

[ori]

> P.S.: several times i tried to contact Stanley Lieber in the past,
> but never got a response.  If this mail gets through (quite some
> did not in the past), would you mind changing my address to
> steffen@@sdaoden.eu, please?  Thank you!!

Also -- you should be able to just resubscribe, no?

Re: [9front] patch smtp: ignore unrecognized certificates

[Steffen Nurpmeso]

ori@eigenstate.org wrote in
<8ACD8978E36CCF9C48952CB2FEE6557C@eigenstate.org>:
 |> P.S.: several times i tried to contact Stanley Lieber in the past,
 |> but never got a response.  If this mail gets through (quite some
 |> did not in the past), would you mind changing my address to
 |> steffen@@sdaoden.eu, please?  Thank you!!
 |
 |Also -- you should be able to just resubscribe, no?

This seems to have worked.  (Unsubscription i do not know yet.)

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: [9front] patch smtp: ignore unrecognized certificates

[sirjofri+ml-9front]

[-- Attachment #1: Type: text/plain, Size: 219 bytes --]

Hey all,

I prepared a better patch, this time including a line in the man page and an adjusted usage line. Also this time I don't do the certificate checks, all certs are ignored. This also changes -c to -C.

sirjofri

[-- Attachment #2: Type: text/plain, Size: 1699 bytes --]

diff -r 639ad985a75b sys/man/8/smtp
--- a/sys/man/8/smtp	Mon Jul 20 18:58:52 2020 -0700
+++ b/sys/man/8/smtp	Wed Jul 29 09:24:42 2020 +0200
@@ -6,7 +6,7 @@
 .ti -0.5i
 .B upas/smtp
 [
-.B -aAdfipst
+.B -aACdfipst
 ] [
 .B -b
 .I busted-mx
@@ -81,6 +81,9 @@
 when trying MX hosts.
 May be repeated.
 .TP
+.B -C
+ignore bad thumbprints for TLS connections.
+.TP
 .B -d
 turn on debugging to standard error.
 .TP
diff -r 639ad985a75b sys/src/cmd/upas/smtp/smtp.c
--- a/sys/src/cmd/upas/smtp/smtp.c	Mon Jul 20 18:58:52 2020 -0700
+++ b/sys/src/cmd/upas/smtp/smtp.c	Wed Jul 29 09:24:42 2020 +0200
@@ -46,6 +46,7 @@
 int	quitting;	/* when error occurs in quit */
 int	tryauth;	/* Try to authenticate, if supported */
 int	trysecure;	/* Try to use TLS if the other side supports it */
+int	nocertcheck; /* ignore unrecognized certs. Still logged */
 
 char	*quitrv;	/* deferred return value when in quit */
 char	ddomain[1024];	/* domain name of destination machine */
@@ -85,7 +86,7 @@
 void
 usage(void)
 {
-	fprint(2, "usage: smtp [-aAdfipst] [-b busted-mx] [-g gw] [-h host] "
+	fprint(2, "usage: smtp [-aACdfipst] [-b busted-mx] [-g gw] [-h host] "
 		"[-u user] [.domain] net!host[!service] sender rcpt-list\n");
 	exits(Giveup);
 }
@@ -185,6 +186,9 @@
 	case 'u':
 		user = EARGF(usage());
 		break;
+	case 'C':
+		nocertcheck = 1;
+		break;
 	default:
 		usage();
 		break;
@@ -413,6 +417,12 @@
 	Bterm(&bin);
 	Binit(&bin, fd, OREAD);
 
+	if (nocertcheck) {
+		syslog(0, "smtp", "ignoring cert for %s", ddomain);
+		err = nil;
+		goto Out;
+	}
+
 	goodcerts = initThumbprints(smtpthumbs, smtpexclthumbs, "x509");
 	if (goodcerts == nil) {
 		syslog(0, "smtp", "bad thumbprints in %s", smtpthumbs);

Re: [9front] patch smtp: ignore unrecognized certificates

[hiro]

you see no risk the syslog gets spammed too much with this? it would
happen for nearly every mail, no?

Re: [9front] patch smtp: ignore unrecognized certificates

[cinap_lenrek]

good, except i dont think theres a point in logging when we explicitely
disabled the cert check.

--
cinap