trying to deal with an smtp server that wants encryption

trying to deal with an smtp server that wants encryption

[A.J. Rossini]

Tested environments:

Gnus 5.10.6 (Debian), Emacs;  No Gnus v0.2, XEmacs

starttls-0.9 (compiled from source),  gnutls-bin (Debian/Sid 1.0.4)

Okay, I've been battling with this for 2 hours now.  Kmail took me 3
minutes, but I really don't want to go there.

I'm trying to send mail from a remote site (my home) using my school's
smtp server.  

According to what I can find on the WWW, I need to do something like:


(require 'smtpmail)
(setq smtpmail-auth-credentials
      '(("smtp.washington.edu"
       "25"
        "myusername"
        "mypassword")))


(setq smtpmail-starttls-credentials
      '(("smtp.washington.edu"
         25 "" "" )))
; I've tried the following self-signed certs:
;        "/home/rossini/Certs/mykey.pem"
;        "/home/rossini/Certs/mycert.pem"

(setq smtpmail-smtp-server "smtp.washington.edu")

;; If you use the default mail user agent and message
(setq send-mail-function 'smtpmail-send-it)
(setq message-send-mail-function 'smtpmail-send-it)

;; send mail debug output to a buffer
(setq smtpmail-debug-info t)


(the *.pem files (key and cert) self-cert'd, I don't think I need
them?)

As far as I'm aware, I can use either starttls that comes with gnus in
the lisp directory, with the starttls 0.9 binary, OR I can use the one
in the contrib directory, with gnutls-cli.  

Using gnutls-cli, I actually get feedback in the *trace* file that I'm
doing something right, but it times out and closes.  Using starttls,
it never really starts the TLS handshake.

Any thoughts or assistance would be much appreciated!

best,
-tony

-- 
rossini@u.washington.edu            http://www.analytics.washington.edu/ 
Biomedical and Health Informatics   University of Washington
Biostatistics, SCHARP/HVTN          Fred Hutchinson Cancer Research Center
UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable
FHCRC  (M/W): 206-667-7025 FAX=206-667-4812 | use Email

CONFIDENTIALITY NOTICE: This e-mail message and any attachments may be
confidential and privileged. If you received this message in error,
please destroy it and notify the sender. Thank you.

Re: trying to deal with an smtp server that wants encryption

[Simon Josefsson]

> (the *.pem files (key and cert) self-cert'd, I don't think I need
> them?)

Right.  Unless the server require them, or you want to use X.509
authentication (but keep in mind that starttls '-verify' doesn't really do
anything, it doesn't verify the server certificate, AFACT).

> Using gnutls-cli, I actually get feedback in the *trace* file that I'm
> doing something right, but it times out and closes.  Using starttls,
> it never really starts the TLS handshake.

Can you post the output in the *trace... buffer?

Also try 'gnutls-cli -s mailserver -p 25' and type 'STARTTLS\n' followed
by ^D to initiate TLS, followed by 'EHLO foo', to verify that gnutls-bin
works.

Re: trying to deal with an smtp server that wants encryption

[A.J. Rossini]

"Simon Josefsson" <jas@extundo.com> writes:

>> (the *.pem files (key and cert) self-cert'd, I don't think I need
>> them?)
>
> Right.  Unless the server require them, or you want to use X.509
> authentication (but keep in mind that starttls '-verify' doesn't really do
> anything, it doesn't verify the server certificate, AFACT).
>
>> Using gnutls-cli, I actually get feedback in the *trace* file that I'm
>> doing something right, but it times out and closes.  Using starttls,
>> it never really starts the TLS handshake.
>
> Can you post the output in the *trace... buffer?

Done

> Also try 'gnutls-cli -s mailserver -p 25' and type 'STARTTLS\n' followed
> by ^D to initiate TLS, followed by 'EHLO foo', to verify that gnutls-bin
> works.

and done.


500$ gnutls-cli -s smtp.washington.edu -p 25
Resolving 'smtp.washington.edu'...
Connecting to '140.142.33.9:25'...

- Simple Client Mode:

220 smtp.washington.edu ESMTP Sendmail
8.12.10+UW03.09/8.12.10+UW03.09; Thu, 22 Jan 2004 23:23:00 -0800
STARTTLS
220 2.0.0 Ready to start TLS
*** Starting TLS handshake
- Certificate type: X.509
 - Got a certificate list of 1 certificates.

 - Certificate[0] info:
 # The hostname in the certificate matches 'smtp.washington.edu'.
 # valid since: Thu Jan 23 12:44:00 PST 2003
 # expires at: Sun Feb  8 15:10:00 PST 2004
 # serial number: 09 d4 97
 # fingerprint: 63 12 43 fc 7d 49 f1 c0 5d 75 99 0a 7b de 4d 51
 # version: #3
 # public key algorithm: RSA
 #   Modulus: 1024 bits
 # Subject's DN: C=US,ST=Washington,L=Seattle,O=University of
Washington,OU=NDC,CN=smtp.washington.edu
 # Issuer's DN: C=ZA,ST=Western Cape,L=Cape Town,O=Thawte Consulting
cc,OU=Certification Services Division,CN=Thawte Server
CA,EMAIL=server-certs@thawte.com


- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS 1.0
- Key Exchange: RSA
- Cipher: ARCFOUR 128
- MAC: SHA
- Compression: NULL
EHLO foo
250-smtp.washington.edu Hello sense-sea-MegaSub-2-231.oz.net
[216.39.172.231], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-EXPN
250-VERB
250-8BITMIME
250-SIZE 60000000
250-ETRN
250-AUTH GSSAPI PLAIN LOGIN
250-DELIVERBY
250 HELP



So it looks good so far.   BUT, with gnutls-cli, I'm getting a
*Backtrace*:



Signaling: (invalid-regexp "Invalid preceding regular expression")
  re-search-forward("*** Handshake has failed" nil t)
  starttls-negotiate(#<process "SMTP" pid 19272 state:run>)
  byte-code("..." [host n name supported-extensions process
response-code get-buffer-create format "*trace of SMTP session to %s*"
erase-buffer smtpmail-open-stream throw done nil set-process-filter
smtpmail-process-filter featurep mule file-coding
set-process-coding-system no-conversion-unix make-local-variable
smtpmail-read-point smtpmail-read-response 400 t smtpmail-send-command
"EHLO %s" smtpmail-fqdn "HELO %s" mapcar #<compiled-function (s)
"...(5)" [s intern] 2> split-string 4 "[ ]" 1 (verb xvrb 8bitmime onex
xone expn size dsn etrn enhancedstatuscodes help xusr auth=login auth
starttls) message "Unknown extension %s" smtpmail-find-credentials
starttls process-id "STARTTLS" starttls-negotiate
smtpmail-try-auth-methods onex xone "ONEX" verb xvrb ...] 8)
  smtpmail-via-smtp(("rossini@oz.net") #<buffer " smtpmail temp">)
  smtpmail-send-it()
  gnus-agent-send-mail()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit(nil)
  call-interactively(message-send-and-exit)



and the *trace of SMTP....* buffer looks like:




220 smtp.washington.edu ESMTP Sendmail 8.12.10+UW03.09/8.12.10+UW03.09; Thu, 22 Jan 2004 23:25:07 -0800^M
EHLO stevedallas^M
250-smtp.washington.edu Hello sense-sea-MegaSub-2-231.oz.net [216.39.172.231], pleased to meet you^M
250-ENHANCEDSTATUSCODES^M
250-PIPELINING^M
250-EXPN^M
250-VERB^M
250-8BITMIME^M
250-SIZE 60000000^M
250-ETRN^M
250-AUTH GSSAPI^M
250-STARTTLS^M
250-DELIVERBY^M
250 HELP^M
STARTTLS^M
220 2.0.0 Ready to start TLS^M
*** Starting TLS handshake
- Certificate type: X.509
 - Got a certificate list of 1 certificates.

 - Certificate[0] info:
 # The hostname in the certificate matches 'smtp.washington.edu'.
 # valid since: Thu Jan 23 12:44:00 PST 2003
 # expires at: Sun Feb  8 15:10:00 PST 2004
 # serial number: 09 d4 97 
 # fingerprint: 63 12 43 fc 7d 49 f1 c0 5d 75 99 0a 7b de 4d 51 
 # version: #3
 # public key algorithm: RSA
 #   Modulus: 1024 bits
 # Subject's DN: C=US,ST=Washington,L=Seattle,O=University of
Washington,OU=NDC,CN=smtp.washington.edu
 # Issuer's DN: C=ZA,ST=Western Cape,L=Cape Town,O=Thawte Consulting
cc,OU=Certification Services Division,CN=Thawte Server
CA,EMAIL=server-certs@thawte.com


- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS 1.0
- Key Exchange: RSA
- Cipher: ARCFOUR 128
- MAC: SHA
- Compression: NULL
QUIT^M
221 2.0.0 smtp.washington.edu closing connection

Process SMTP killed



BTW, for completeness, I threw back in the ^M's that I'm getting in
the trace buffer, but got converted upon cut-and-paste.

best,
-tony


-- 
rossini@u.washington.edu            http://www.analytics.washington.edu/ 
Biomedical and Health Informatics   University of Washington
Biostatistics, SCHARP/HVTN          Fred Hutchinson Cancer Research Center
UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable
FHCRC  (M/W): 206-667-7025 FAX=206-667-4812 | use Email

CONFIDENTIALITY NOTICE: This e-mail message and any attachments may be
confidential and privileged. If you received this message in error,
please destroy it and notify the sender. Thank you.

Re: trying to deal with an smtp server that wants encryption

[Simon Josefsson]

> Signaling: (invalid-regexp "Invalid preceding regular expression")
>   re-search-forward("*** Handshake has failed" nil t)

I think I have fixed this in CVS now.

Re: trying to deal with an smtp server that wants encryption

[A.J. Rossini]

"Simon Josefsson" <jas@extundo.com> writes:

>> Signaling: (invalid-regexp "Invalid preceding regular expression")
>>   re-search-forward("*** Handshake has failed" nil t)
>
> I think I have fixed this in CVS now.

And it's fixed for me.  

Using the config template I posted earlier in this thread, with
Debian/Sid XEmacs 21.4.14, gnutls-bin 1.0.4, CVS version of No Gnus
updated after Simon's message above, and copying contrib/starttls.el
to be found first on the load-path, instead of lisp/starttls.el.

It doesn't work for me yet under Emacs 21.3 (Debian/Sid) but ought to
be able to without too much more work.  The two problems that I will
be starting to work on at this point are:

1. dealing with  updating smtpmail (masking the system version) 

2. playing with contrib/starttls.el (what's the right replacement for
   the XEmacs defvaralias  in Emacs?)

best,
-tony

-- 
rossini@u.washington.edu            http://www.analytics.washington.edu/ 
Biomedical and Health Informatics   University of Washington
Biostatistics, SCHARP/HVTN          Fred Hutchinson Cancer Research Center
UW (Tu/Th/F): 206-616-7630 FAX=206-543-3461 | Voicemail is unreliable
FHCRC  (M/W): 206-667-7025 FAX=206-667-4812 | use Email

CONFIDENTIALITY NOTICE: This e-mail message and any attachments may be
confidential and privileged. If you received this message in error,
please destroy it and notify the sender. Thank you.