Trouble connecting with IMAP (Windows Emacs with Cygwin SSL/TLS tools)

Trouble connecting with IMAP (Windows Emacs with Cygwin SSL/TLS tools)

[Steven E. Harris]

I have a recent CVS snapshot of Gnus setup with the Cygwin XEmacs build
and the rest of the Cygwin tools (openssl, starttls) working just
fine. I can connect to various IMAP servers with encrypted connections.

Recently I set up a Windows build of Emacs (GNU Emacs 22.2.1) and I'm
trying to get Gnus to work with it similarly. Gnus is able to connect
(unencrypted) to NNTP servers, but all of my attempts to connect to IMAP
servers fail. I suspect it's a coding system problem with using Cygwin
programs to handle the encrypted connections.

Here's the *imap-log* buffer resulting from an attempted connection to
my IMAP server:

,----[ *imap-log* ]
| depth=0 /O=mail.panix.com/OU=Go to https://www.thawte.com/repository/index.html/OU=Thawte SSL123 certificate/OU=Domain Validated/CN=mail.panix.com
| verify error:num=20:unable to get local issuer certificate
| verify return:1
| depth=0 /O=mail.panix.com/OU=Go to https://www.thawte.com/repository/index.html/OU=Thawte SSL123 certificate/OU=Domain Validated/CN=mail.panix.com
| verify error:num=27:certificate not trusted
| verify return:1
| depth=0 /O=mail.panix.com/OU=Go to https://www.thawte.com/repository/index.html/OU=Thawte SSL123 certificate/OU=Domain Validated/CN=mail.panix.com
| verify error:num=21:unable to verify the first certificate
| verify return:1
| * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS QUOTA AUTH=PLAIN AUTH=LOGIN] Dovecot ready.
| 55 LOGOUT
| * BYE Logging out
| 55 OK Logout completed.
| read:errno=0
| 56 LOGIN "seh" "mypass"
`----

Some things look strange here. Not the "^M" terminators on the lines
sent and received from the server. Also note how one side logs out
/before/ it logs in.

Here's part of the *Messages* buffer during the login attempt:

,----[ *Messages* ]
| Checking new news...
| Opening nnimap server on panix...
| imap: Connecting to mail.panix.com...
| imap: Opening SSL connection with `openssl s_client -CApath /usr/ssl/certs -quiet -ssl3 -connect %s:%p'...done
| Waiting for response from mail.panix.com...done
| Parsing authinfo file `~/.authinfo'.
| imap: Plaintext authentication...
| Unable to open server due to: Buffer  *nnimap* panix has no process
| Unable to open nnimap:panix, go offline? (y or n) 
| Opening nnimap server on panix...failed
| Checking new news...done
`----

I see that there are the two variables `imap-coding-system-for-write'
and `imap-coding-system-for-read', which are set to `binary' by
default. Might resetting those be related to solving this problem?

Any advice would be appreciated.

-- 
Steven E. Harris

Re: Trouble connecting with IMAP (Windows Emacs with Cygwin SSL/TLS tools)

[Steven E. Harris]

"Steven E. Harris" <seh@panix.com> writes:

> Not[e] the "^M" terminators on the lines sent and received from the
> server.

Of course, those got stripped. In the *imap-log* buffer as displayed, it
/looks/ like this:

,----[ *imap-log* with faked control characters]
| * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS QUOTA AUTH=PLAIN AUTH=LOGIN] Dovecot ready.^M
| 55 LOGOUT^M
| * BYE Logging out^M
| 55 OK Logout completed.^M
| read:errno=0
| 56 LOGIN "seh" "mypass"^M
`----

Here I've replaced the extra CR at the end of each line with "^M".

When I look at the *imap-log* buffer in hexl-mode, I can see that each
of the offending lines ends with CR CR LF. It's hard to tell what's
really going back and forth across the network.

-- 
Steven E. Harris

Re: Trouble connecting with IMAP (Windows Emacs with Cygwin SSL/TLS tools)

[Ted Zlatanov]

On Sat, 16 Aug 2008 10:22:30 -0400 "Steven E. Harris" <seh@panix.com> wrote: 

SEH> I have a recent CVS snapshot of Gnus setup with the Cygwin XEmacs build
SEH> and the rest of the Cygwin tools (openssl, starttls) working just
SEH> fine. I can connect to various IMAP servers with encrypted connections.

SEH> Recently I set up a Windows build of Emacs (GNU Emacs 22.2.1) and I'm
SEH> trying to get Gnus to work with it similarly. Gnus is able to connect
SEH> (unencrypted) to NNTP servers, but all of my attempts to connect to IMAP
SEH> servers fail. I suspect it's a coding system problem with using Cygwin
SEH> programs to handle the encrypted connections.
...
SEH> Some things look strange here. Not the "^M" terminators on the lines
SEH> sent and received from the server. Also note how one side logs out
SEH> /before/ it logs in.
...
SEH> I see that there are the two variables `imap-coding-system-for-write'
SEH> and `imap-coding-system-for-read', which are set to `binary' by
SEH> default. Might resetting those be related to solving this problem?
...
SEH> When I look at the *imap-log* buffer in hexl-mode, I can see that each
SEH> of the offending lines ends with CR CR LF. It's hard to tell what's
SEH> really going back and forth across the network.

Hi,

I haven't replied because I don't know much about Emacs coding systems,
but it's been a while since you made this report.  Have you found a
workaround, have things started working with a newer Emacs CVS, or are
you still stuck?  I don't have a Cygwin setup here but I can make one if
no one else can help debug this.

Thanks
Ted

Re: Trouble connecting with IMAP (Windows Emacs with Cygwin SSL/TLS tools)

[Steven E. Harris]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I haven't replied because I don't know much about Emacs coding
> systems, but it's been a while since you made this report.

Thanks for reviving the topic. Steinar Bang recently took up a similar
discussion on the ding list.¹

> Have you found a workaround, have things started working with a newer
> Emacs CVS, or are you still stuck?

Unfortunately, I'm still stuck. I haven't installed a newer Emacs; the
one I have installed is GNU Emacs 22.3.1, built on 6 September 2008. I
have updated my Gnus installation to track the CVS repository.

As I mentioned earlier, I had been building up my Emacs installation to
be parallel with the XEmacs setup I've built over the last ten years or
so, but I'm never going to make the switch until I get Gnus working
properly.

> I don't have a Cygwin setup here but I can make one if no one else can
> help debug this.

What would happen if someone installed Emacs on Windows without Cygwin?
Could Gnus running in Emacs still make secure connections to an IMAP
server? I'm loath to install another OpenSSL package, given that the
Cygwin one is already there on my computer, but perhaps that's what it's
going to take.

What's odd here is that a non-Cygwin Emacs is able to use various Cygwin
programs without trouble. What is special about this case, where
apparently openssl as invoked by Emacs is able to negotiate a secure
connection properly, but not get the actual payload content through as
expected?

Dangerous as it may be, how does one convince Gnus /not/ to use a secure
connection to an IMAP server? I'd like to try that temporarily to see if
it works when we take openssl out of the picture.


Footnotes: 
¹ http://thread.gmane.org/gmane.emacs.gnus.general/67801/focus=67809

-- 
Steven E. Harris

Re: Trouble connecting with IMAP (Windows Emacs with Cygwin SSL/TLS tools)

[Steven E. Harris]

"Steven E. Harris" <seh@panix.com> writes:

> Unfortunately, I'm still stuck.

I made some progress this afternoon, as reported here:

  Re: nnimap with openssl stopping up after connecting in Windows
  http://thread.gmane.org/gmane.emacs.gnus.general/67801/focus=68024

It's still not a wholly understood or solved problem, but there's more
there to consider.

-- 
Steven E. Harris