A few basic nnimap problems

A few basic nnimap problems

[Bjørn Mork]

I am slow...

I have spent some time now trying to just get a basic nnimap connection
going on a test account, so that I am able to experiment a bit without
risking anything.

The problems are:

1. nnimap seems to default to "imaps", so you actually have to configure
 some none-default setting to get a connection on any sane server (using
 STARTTLS on port 143)

2. setting  (nnimap-stream 'starttls) doesn't help either, as this
 requires some odd binary I didn't have:
 "Unable to open server nnimap+example.net due to: Searching for program: no such file or directory, gnutls-cli"
 Note that I do have openssl, and have no problems using it to initiate
 a STARTTLS session:

 $ openssl s_client -starttls imap -connect example.com:143 -no_ssl2 -ign_eof
 CONNECTED(00000003)
 [..]
 SSL-Session:
     Protocol  : TLSv1
     Cipher    : AES256-SHA
 [..]
 ---
 . OK CAPABILITY completed


3. Even after doing "apt-get install gnutls-bin", starttls is still
 no-go:  "Unable to contact example.com:nil via (quote starttls)"


4. Giving up STARTTLS and using (nnimap-stream 'network) does not make
  much difference.  Similar error


5. Going for (nnimap-stream 'shell), which I *know* works since I use
  that for my regular mail, also gives a few problems.  The
  imap-shell-program variable does not seem to be respected, but instead
  gnus defaults to ssh'ing to the imap server.  


6. Even allowing the ssh is not enough.  Gnus complains about a missing
  ssh-askpass binary unless the host key was already cached.



OK, I finally got the imap connection up, but I believe there is far too
much hassle and it seems currently impossible to configure Gnus for a
"normal" IMAP connection.  I believe this:

  (setq gnus-select-method  '(nnimap "example.com"))

should result in a working connection to any IMAP server listening on
port 143/tcp, and automatically enable STARTTLS if supported by the
server.



Bjørn

Re: A few basic nnimap problems

[Lars Magne Ingebrigtsen]

Bjørn Mork <bjorn@mork.no> writes:

> 1. nnimap seems to default to "imaps", so you actually have to configure
>  some none-default setting to get a connection on any sane server (using
>  STARTTLS on port 143)

I think imaps is probably more popular these days than imap + starttls. 

> 2. setting  (nnimap-stream 'starttls) doesn't help either, as this
>  requires some odd binary I didn't have:
>  "Unable to open server nnimap+example.net due to: Searching for program: no such file or directory, gnutls-cli"
>  Note that I do have openssl, and have no problems using it to initiate
>  a STARTTLS session:

Yeah, I don't know what's going on in the starttls.el file.  There's a
gazillion variables to set there, but nothing to explain how to just use
openssl s_client.  Does anybody have a recipe? 

> 3. Even after doing "apt-get install gnutls-bin", starttls is still
>  no-go:  "Unable to contact example.com:nil via (quote starttls)"

That sounds odd.  You probably have something like:

  '(nnimap "..."
           (nnimap-stream 'starttls))

And the second quote shouldn't be there.           
           
> 4. Giving up STARTTLS and using (nnimap-stream 'network) does not make
>   much difference.  Similar error

Same problem.

> 5. Going for (nnimap-stream 'shell), which I *know* works since I use
>   that for my regular mail, also gives a few problems.  The
>   imap-shell-program variable does not seem to be respected, but instead
>   gnus defaults to ssh'ing to the imap server.  

See nnimap-shell-program.

>   (setq gnus-select-method  '(nnimap "example.com"))
>
> should result in a working connection to any IMAP server listening on
> port 143/tcp, and automatically enable STARTTLS if supported by the
> server.

It does work with imap.gmail.com, since it's an imaps server.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: A few basic nnimap problems

[Bjørn Mork]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Bjørn Mork <bjorn@mork.no> writes:
>
>> 1. nnimap seems to default to "imaps", so you actually have to configure
>>  some none-default setting to get a connection on any sane server (using
>>  STARTTLS on port 143)
>
> I think imaps is probably more popular these days than imap + starttls. 

really?  Anyway, I still expect an imap client to connect to the imap
port by default.  Just as I expect my browser to connect to port 80 by
default, although 443 might become more popular over time.


>> 2. setting  (nnimap-stream 'starttls) doesn't help either, as this
>>  requires some odd binary I didn't have:
>>  "Unable to open server nnimap+example.net due to: Searching for program: no such file or directory, gnutls-cli"
>>  Note that I do have openssl, and have no problems using it to initiate
>>  a STARTTLS session:
>
> Yeah, I don't know what's going on in the starttls.el file.  There's a
> gazillion variables to set there, but nothing to explain how to just use
> openssl s_client.  Does anybody have a recipe? 
>
>> 3. Even after doing "apt-get install gnutls-bin", starttls is still
>>  no-go:  "Unable to contact example.com:nil via (quote starttls)"
>
> That sounds odd.  You probably have something like:
>
>   '(nnimap "..."
>            (nnimap-stream 'starttls))
>
> And the second quote shouldn't be there.           

You mean like
   '(nnimap "..."
            (nnimap-stream starttls))
?

I wondered about that, so I looked at the documentation which had an
example with the quote. Anyway, with only this in ~/.gnus:

(setq gnus-select-method '(nntp "news.gmane.org"))
(setq gnus-secondary-select-methods '(
      (nnimap "mail.mork.no"
        (nnimap-stream starttls)))
)


I get:

Warning: Opening nnimap server on mail.mork.no...failed: ; Denied server nnimap+mail.mork.no; Opening nnimap server on mail.mork.no...failed: ; Unable to open server nnimap+mail.mork.no due to: Buffer *nnimap mail.mork.no nil  *nntpd** has no process


>> 4. Giving up STARTTLS and using (nnimap-stream 'network) does not make
>>   much difference.  Similar error
>
> Same problem.

Yup, removing the quote does fix this.

>
>> 5. Going for (nnimap-stream 'shell), which I *know* works since I use
>>   that for my regular mail, also gives a few problems.  The
>>   imap-shell-program variable does not seem to be respected, but instead
>>   gnus defaults to ssh'ing to the imap server.  
>
> See nnimap-shell-program.

Aw, right.  I should have guessed.

>>   (setq gnus-select-method  '(nnimap "example.com"))
>>
>> should result in a working connection to any IMAP server listening on
>> port 143/tcp, and automatically enable STARTTLS if supported by the
>> server.
>
> It does work with imap.gmail.com, since it's an imaps server.  :-)

So I need to listen to yet another port just to do ssl...  The world
isn't going forward, is it?


Bjørn

Re: A few basic nnimap problems

[Lars Magne Ingebrigtsen]

Bjørn Mork <bjorn@mork.no> writes:

>> I think imaps is probably more popular these days than imap + starttls. 
>
> really?  Anyway, I still expect an imap client to connect to the imap
> port by default.  Just as I expect my browser to connect to port 80 by
> default, although 443 might become more popular over time.

I expect my mail client to default to using whatever is the most likely
default, and not worry my little head about port numbers and other rilly
technical stuff.

> I wondered about that, so I looked at the documentation which had an
> example with the quote.

So it did.  Fixed now.

> Anyway, with only this in ~/.gnus:
>
> (setq gnus-select-method '(nntp "news.gmane.org"))
> (setq gnus-secondary-select-methods '(
>       (nnimap "mail.mork.no"
>         (nnimap-stream starttls)))
> )
>
> I get:
>
> Warning: Opening nnimap server on mail.mork.no...failed: ; Denied server nnimap+mail.mork.no; Opening nnimap server on mail.mork.no...failed: ; Unable to open server nnimap+mail.mork.no due to: Buffer *nnimap mail.mork.no nil  *nntpd** has no process

This means that your starttls stuff doesn't work at all, which isn't
surprising.  I don't think the default starttls thing will work by
default for many people.

We have to clean up the tls/starttls stuff.  I think tls.el seems like a
more modern approach.  Could we adjust it to also allow starttls in
addition to tls, and then just get rid of starttls.el?  Anybody?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: A few basic nnimap problems

[Steinar Bang]

>>>>> Lars Magne Ingebrigtsen <larsi@gnus.org>:

> Bjørn Mork <bjorn@mork.no> writes:
>>> I think imaps is probably more popular these days than imap + starttls. 
>> 
>> really?  Anyway, I still expect an imap client to connect to the imap
>> port by default.  Just as I expect my browser to connect to port 80 by
>> default, although 443 might become more popular over time.

> I expect my mail client to default to using whatever is the most likely
> default, and not worry my little head about port numbers and other rilly
> technical stuff.

I expect imap to mean imap and imaps to mean imaps, the way
/etc/services meant them to do.

All other clients I have tried configuring have had imap config starting
out with port 143 as the default, and switching over to 993 if the
checkbox for use secure connection is ticked.

Re: A few basic nnimap problems

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> We have to clean up the tls/starttls stuff.  I think tls.el seems like a
> more modern approach.  Could we adjust it to also allow starttls in
> addition to tls, and then just get rid of starttls.el?  Anybody?

I just made an attempt, and I think I got it working.

I'll just be so happy when all those moronic external programs goes away
here.  It's mindboggling that it's taken until 2010 for Emacs to get
built-in tls support, and we still have to deal with the quirks of these
...  gnutls-cli takes a -s to give you starttls, which is sensible.
openssl s_client is "-starttls imap" (or whatever other protocol you're
using) -- and THEN IT EATS THE GREETING.

Gah!

So frustrating.  Bah-nanas.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: A few basic nnimap problems

[Bjørn Mork]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> openssl s_client is "-starttls imap" (or whatever other protocol you're
> using) -- and THEN IT EATS THE GREETING.

You mean the problem that you end up with just the final

 ". OK CAPABILITY completed"

and not the actual capability list?

Doesn't matter much, does it? Send a cabability command if you cannot
parse IMAP4rev1 out of the OK CAPABILITY.  In fact reading the RFC
3501, sending the CAPABILITY as part of the preauth OK is optional:

<quote rfc3501  7.2.1.  CAPABILITY Response>
      A server MAY send capabilities automatically, by using the
      CAPABILITY response code in the initial PREAUTH or OK responses,
      and by sending an updated CAPABILITY response code in the tagged
      OK response as part of a successful authentication.  It is
      unnecessary for a client to send a separate CAPABILITY command if
      it recognizes these automatic capabilities.
</quote>

although I haven't yet seen an IMAP4rev1 server not sending it
automatically...  But you should handle such servers, and that will also
handle the case where openssl ate your automatic capability.

One question I can't find the RFC answer to right now: Is it possible
for the capabilities to change depending on the logged in user?  Maybe
some users are allowed to use server resource intensive commands like
sorting and threading while others are not?  The quoted text above
certainly indicates that this may happen. If this is possible, then
you should probable issue another capability command right after
authenticating as well.


Bjørn

Re: A few basic nnimap problems

[Daniel Pittman]

Bjørn Mork <bjorn@mork.no> writes:
> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

[...]

> One question I can't find the RFC answer to right now: Is it possible for
> the capabilities to change depending on the logged in user?

Yes, absolutely.  This is routine: many servers only offer LOGIN and PLAIN to
users who have an SSL or TLS secured connection.  Since STARTTLS is a state
change in a running connection this is already legal.

> Maybe some users are allowed to use server resource intensive commands like
> sorting and threading while others are not?

This would also be ... not against the rules, given that other protocol state
changes can trigger the same thing.

        Daniel
-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

Re: A few basic nnimap problems

[Lars Magne Ingebrigtsen]

Bjørn Mork <bjorn@mork.no> writes:

> You mean the problem that you end up with just the final
>
>  ". OK CAPABILITY completed"
>
> and not the actual capability list?

No, the "* OK Dovecot bla bla" message.

It was too late in the evening and I was just confused.  Of course the
starttls thing would eat the greeting -- it has to, since it's sending
"FOO STARTTLS" to the server, and then getting stuff back again.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen