Some issues with new nnimap

Some issues with new nnimap

[Drew Hess]

Hi Lars,

I'm really happy to see you working on Gnus again. Thanks for a great
mail reader!

I'm having a few issues with the new nnimap backend. Here's my server
definition:

(setq gnus-select-method '(nnnil ""))
(setq gnus-secondary-select-methods
      '(
        (nnimap "imap.bothan.net"
         (nnir-search-engine imap))))

Without a .authinfo file, I'm unable to login. The login process
appears to hang after the IMAP server sends its greeting. Here are the
contents of my *nnimap.. buffer:

---
Resolving 'imap.bothan.net'...
Connecting to '69.36.241.48:993'...
- Ephemeral Diffie-Hellman parameters
 - Using prime: 1024 bits
 - Secret key: 1023 bits
 - Peer's public key: 1021 bits
- Certificate type: X.509
 - Got a certificate list of 1 certificates.
 - Certificate[0] info:
  - subject `C=US,ST=California,L=San Francisco,O=Bothan Spy
Network,OU=Server,CN=imap.bothan.net', issuer
`C=US,ST=California,L=San Francisco,O=Bothan Spy
Network,OU=Certificate Authority,CN=Bothan Spy Network Certificate
Authority', RSA key 2048 bits, signed using RSA-SHA1, activated
`2010-06-25 00:29:42 UTC', expires `2011-06-25 00:29:42 UTC', SHA-1
fingerprint `5cb747fecd4e1a74928bd40deb307b7e92669c58'
- The hostname in the certificate matches 'imap.bothan.net'.
- Peer's certificate issuer is unknown
- Peer's certificate is NOT trusted
- Version: TLS1.0
- Key Exchange: DHE-RSA
- Cipher: AES-128-CBC
- MAC: SHA1
- Compression: NULL
- Handshake was completed

- Simple Client Mode:

* OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE
AUTH=PLAIN] Dovecot ready.
---

Here are the contents of the *Messages* buffer:

---
Loading /Users/dhess/elisp/gnus/lisp/gnus-load.el (source)...done
Reading /Users/dhess/gnus/.newsrc.eld...
Reading active file via nnnil...done
Reading active file from imap.bothan.net via nnimap...
Opening nnimap server on imap.bothan.net...
Opening TLS connection to `imap.bothan.net'...
Opening TLS connection with `gnutls-cli -p imaps imap.bothan.net'...done
Opening TLS connection to `imap.bothan.net'...done
Opening nnimap server on imap.bothan.net...failed:
Reading active file from archive via nnfolder...
Opening nnfolder server on archive...done
Reading active file from archive via nnfolder...done
Opening nnimap server on imap.bothan.net...
Denied server nnimap+imap.bothan.net
Opening nnimap server on imap.bothan.net...failed:
No new newsgroups
Checking new news...
Reading active file from gnus-help via nndoc...
Cannot read active file from nndoc server
Reading active file via nndraft...done
Checking new news...done
No Gnus is good news
No more unread newsgroups
Mark set
---

I looked through the mailing list and this appears to be similar to a
problem someone else was having, which you chalked up to a race
condition and fixed with commit
3a31df78354f7ddba9a08aed8f1a5e180f1ab032, I believe.

So then I created a .authinfo.gpg file. I'm using EasyPG with
(epa-file-enable) in my .emacs, and I'm able to open this file in a
new buffer seamlessly, but when I try to connect to my IMAP server,
I'm just repeatedly asked for my GPG passphrase. I'm certain I'm
typing it correctly. Here's what my *Messages* buffer looks like when
this happens:

---
Loading /Users/dhess/elisp/gnus/lisp/gnus-load.el (source)...done
Reading /Users/dhess/gnus/.newsrc.eld...
Reading active file via nnnil...done
Reading active file from imap.bothan.net via nnimap...
Opening nnimap server on imap.bothan.net...
Opening TLS connection to `imap.bothan.net'...
Opening TLS connection with `gnutls-cli -p imaps imap.bothan.net'...done
Opening TLS connection to `imap.bothan.net'...done
/Users/dhess/.authin: 0% (0/655)
/Users/dhess/.authin: 100% (655/655)
/Users/dhess/.authin: 0% (0/655)
/Users/dhess/.authin: 100% (655/655)
/Users/dhess/.authin: 0% (0/655)
---

This continues until I hit C-g to abort, at which time Gnus gives up
trying to connect.

Finally, if I just create an unencrypted .authinfo file, everything
works fine:

---
Loading /Users/dhess/elisp/gnus/lisp/gnus-load.el (source)...done
Reading /Users/dhess/gnus/.newsrc.eld...
Reading active file via nnnil...done
Reading active file from imap.bothan.net via nnimap...
Opening nnimap server on imap.bothan.net...
Opening TLS connection to `imap.bothan.net'...
Opening TLS connection with `gnutls-cli -p imaps imap.bothan.net'...done
Opening TLS connection to `imap.bothan.net'...done
Opening nnimap server on imap.bothan.net...done
Reading active file from imap.bothan.net via nnimap...done
Reading active file from archive via nnfolder...
Opening nnfolder server on archive...done
Reading active file from archive via nnfolder...done
No new newsgroups
Checking new news...
Reading active file from gnus-help via nndoc...
Cannot read active file from nndoc server
Reading active file via nndraft...done
Checking new news...done
---

There's only one catch: every time I hit 'g', Emacs beeps at me. All I
get in the *Messages* buffer is this:

---
Checking new news...
Reading active file from gnus-help via nndoc...
Cannot read active file from nndoc server
Reading active file via nndraft...done
Checking new news...done
---

The groups are refreshing properly, etc., so it's not strictly a
problem, it's just annoying. I didn't have this issue with bundled
Gnus.

FYI, I'm running Emacs 23.2 with YAMAMOTO Mitsuharu's Mac patches. I'm
willing to try this with stock Emacs 23.2 if you think it might be
related.

thanks again!
d

Re: Some issues with new nnimap

[Lars Magne Ingebrigtsen]

Drew Hess <drew.hess@gmail.com> writes:

> Without a .authinfo file, I'm unable to login. The login process
> appears to hang after the IMAP server sends its greeting. Here are the
> contents of my *nnimap.. buffer:

Ok, I see the problem.  If the auth sources don't exist (i.e., no
~/.authinfo), auth-sources.el won't prompt for user name/password at
all.  I'll fix that.

> So then I created a .authinfo.gpg file. I'm using EasyPG with
> (epa-file-enable) in my .emacs, and I'm able to open this file in a
> new buffer seamlessly, but when I try to connect to my IMAP server,
> I'm just repeatedly asked for my GPG passphrase. I'm certain I'm
> typing it correctly. Here's what my *Messages* buffer looks like when
> this happens:
>
> ---
> Loading /Users/dhess/elisp/gnus/lisp/gnus-load.el (source)...done
> Reading /Users/dhess/gnus/.newsrc.eld...
> Reading active file via nnnil...done
> Reading active file from imap.bothan.net via nnimap...
> Opening nnimap server on imap.bothan.net...
> Opening TLS connection to `imap.bothan.net'...
> Opening TLS connection with `gnutls-cli -p imaps imap.bothan.net'...done
> Opening TLS connection to `imap.bothan.net'...done
> /Users/dhess/.authin: 0% (0/655)
> /Users/dhess/.authin: 100% (655/655)
> /Users/dhess/.authin: 0% (0/655)
> /Users/dhess/.authin: 100% (655/655)
> /Users/dhess/.authin: 0% (0/655)

I think this sounds like a bug in the epa code, possibly?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Some issues with new nnimap

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Ok, I see the problem.  If the auth sources don't exist (i.e., no
> ~/.authinfo), auth-sources.el won't prompt for user name/password at
> all.  I'll fix that.

I've now done this for the ~/.authinfo and ~/.authinfo.gpg cases, but
I'm not sure how the Secret Service API is supposed to work, so I'll
leave that bit to Ted.  :-)

Looking at the code, it only seems to store the (queried-for) password,
and not the user name?  That can't be right...

So now there's two saving things in auth-source-create -- one in the
loop in the passwd branch, and one at the end after both the user name
and passwords have been collected.  That should be fixed.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

nndoc groups beeping (was: Some issues with new nnimap)

[Andreas Seltenreich]

Drew Hess writes:

> There's only one catch: every time I hit 'g', Emacs beeps at me. All I
> get in the *Messages* buffer is this:
>
> ---
> Checking new news...
> Reading active file from gnus-help via nndoc...
> Cannot read active file from nndoc server
> Reading active file via nndraft...done
> Checking new news...done
> ---

I see this too with nndoc groups since pulling last month's changes.
`g' doesn't refresh them any longer — I need to `M-g' them now.

regards,
andreas

Re: nndoc groups beeping (was: Some issues with new nnimap)

[Drew Hess]

On Sun, Sep 26, 2010 at 12:57 PM, Andreas Seltenreich
<andreas+ding@gate450.dyndns.org> wrote:
> Drew Hess writes:
>
>> There's only one catch: every time I hit 'g', Emacs beeps at me. All I
>> get in the *Messages* buffer is this:
>>
>> ---
>> Checking new news...
>> Reading active file from gnus-help via nndoc...
>> Cannot read active file from nndoc server
>> Reading active file via nndraft...done
>> Checking new news...done
>> ---
>
> I see this too with nndoc groups since pulling last month's changes.
> `g' doesn't refresh them any longer — I need to `M-g' them now.

I thought it might be that. Sorry for the naive question, but
how do I remove the nndoc server? I tried the usual suspects
from the server buffer ('k' etc.), but Gnus responds that it's
a read-only server.

d

Re: nndoc groups beeping

[Andreas Seltenreich]

Drew Hess writes:

> I thought it might be that. Sorry for the naive question, but
> how do I remove the nndoc server? I tried the usual suspects
> from the server buffer ('k' etc.), but Gnus responds that it's
> a read-only server.

The server is created implicitly by some group.  You should be able find
it using `L' in the Group buffer and the server should disappear after
killing the group.

regards,
andreas

Re: nndoc groups beeping

[Drew Hess]

On Sun, Sep 26, 2010 at 1:11 PM, Andreas Seltenreich
<andreas+ding@gate450.dyndns.org> wrote:
> Drew Hess writes:
>
>> I thought it might be that. Sorry for the naive question, but
>> how do I remove the nndoc server? I tried the usual suspects
>> from the server buffer ('k' etc.), but Gnus responds that it's
>> a read-only server.
>
> The server is created implicitly by some group.  You should be able find
> it using `L' in the Group buffer and the server should disappear after
> killing the group.

Thank you! That did the trick: no more beeping.

d

Re: Some issues with new nnimap

[Drew Hess]

On Sun, Sep 26, 2010 at 5:50 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> Ok, I see the problem.  If the auth sources don't exist (i.e., no
>> ~/.authinfo), auth-sources.el won't prompt for user name/password at
>> all.  I'll fix that.
>
> I've now done this for the ~/.authinfo and ~/.authinfo.gpg cases, but
> I'm not sure how the Secret Service API is supposed to work, so I'll
> leave that bit to Ted.  :-)
>
> Looking at the code, it only seems to store the (queried-for) password,
> and not the user name?  That can't be right...
>
> So now there's two saving things in auth-source-create -- one in the
> loop in the passwd branch, and one at the end after both the user name
> and passwords have been collected.  That should be fixed.

Thanks! I can now login without a .authinfo file.

The problem I was experiencing with .authinfo.gpg persists. I'm pretty
certain that the Mac patches to Emacs 23.2 don't touch the epa code.
Are you using epa as bundled with Emacs 23.2, or something different?
How are you enabling epa in your .emacs?

d

Re: Some issues with new nnimap

[Lars Magne Ingebrigtsen]

Drew Hess <drew.hess@gmail.com> writes:

> The problem I was experiencing with .authinfo.gpg persists. I'm pretty
> certain that the Mac patches to Emacs 23.2 don't touch the epa code.
> Are you using epa as bundled with Emacs 23.2, or something different?
> How are you enabling epa in your .emacs?

I'm not doing anything to epa in my .emacs at all, and I'm using bzr
Emacs (aka Emacs 24).

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Some issues with new nnimap

[Lars Magne Ingebrigtsen]

Drew Hess <drew.hess@gmail.com> writes:

> There's only one catch: every time I hit 'g', Emacs beeps at me. All I
> get in the *Messages* buffer is this:
>
> ---
> Checking new news...
> Reading active file from gnus-help via nndoc...
> Cannot read active file from nndoc server

I think I've fixed this now...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nndoc groups beeping

[Lars Magne Ingebrigtsen]

Andreas Seltenreich <andreas+ding@gate450.dyndns.org> writes:

> I see this too with nndoc groups since pulling last month's changes.
> `g' doesn't refresh them any longer — I need to `M-g' them now.

Hm, yes.  nndoc needs a -retrieve-groups function.  I'll add one, but
not tonight.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Some issues with new nnimap

[Drew Hess]

On Sun, Sep 26, 2010 at 2:00 PM, Drew Hess <drew.hess@gmail.com> wrote:
> On Sun, Sep 26, 2010 at 5:50 AM, Lars Magne Ingebrigtsen <larsi@gnus.org> wrote:
>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>
>>> Ok, I see the problem.  If the auth sources don't exist (i.e., no
>>> ~/.authinfo), auth-sources.el won't prompt for user name/password at
>>> all.  I'll fix that.
>>
>> I've now done this for the ~/.authinfo and ~/.authinfo.gpg cases, but
>> I'm not sure how the Secret Service API is supposed to work, so I'll
>> leave that bit to Ted.  :-)
>>
>> Looking at the code, it only seems to store the (queried-for) password,
>> and not the user name?  That can't be right...
>>
>> So now there's two saving things in auth-source-create -- one in the
>> loop in the passwd branch, and one at the end after both the user name
>> and passwords have been collected.  That should be fixed.
>
> Thanks! I can now login without a .authinfo file.

There is, however, still one problem. In older versions of Gnus, if auth
failed (e.g., bad password), Gnus would prompt for credentials again, but
now it doesn't, so if you mistype your password, you're simply denied
access to the IMAP server without another chance to authenticate.

d

Re: Some issues with new nnimap

[Lars Magne Ingebrigtsen]

Drew Hess <drew.hess@gmail.com> writes:

> There is, however, still one problem. In older versions of Gnus, if auth
> failed (e.g., bad password), Gnus would prompt for credentials again, but
> now it doesn't, so if you mistype your password, you're simply denied
> access to the IMAP server without another chance to authenticate.

Ah, yes.  Hm.  If the response to LOGIN is NO, then nnimap should flush
the credentials.  Let's see.  Yes, calling
`auth-source-forget-user-or-password' should be sufficient.  I'll add
that. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nndoc groups beeping

[Lars Magne Ingebrigtsen]

Andreas Seltenreich <andreas+ding@gate450.dyndns.org> writes:

> I see this too with nndoc groups since pulling last month's changes.
> `g' doesn't refresh them any longer — I need to `M-g' them now.

This has now been fixed.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen