nnimap-authinfo-file question

nnimap-authinfo-file question

[James Felix Black]

Hello,

Just today I've decided to try IMAP, and things appear to be going
swimmingly.  However, I can't seem to get gnus (oGnus, cvs updated
this morning) to read my authinfo.

My situation is a bit strange: I tunnel IMAP (and SMTP) over ssh to my
ISP.  I do this not out of sheer pig-headedness, or a desire to be
difficult, but rather because I buy my network connectivity from a
different provider than my mail service.  

Anyway.  No matter what machine name I list in the
nnimap-authinfo-file file, gnus asks me for a username and password.

Ideas?

TIA,
(jfb)

-- 
Perfection in design is achieved not when there is nothing more to
add, but rather when there is nothing more to take away.

Re: nnimap-authinfo-file question

[Kai Großjohann]

On 29 May 2001, James Felix Black wrote:

> Anyway.  No matter what machine name I list in the
> nnimap-authinfo-file file, gnus asks me for a username and password.

I have been successful with (add-to-list
'gnus-secondary-select-methods '(nnimap "foo"...)) and using `machine
foo' in ~/.authinfo.

I think.

kai
-- 
~/.signature: No such file or directory

Re: nnimap-authinfo-file question

[Kai Großjohann]

On 30 May 2001, Kai Großjohann wrote:

> I have been successful with (add-to-list
> 'gnus-secondary-select-methods '(nnimap "foo"...)) and using
> `machine foo' in ~/.authinfo.
> 
> I think.

Because of the `I think' part I'd like to make sure whether you tried
that and what happened when you did.

kai
-- 
~/.signature: No such file or directory

Re: nnimap-authinfo-file question

[James Felix Black]

> Because of the `I think' part I'd like to make sure whether you
> tried that and what happened when you did.

Didn't work.  No combination of "machine <name>" and "nnimap-address
<name>" or "(nnimap <name> ..." seems to work.  A thought strikes: as
I use Gnus exclusively for mail, my gnus-select-method is a '(nnimap
form.  Could this be a problem?

Hmmm.  No.  Tried it at work.

Time to go to the source, I guess.

(jfb)

-- 
Perfection in design is achieved not when there is nothing more to
add, but rather when there is nothing more to take away.

Re: nnimap-authinfo-file question

[Chris Brierley]

On 2001-05-29, James Felix Black <jfb@visi.com> wrote:

> My situation is a bit strange: I tunnel IMAP (and SMTP) over ssh to my
> ISP.
[...]
> Anyway.  No matter what machine name I list in the
> nnimap-authinfo-file file, gnus asks me for a username and
> password.

Did you try localhost?  If you're tunneling those
connections over SSH, then you are actually connecting to a
port on localhost.

I do exactly the same thing (with three different IMAP
servers), here's my .authinfo, note the machine's are all
localhost; they are made discrete my their port numbers
(which are the ports I'm tunneling):

,----[ .authinfo ]
| # Personal IMAP
| machine localhost login USERNAME port 10143
| # Work IMAP
| machine localhost login USERNAME port 8143
| # Client IMAP
| machine localhost login USERNAME port 9143
`----

Re: nnimap-authinfo-file question

[James Felix Black]

> Did you try localhost?  

Yes, but I hadn't tried specifing the port numbers.  Which I did, and
now it works.

Thanks,
(jfb)
-- 
Perfection in design is achieved not when there is nothing more to
add, but rather when there is nothing more to take away.

Re: nnimap-authinfo-file question

[Andi Hechtbauer]

Chris Brierley <brier@pobox.com> writes:

> On 2001-05-29, James Felix Black <jfb@visi.com> wrote:
> 
> > My situation is a bit strange: I tunnel IMAP (and SMTP) over ssh to my
> > ISP.

Nothing strange about that IMHO.

> > Anyway.  No matter what machine name I list in the
> > nnimap-authinfo-file file, gnus asks me for a username and
> > password.
> 
> Did you try localhost?  If you're tunneling those
> connections over SSH, then you are actually connecting to a
> port on localhost.
> 
> I do exactly the same thing (with three different IMAP
> servers), here's my .authinfo, note the machine's are all
> localhost; they are made discrete my their port numbers
> (which are the ports I'm tunneling):
> 
> ,----[ .authinfo ]
> | # Personal IMAP
> | machine localhost login USERNAME port 10143
> | # Work IMAP
> | machine localhost login USERNAME port 8143
> | # Client IMAP
> | machine localhost login USERNAME port 9143
> `----

It's easier and more convenient of course to have your ssh keys and
ssh-agent set up right (cf. ssh(1) "public-key cryptography", 
ssh-agent(1), and ssh-add(1)) and just 

(setq imap-shell-program '( "ssh -x -C remote.imap.host /usr/sbin/imapd" ))

(setq gnus-secondary-select-methods
      '(
        (nnimap "host"
                (nnimap-address "remote.imap.host")
                (nnimap-stream shell)
                (nnimap-list-pattern ("INBOX" "mail/*"))
        )
))

in your .gnus.el; So you M-x gnus and it just opens a ssh
connection and starts your imapd preauthenticated. No need for
username and password at all.

I don't know how the setup would be for more than one imap
server...

Someone mentioned this on this list some time ago - sorry for
this inaccurate crediting :-)

And I'm using this with great joy :-) Didn't even know about
starting imapd preauthenticated from the shell, before. And of 
course you can't be sure that every imapd knows about preauth.

Mine does though.

Gnus-Info should get a blink tag to highlight this feature. Well,
maybe not.

regards,
-- 
Andi "<blink>:-b</blink>" Hechtbauer                      anti@spin.de 
root                                          voice: +49 941 942 77 23
SPiN AG                 http://www.spin.de/     fax: +49 941 942 77 22  
------ PGPf16: B4 FC 30 E0 31 2F BC F5  38 D0 A2 FC EA A9 82 5D ------

Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Doug Alcorn]

Andi Hechtbauer <anti@spin.de> writes:

> It's easier and more convenient of course to have your ssh keys and
> ssh-agent set up right (cf. ssh(1) "public-key cryptography", 
> ssh-agent(1), and ssh-add(1)) and just 
> 
> (setq imap-shell-program '( "ssh -x -C remote.imap.host /usr/sbin/imapd" ))
> 
> (setq gnus-secondary-select-methods
>       '(
>         (nnimap "host"
>                 (nnimap-address "remote.imap.host")
>                 (nnimap-stream shell)
>                 (nnimap-list-pattern ("INBOX" "mail/*"))
>         )
> ))
> 
> in your .gnus.el; So you M-x gnus and it just opens a ssh
> connection and starts your imapd preauthenticated. No need for
> username and password at all.

Thanks for reposting this.  I was happy to read it and happier to see
it working for me.

The only problem I ran into is that my ssh shells timeout with no
activity.  The ends up with messages about the stream being currupt
and then showing the ssh connection reset message (sorry I don't have
a copy of the exact message now).

Noticing another thread about problems with nnimap connections timeing
out, would it be possible to have nnimap send a keepalive message to
the server?
-- 
 (__) Doug Alcorn (mailto:doug@lathi.net http://www.lathi.net)
 oo / PGP 02B3 1E26 BCF2 9AAF 93F1  61D7 450C B264 3E63 D543
 |_/  If you're a capitalist and you have the best goods and they're
      free, you don't have to proselytize, you just have to wait. 

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Simon Josefsson]

Doug Alcorn <doug@lathi.net> writes:

> Noticing another thread about problems with nnimap connections timeing
> out, would it be possible to have nnimap send a keepalive message to
> the server?

I don't think so, it would destroy the idea of having idle timers on
the server.

Gnus should cope with destroyed network connections, it shouldn't
include workarounds to make sure the network connection isn't
destroyed.  In theory anyway.  Did this actually cause anything to
fail?  If so, what?  Gnus should simply re-open the connection, I
think.

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Doug Alcorn]

Simon Josefsson <jas@extundo.com> writes:

> Doug Alcorn <doug@lathi.net> writes:
> 
> > Noticing another thread about problems with nnimap connections timeing
> > out, would it be possible to have nnimap send a keepalive message to
> > the server?
>
> Gnus should cope with destroyed network connections, it shouldn't
> include workarounds to make sure the network connection isn't
> destroyed.  

OK, I'll buy that.

> Did this actually cause anything to fail?  If so, what?  Gnus should
> simply re-open the connection, I think.

Well, kindof.  What happens is that the next operation that requires a
connection to the server hangs indefinitely.  The good news is taht
C-g will interrupt it and re-doing the same operation does cause
nnimap to open a new connection.

In practice this is highly annoying.  When I leave Gnus for a while
(in whatever state) and come back I always have to do something, then
interrupt it, then do it again.

Another point.  The warning message or complaint from nnimap doesn't
show up in any buffer I can find.  " *Message-Log*" has no record of
it (even though the message shows up in the minibuffer). " *nnimap*"
is empty as is "*Warnings*".  I can't find any other buffers that
would seem to be related.
-- 
 (__) Doug Alcorn (mailto:doug@lathi.net http://www.lathi.net)
 oo / PGP 02B3 1E26 BCF2 9AAF 93F1  61D7 450C B264 3E63 D543
 |_/  If you're a capitalist and you have the best goods and they're
      free, you don't have to proselytize, you just have to wait. 

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Doug Alcorn]

Doug Alcorn <doug@lathi.net> writes:

> Simon Josefsson <jas@extundo.com> writes:
> 
> > Doug Alcorn <doug@lathi.net> writes:
> > 
> > > Noticing another thread about problems with nnimap connections timeing
> > > out, would it be possible to have nnimap send a keepalive message to
> > > the server?
> >
> > Gnus should cope with destroyed network connections, it shouldn't
> > include workarounds to make sure the network connection isn't
> > destroyed.  
> 
> OK, I'll buy that.
> 
> > Did this actually cause anything to fail?  If so, what?  Gnus should
> > simply re-open the connection, I think.

I've got output from the Message-Log now.

Checking new news...
Opening nntp server on localhost...
Denied server
nnimap: Checking mailboxes...
nnimap: Checking mailbox Mail/sent.archive.12Apr01
Garbage: Read from remote host mail.lathi.net: Connection reset by peer
Quit
Checking new news...
Opening nntp server on localhost...
Denied server
Opening nnimap server...
imap: Connecting to mail.lathi.net...
imap: Opening IMAP connection with `ssh -x -C mail.lathi.net /usr/sbin/imapd'...
imap: Opening IMAP connection with `ssh -x -C mail.lathi.net /usr/sbin/imapd'...done
Waiting for response from mail.lathi.net...done
imap: Connecting to mail.lathi.net...done


Just ignore the nntp stuff.  I used to run leaf-node here locally and
now don't.  I've just never got around to changing my select methods
correctly.
-- 
 (__) Doug Alcorn (mailto:doug@lathi.net http://www.lathi.net)
 oo / PGP 02B3 1E26 BCF2 9AAF 93F1  61D7 450C B264 3E63 D543
 |_/  If you're a capitalist and you have the best goods and they're
      free, you don't have to proselytize, you just have to wait. 

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Kai Großjohann]

On 05 Jun 2001, Simon Josefsson wrote:

> Gnus should cope with destroyed network connections, it shouldn't
> include workarounds to make sure the network connection isn't
> destroyed.  In theory anyway.  Did this actually cause anything to
> fail?  If so, what?  Gnus should simply re-open the connection, I
> think.

Gnus can't know that the connection is destroyed, for here's what I
think is happening: Doug is tunneling his IMAP connection through
ssh.  ssh has connected to the remote host and established the
tunnel.  Now Gnus is opening a connection to the real IMAP server
through this tunnel.  Time passes.  The real IMAP server hits the idle
time out and closes the connection.  But the ssh tunnel is still
there.  Gnus doesn't know anything is wrong -- the ssh process is
still running.  Only when Gnus tries to send a command, ssh finds out
it can't and discovers the tunnel is closed.  But Gnus has already
sent the command and doesn't expect it to fail -- after all,
everything was alright just before sending the command.

I'm not sure this is it, but I think so.  Thoughts?

kai
-- 
~/.signature: No such file or directory

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Kai Großjohann]

On 05 Jun 2001, Kai Großjohann wrote:

> I'm not sure this is it, but I think so.  Thoughts?

Here's one thought: have imap.el memorize when was the last command
sent to the server.  And always before sending a command, look whether
more than N seconds have passed since the last command.  If so,
suspect that the server might have timed out and send a no-op
command.  If the no-op command gets a reply soon, all is well.  But if
there is no timely reply to the no-op, assume the connection has been
severed, close it and open a new one.

How's that sound?

(That's what I'm going to do for Tramp which has the same problem.
But since I haven't done it yet, I don't know how it will go.)

kai
-- 
~/.signature: No such file or directory

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Paul Jarc]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> The real IMAP server hits the idle time out and closes the
> connection.  But the ssh tunnel is still there.

Why?  Can the ssh tunnel be made to go away when the IMAP connection
does?


paul

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Kai Großjohann]

On 05 Jun 2001, Paul Jarc wrote:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
>> The real IMAP server hits the idle time out and closes the
>> connection.  But the ssh tunnel is still there.
> 
> Why?

I wish I knew.  Anyone?

kai
-- 
~/.signature: No such file or directory

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Itai Zukerman]

> [...] Doug is tunneling his IMAP connection through
> ssh.  ssh has connected to the remote host and established the
> tunnel.  Now Gnus is opening a connection to the real IMAP server
> through this tunnel.  Time passes.  The real IMAP server hits the idle
> time out and closes the connection.  But the ssh tunnel is still
> there.

My experience has been this:

1 Do "ssh <host> imapd".

2 Putter along happily without sending anything to the ssh process.

3 The remote end of the connection gets closed (by sshd, not by
  imapd!) but the local ssh client doesn't know that, yet.

4 Try to send a command to the ssh process and you get back a message
  about a connection timeout and the process dies.

Hence the "Garbage..." message.

At least, I think this is what's happening.  Perhaps this is really a
bug in ssh.  Would gnus do the right thing if the process died in #3
above?

-itai

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Kai Großjohann]

On 05 Jun 2001, Itai Zukerman wrote:

> 3 The remote end of the connection gets closed (by sshd, not by
>   imapd!) but the local ssh client doesn't know that, yet.

Why doesn't the remote sshd tell the local ssh client?  It would be
polite to do that.  Looks like a bug to me.

kai
-- 
~/.signature: No such file or directory

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Doug Alcorn]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> On 05 Jun 2001, Itai Zukerman wrote:
> 
> > 3 The remote end of the connection gets closed (by sshd, not by
> >   imapd!) but the local ssh client doesn't know that, yet.
> 
> Why doesn't the remote sshd tell the local ssh client?  It would be
> polite to do that.  Looks like a bug to me.

I think I can confirm that this is a sshd/ssh behavior.  My normal
shells work the same way.  If I leave a ssh idle for a long time, when
I come back to it everything looks OK.  It's not until I send another
command (or another character) that the client learns the connection
is lost.
-- 
 (__) Doug Alcorn (mailto:doug@lathi.net http://www.lathi.net)
 oo / PGP 02B3 1E26 BCF2 9AAF 93F1  61D7 450C B264 3E63 D543
 |_/  If you're a capitalist and you have the best goods and they're
      free, you don't have to proselytize, you just have to wait. 

Re: Tunelling nnimap through ssh shell

[Karl Kleinpaste]

Doug Alcorn <doug@lathi.net> writes:
> If I leave a ssh idle for a long time

How long is "a long time"?  I use ssh with idle connections for hours
and sometimes days at a time, with no ill effects.

Lately, what I'm using is openssh 2.5.2 as distributed with RH7.1, but
I don't recall seeing such a problem with the older ssh 1.2.27, either.

Re: Tunelling nnimap through ssh shell

[Josh Huber]

Karl Kleinpaste <karl@charcoal.com> writes:

> How long is "a long time"?  I use ssh with idle connections for hours
> and sometimes days at a time, with no ill effects.
> 
> Lately, what I'm using is openssh 2.5.2 as distributed with RH7.1, but
> I don't recall seeing such a problem with the older ssh 1.2.27, either.

Yeah, this sounds fishy.

are you *sure* this isn't some NAT box in between that's timing out,
or something of that sort?

-- 
Josh Huber

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Andi Hechtbauer]

Hi,

I think the relevant line in the log you posted is:

>>>>> On 05 Jun 2001 17:11:16 -0400, Doug Alcorn <doug@lathi.net> said:

    Doug> from remote host mail.lathi.net: Connection reset by peer
                                                            ^^^^^^^
This indicates a NAT box having timed out your connection.
Neither imap nor ssh can know this has happened before you
try to access the connection, i.e. send some traffic.

Not sure if gnus is able to catch and work around that.

regards,

-- 
Andi Hechtbauer                                           anti@spin.de 
root                                          voice: +49 941 942 77 23
SPiN AG                 http://www.spin.de/     fax: +49 941 942 77 22  
------ PGPf16: B4 FC 30 E0 31 2F BC F5  38 D0 A2 FC EA A9 82 5D ------

Re: Tunelling nnimap through ssh shell

[Doug Alcorn]

Josh Huber <huber@alum.wpi.edu> writes:

> Karl Kleinpaste <karl@charcoal.com> writes:
> 
> > How long is "a long time"?  I use ssh with idle connections for hours
> > and sometimes days at a time, with no ill effects.
> > 
> > Lately, what I'm using is openssh 2.5.2 as distributed with RH7.1, but
> > I don't recall seeing such a problem with the older ssh 1.2.27, either.
> 
> Yeah, this sounds fishy.
> 
> are you *sure* this isn't some NAT box in between that's timing out,
> or something of that sort?

The timeout is about 20 minutes or so.  No, I'm not sure it isn't NAT.
I do know that NAT is involved.  My IMAP server is sitting behind a
linux firewal that is doing ipmasq between the 'net and the dmz.  I
don't know what the default timeout is; but, I just found the syntax
for increasing it.  Hopefully this will fix my problem.  But it still
leaves folks who can't control their firewalls out of luck.
-- 
 (__) Doug Alcorn (mailto:doug@lathi.net http://www.lathi.net)
 oo / PGP 02B3 1E26 BCF2 9AAF 93F1  61D7 450C B264 3E63 D543
 |_/  If you're a capitalist and you have the best goods and they're
      free, you don't have to proselytize, you just have to wait. 

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Josh Huber]

Andi Hechtbauer <anti@spin.de> writes:

>     Doug> from remote host mail.lathi.net: Connection reset by peer
>                                                             ^^^^^^^
> This indicates a NAT box having timed out your connection.
> Neither imap nor ssh can know this has happened before you
> try to access the connection, i.e. send some traffic.

Yeah, and as far as I recall, the 2.2 kernel IP filtering (ipchains)
had a pretty ridiculously low default timeout.  2.4 (netfilter) is
much more sane in this respect.

ttyl,

-- 
Josh Huber

Re: Tunelling nnimap through ssh shell (Was: nnimap-authinfo-file question)

[Steinar Bang]

>>>>> Doug Alcorn <doug@lathi.net>:

[snip!]
> The only problem I ran into is that my ssh shells timeout with no
> activity.  The ends up with messages about the stream being currupt
> and then showing the ssh connection reset message (sorry I don't
> have a copy of the exact message now).

There seem to be a general problem with long lived nnimap connections.
They get stuck after a while.  I've avoided the problem by using
agent, and spending most of the time in disconnected mode.