nnimap ignoring server options?

nnimap ignoring server options?

[Andrew J Cosgriff]

I'm trying to talk to an Exchange server at my new job, and it's only
accessible via plain-text IMAP.

nnimap seems to want to try TLS first, which is perfectly
understandable, but it never seems to fall back to plaintext on port
143.

I've got the following server config...

(nnimap "some.host.name"
	(nnimap-address "10.x.y.z")
	(nnimap-server-port imap)
	(nnimap-stream network))

...but it seems that these configured things are never used - for
instance, if I change nnimap-address to be "localhost", I still see Gnus
saying in the echo area that it's making a TLS connection to
some.host.name instead.

I can run stunnel on my laptop in order to pretend that I've got SSL
access to Exchange, and it works, but it'd be nice not to have to do this.

Cheers,
 Cos.
-- 
Andrew J Cosgriff <ajc@polydistortion.net> feeling like a robot by the river

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <ajc@polydistortion.net> writes:

> (nnimap "some.host.name"
> 	(nnimap-address "10.x.y.z")
> 	(nnimap-server-port imap)
> 	(nnimap-stream network))
>
> ...but it seems that these configured things are never used - for
> instance, if I change nnimap-address to be "localhost", I still see Gnus
> saying in the echo area that it's making a TLS connection to
> some.host.name instead.

Are you sure that that's the server definition the groups use?  `G E' on
a group to have a look.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>
>> (nnimap "some.host.name"
>> 	(nnimap-address "10.x.y.z")
>> 	(nnimap-server-port imap)
>> 	(nnimap-stream network))
>>
>> ...but it seems that these configured things are never used - for
>> instance, if I change nnimap-address to be "localhost", I still see Gnus
>> saying in the echo area that it's making a TLS connection to
>> some.host.name instead.
>
> Are you sure that that's the server definition the groups use?  `G E' on
> a group to have a look.

...but I can't subscribe to any groups on that server if I can't connect
to it, right?

stepping through some of the nnimap functions, it seems like
nnimap-stream and the other parameters are ignored, yes, as
nnimap-stream says ssl (for example) despite me having set it as shown
above.

-- 
Andrew J Cosgriff <ajc@polydistortion.net> i'm so confused

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <confusenet@polydistortion.net> writes:

>>> (nnimap "some.host.name"
>>> 	(nnimap-address "10.x.y.z")
>>> 	(nnimap-server-port imap)
>>> 	(nnimap-stream network))

[...]

> ...but I can't subscribe to any groups on that server if I can't connect
> to it, right?

So where are you setting this, and how are you attempting to make Gnus
contact this?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

> Andrew J Cosgriff <confusenet@polydistortion.net> writes:
>
>>>> (nnimap "some.host.name"
>>>> 	(nnimap-address "10.x.y.z")
>>>> 	(nnimap-server-port imap)
>>>> 	(nnimap-stream network))
>
> [...]
>
>> ...but I can't subscribe to any groups on that server if I can't connect
>> to it, right?
>
> So where are you setting this, and how are you attempting to make Gnus
> contact this?

Here's what happens with Gnus pulled from git about 15 mintes ago
(say, about Wed Oct 13 22:00:00 UTC 2010), with the top entry in the
Changelog saying

> 2010-10-13  Lars Magne Ingebrigtsen  <larsi@gnus.org>
> 
>         * tls.el (tls-program): Remove spurious %s from openssl.

Right.

I hit "^" in the Group buffer to enter the Server list, and add a new
server by hitting "a".

At that point I enter "nnimap" and the server name, then I hit "e" and
fix it to say something like:

(nnimap "myhostname"
        (nnimap-stream network)
        (nnimap-server-port imap))

I then C-c C-c to save the settings, and I'm back in the Server
buffer. I hit "o" to try and open the server, but it says "Couldn't open
myhostname".

If I edebug nnimap-open-connection, it seems like nnimap-stream still
evaluates to ssl as I step through it. Similarly, nnimap-server-port
is nil, so it ends up deciding to choose imaps.

If it's relevant, I do have another nnimap server defined, which uses
SSL (and hence works fine). However, there are no non-default parameters
configured for it if I hit "e" in the Server buffer. This was also
originally added by hitting "a" in the Server buffer and following the
prompts.

Cheers,
 Cos.
-- 
Andrew J Cosgriff <ajc@polydistortion.net> do not expose to solar flares

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <ajc@polydistortion.net> writes:

> I hit "^" in the Group buffer to enter the Server list, and add a new
> server by hitting "a".
>
> At that point I enter "nnimap" and the server name, then I hit "e" and
> fix it to say something like:
>
> (nnimap "myhostname"
>         (nnimap-stream network)
>         (nnimap-server-port imap))
>
> I then C-c C-c to save the settings, and I'm back in the Server
> buffer. I hit "o" to try and open the server, but it says "Couldn't open
> myhostname".

Yes, I get the same if I do this, so there's a bug.  However, it doesn't
seem to be nnimap-specific, and if I exit Gnus and then restart Gnus
again, I can `O' the server just fine.

So it looks like the server mode editing stuff doesn't flush the backend
data properly.  I haven't looked at that code in a couple of decades,
but I'll have a peek.

So if you exit and start again, do you still get the same result?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

> So if you exit and start again, do you still get the same result?

at that point, nnimap-stream and nnimap-server-port seem to evaluate to
the correct values, but open-network-stream still fails...

-- 
Andrew J Cosgriff <ajc@polydistortion.net> a surreal dilemma

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <ajc@polydistortion.net> writes:

> at that point, nnimap-stream and nnimap-server-port seem to evaluate to
> the correct values, but open-network-stream still fails...

Could you edebug through `nnimap-open-connection' and see what happens?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>
>> at that point, nnimap-stream and nnimap-server-port seem to evaluate to
>> the correct values, but open-network-stream still fails...
>
> Could you edebug through `nnimap-open-connection' and see what
> happens?

yeah, I did - it gets here:

        (message "Opening connection to %s..." nnimap-address)                                      
        (open-network-stream                                                                        
         "*nnimap*" (current-buffer) nnimap-address                                                 
         (setq port                                                                                 
               (or nnimap-server-port                                                               
               (if (netrc-find-service-number "imap")                                               
                   "imap"                                                                           
                 "143"))))                                                                          

...and open-network-stream fails.

that appears to be a tiny wrapper around a C function called
make-network-process. I'm not yet sure why that fails.

-- 
http://andrew.j.cosgriff.name/ | eh ?

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <confusenet@polydistortion.net> writes:

>         (message "Opening connection to %s..." nnimap-address)                                      
>         (open-network-stream                                                                        
>          "*nnimap*" (current-buffer) nnimap-address                                                 
>          (setq port                                                                                 
>                (or nnimap-server-port                                                               
>                (if (netrc-find-service-number "imap")                                               
>                    "imap"                                                                           
>                  "143"))))                                                                          
>
> ...and open-network-stream fails.
>
> that appears to be a tiny wrapper around a C function called
> make-network-process. I'm not yet sure why that fails.

What were the values of nnimap-address and port at this point?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

->> that appears to be a tiny wrapper around a C function called
>> make-network-process. I'm not yet sure why that fails.
>
> What were the values of nnimap-address and port at this point?

aha! it's still passing "imap" instead of translating it into 143.

if i set nnimap-server-port to 143 in the server options instead, it all
works fine.

-- 
Andrew J Cosgriff <ajc@polydistortion.net> exterminate all rational thought !

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <ajc@polydistortion.net> writes:

> aha! it's still passing "imap" instead of translating it into 143.

It's not supposed to -- `open-network-stream' (etc.) should do the
translation (based on /etc/services and the like).

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>
>> aha! it's still passing "imap" instead of translating it into 143.
>
> It's not supposed to -- `open-network-stream' (etc.) should do the
> translation (based on /etc/services and the like).

Well, I guess that's what's not working.

143 works.
imap doesn't.

I can swap between them and reproduce this.

ah, here we go. if I change back to imap and quit, then start gnus, I
get this message:

Warning: Unable to open server nnimap+myhostname due to: Wrong type argument: stringp, imap


...oh, I get it.

trying something slightly different, I see that "imap" (with quotes) works.

it's meant to work non-quoted though, right?

-- 
Andrew J Cosgriff <ajc@polydistortion.net> store at room temperature

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <ajc@polydistortion.net> writes:

> trying something slightly different, I see that "imap" (with quotes) works.
>
> it's meant to work non-quoted though, right?

No, only a string or a number are supposed to work.  The stream
functions don't accept symbols.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

lmi wrote :

> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>
>> trying something slightly different, I see that "imap" (with quotes) works.
>>
>> it's meant to work non-quoted though, right?
>
> No, only a string or a number are supposed to work.  The stream
> functions don't accept symbols.

Hmm. Perhaps the manual ("Customizing the IMAP Connection") needs
clarifying, then?

-- 
Andrew J Cosgriff <ajc@polydistortion.net> feeling like a robot by the river

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Andrew J Cosgriff <ajc@polydistortion.net> writes:

> Hmm. Perhaps the manual ("Customizing the IMAP Connection") needs
> clarifying, then?

Hm...

---
If the server uses a non-standard port, that can be specified here.  A
typical port would be @samp{imap} or @samp{imaps}.
---

Oh, @samp{...} is rendered as `...'.  What's the TeXinfo incantation for
strings?  @code{"..."}?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Katsumi Yamaoka]

Lars Magne Ingebrigtsen wrote:
> Oh, @samp{...} is rendered as `...'.  What's the TeXinfo incantation for
> strings?  @code{"..."}?

echo \`\`foo\'\' | makeinfo --no-headers
echo '@dfn{foo}' | makeinfo --no-headers

I'm not a TeXinfo specialist though. ;-)

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> echo \`\`foo\'\' | makeinfo --no-headers
> echo '@dfn{foo}' | makeinfo --no-headers
>
> I'm not a TeXinfo specialist though. ;-)

I went with @code{"imaps"}.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Andreas Schwab]

Katsumi Yamaoka <yamaoka@jpl.org> writes:

> Lars Magne Ingebrigtsen wrote:
>> Oh, @samp{...} is rendered as `...'.  What's the TeXinfo incantation for
>> strings?  @code{"..."}?
>
> echo \`\`foo\'\' | makeinfo --no-headers
> echo '@dfn{foo}' | makeinfo --no-headers

Just because makeinfo renders it with a pair of ascii double quotes does
not make it a suitable markup (the string is definitely not defining
something).

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: nnimap ignoring server options?

[Andreas Schwab]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>
>> Hmm. Perhaps the manual ("Customizing the IMAP Connection") needs
>> clarifying, then?
>
> Hm...
>
> ---
> If the server uses a non-standard port, that can be specified here.  A
> typical port would be @samp{imap} or @samp{imaps}.
> ---
>
> Oh, @samp{...} is rendered as `...'.  What's the TeXinfo incantation for
> strings?  @code{"..."}?

I'd just say that it is a string or a number.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: nnimap ignoring server options?

[Ted Zlatanov]

On Mon, 18 Oct 2010 23:59:13 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>> trying something slightly different, I see that "imap" (with quotes) works.
>> 
>> it's meant to work non-quoted though, right?

LMI> No, only a string or a number are supposed to work.  The stream
LMI> functions don't accept symbols.

That seems easy to fix:

(when (symbolp port) (setq port (symbol-name port)))

...and I think it would be a good fit, generally, since service names
tend to be within the symbol character set.  Any reasons not to do this?

Ted

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> That seems easy to fix:
>
> (when (symbolp port) (setq port (symbol-name port)))
>
> ...and I think it would be a good fit, generally, since service names
> tend to be within the symbol character set.  Any reasons not to do this?

This should be done in the code that maps service names to numbers to
avoid having some taking 'imaps and other taking "imaps".  That would
probably be Fmake_network_process.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Ted Zlatanov]

On Tue, 19 Oct 2010 19:54:42 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> That seems easy to fix:
>> 
>> (when (symbolp port) (setq port (symbol-name port)))
>> 
>> ...and I think it would be a good fit, generally, since service names
>> tend to be within the symbol character set.  Any reasons not to do this?

LMI> This should be done in the code that maps service names to numbers to
LMI> avoid having some taking 'imaps and other taking "imaps".  That would
LMI> probably be Fmake_network_process.

Here's a patch.  It also corrects the docs (getservbyname will be called
if getaddrinfo is not available, so either way a string will work, which
is not what the docs say).  WDYT?

Ted

=== modified file 'src/process.c'
--- src/process.c       2010-10-08 10:14:47 +0000
+++ src/process.c       2010-10-19 18:26:51 +0000
@@ -2978,10 +2978,11 @@
 host, and only clients connecting to that address will be accepted.
 
 :service SERVICE -- SERVICE is name of the service desired, or an
-integer specifying a port number to connect to.  If SERVICE is t,
-a random port number is selected for the server.  (If Emacs was
-compiled with getaddrinfo, a port number can also be specified as a
-string, e.g. "80", as well as an integer.  This is not portable.)
+integer specifying a port number to connect to.  If SERVICE is t, a
+random port number is selected for the server.  A port number can also
+be specified as a string, e.g. "80", or a symbol whose name will be
+used, as well as an integer.  This is not necessarily portable; either
+getaddrinfo or getservbyname will be used to look up the port number.
 
 :type TYPE -- TYPE is the type of connection.  The default (nil) is a
 stream type connection, `datagram' creates a datagram type connection,
@@ -3303,6 +3304,11 @@
      Otherwise, use getservbyname to lookup the service.  */
   if (!NILP (host))
     {
+      /* Take a symbol as the service and convert it to a string.  */
+      if (SYMBOLP (service))
+       {
+         service = Fsymbol_name (service);
+       }
 
       /* SERVICE can either be a string or int.
         Convert to a C string for later use by getaddrinfo.  */
@@ -3347,6 +3353,12 @@
   /* We end up here if getaddrinfo is not defined, or in case no hostname
      has been specified (e.g. for a local server process).  */
 
+  /* Take a symbol as the service and convert it to a string.  */
+  if (SYMBOLP (service))
+    {
+      service = Fsymbol_name (service);
+    }
+
   if (EQ (service, Qt))
     port = 0;
   else if (INTEGERP (service))

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Here's a patch.  It also corrects the docs (getservbyname will be called
> if getaddrinfo is not available, so either way a string will work, which
> is not what the docs say).  WDYT?

Hehe.  I posted kinda the same thing on the emacs-devel list just
now...  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: nnimap ignoring server options?

[Bruno Tavernier]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Andrew J Cosgriff <ajc@polydistortion.net> writes:
>
>> (nnimap "some.host.name"
>> 	(nnimap-address "10.x.y.z")
>> 	(nnimap-server-port imap)
>> 	(nnimap-stream network))
>>
>> ...but it seems that these configured things are never used - for
>> instance, if I change nnimap-address to be "localhost", I still see Gnus
>> saying in the echo area that it's making a TLS connection to
>> some.host.name instead.
>
> Are you sure that that's the server definition the groups use?  `G E' on
> a group to have a look.

I think the culprit is that Gnus force starttls if the server is
starttls capable. No matter if `nnimap-stream' is set to network.
Connecting from a vpn, ssh or localhost to a starttls capable server
*always* enable starttls.

I am refering to this portion of the code in nnimap.el
,----
| ;; If this is a STARTTLS-capable server, then sever the
| ;; connection and start a STARTTLS connection instead.
| (cond
|  ((and (or (and (eq nnimap-stream 'network)
|   (member "STARTTLS"
|    (nnimap-capabilities nnimap-object)))
`----

Could we assume that users who set (nnimap-stream network) know what
they're doing? :-)

-- 
Bruno

Re: nnimap ignoring server options?

[Andrew J Cosgriff]

Bruno Tavernier <tavernier.bruno <at> gmail.com> writes:

> I think the culprit is that Gnus force starttls if the server is
> starttls capable. No matter if `nnimap-stream' is set to network.
> Connecting from a vpn, ssh or localhost to a starttls capable server
> *always* enable starttls.

no, that's not it. here's our Exchange server's capabilities:

* OK Microsoft Exchange Server 2003 IMAP4rev1 server version 6.5.7638.1
(myhostname.company) ready.
. capability
* CAPABILITY IMAP4 IMAP4rev1 IDLE LOGIN-REFERRALS MAILBOX-REFERRALS
NAMESPACE LITERAL+ UIDPLUS CHILDREN AUTH=NTLM
. OK CAPABILITY completed.

Re: nnimap ignoring server options?

[Lars Magne Ingebrigtsen]

Bruno Tavernier <tavernier.bruno@gmail.com> writes:

> Could we assume that users who set (nnimap-stream network) know what
> they're doing? :-)

We could, but the default in the old nnimap is to use starttls if it is
available. 

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen