allow GSSAPI for imap access

allow GSSAPI for imap access

[Jochen Hein]

These patches allow GSSAPI access to IMAP servers.  The patches are
against current master.  Comments are welcome.

[PATCH 1/4] open-gssapi-stream: remove parameter user

[Jochen Hein]

Remove the parameter user from open-gssapi-stream, the processing of
that parameter and '?l' from the gssapi-program template.

Now we can call open-gssapi-stream from open-network-stream.

	* lisp/gnus/gssapi.el (gssapi-program): remove username in
	command templates.
	(open-gssapi-stream): remove the parameter user, so we can
	call open-gssapi-stream from open-network-stream.
---
 lisp/gnus/gssapi.el | 11 ++++-------
 1 file changed, 4 insertions(+), 7 deletions(-)

diff --git a/lisp/gnus/gssapi.el b/lisp/gnus/gssapi.el
index 1f72805..08b2ec3 100644
--- a/lisp/gnus/gssapi.el
+++ b/lisp/gnus/gssapi.el
@@ -29,9 +29,8 @@
 
 (defcustom gssapi-program (list
 			   (concat "gsasl %s %p "
-				   "--mechanism GSSAPI "
-				   "--authentication-id %l")
-			   "imtest -m gssapi -u %l -p %p %s")
+				   "--mechanism GSSAPI ")
+			   "imtest -m gssapi -p %p %s")
   "List of strings containing commands for GSSAPI (krb5) authentication.
 %s is replaced with server hostname, %p with port to connect to,
 and %l with the user name.  The program should accept commands on
@@ -41,7 +40,7 @@ gssapi-program
   :group 'network
   :type '(repeat string))
 
-(defun open-gssapi-stream (name buffer server port user)
+(defun open-gssapi-stream (name buffer server port)
   (let ((cmds gssapi-program)
 	cmd done)
     (with-current-buffer buffer
@@ -57,8 +56,7 @@ open-gssapi-stream
 			  cmd
 			  (format-spec-make
 			   ?s server
-			   ?p (number-to-string port)
-			   ?l user))))
+			   ?p (number-to-string port)))))
 	       response)
 	  (when process
 	    (while (and (memq (process-status process) '(open run))
@@ -92,7 +90,6 @@ open-gssapi-stream
 				  (setq response (match-string 1)))))
 	      (accept-process-output process 1)
 	      (sit-for 1))
-	    (erase-buffer)
 	    (message "GSSAPI connection: %s" (or response "failed"))
 	    (if (and response (let ((case-fold-search nil))
 				(not (string-match "failed" response))))
-- 
2.1.4

[PATCH 2/4] network-stream: allow type gssapi

[Jochen Hein]

	* lisp/net/network-stram.el (open-network-stream): allow type 'gssapi.
	(network-stream-open-gssapi): new function.
---
 lisp/net/network-stream.el | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/lisp/net/network-stream.el b/lisp/net/network-stream.el
index e5557b8..3f1d9fd 100644
--- a/lisp/net/network-stream.el
+++ b/lisp/net/network-stream.el
@@ -44,6 +44,7 @@
 
 (require 'tls)
 (require 'starttls)
+(require 'gssapi)
 (require 'auth-source)
 (require 'nsm)
 (require 'puny)
@@ -87,6 +88,7 @@ open-network-stream
   `tls'      -- A TLS connection.
   `ssl'      -- Equivalent to `tls'.
   `shell'    -- A shell connection.
+  `gssapi'   -- a GSSAPI connection.
 
 :return-list specifies this function's return value.
   If omitted or nil, return a process object.  A non-nil means to
@@ -161,6 +163,7 @@ open-network-stream
 			'network-stream-open-starttls)
 		       ((memq type '(tls ssl)) 'network-stream-open-tls)
 		       ((eq type 'shell) 'network-stream-open-shell)
+		       ((eq type 'gssapi) 'network-stream-open-gssapi)
 		       (t (error "Invalid connection type %s" type))))
 	    result)
 	(unwind-protect
@@ -177,6 +180,24 @@ open-network-stream
 		  :error        (nth 4 result))
 	  (car result))))))
 
+(defun network-stream-open-gssapi (name buffer host service parameters)
+  (let* ((start (with-current-buffer buffer (point)))
+	 (capability-command  (plist-get parameters :capability-command))
+	 (eoc                 (plist-get parameters :end-of-command))
+	 (eo-capa             (or (plist-get parameters :end-of-capability)
+				  eoc))
+	 (stream (open-gssapi-stream name buffer host service))
+         (greeting (network-stream-get-response stream start eoc))
+         (capabilities (when capability-command
+			 (network-stream-command stream
+					       capability-command
+					       (or eo-capa eoc)))))
+	 ;; Return (STREAM GREETING CAPABILITIES RESULTING-TYPE)
+	 (list stream
+	       greeting
+	       capabilities
+	       'gssapi)))
+
 (defun network-stream-certificate (host service parameters)
   (let ((spec (plist-get :client-certificate parameters)))
     (cond
-- 
2.1.4

[PATCH 3/4] nnimap: handle imap connections with type 'gssapi

[Jochen Hein]

	* lisp/gnus/nnimap.el (nnimap-open-connection-1): handle imap
        connections with type gssapi.
---
 lisp/gnus/nnimap.el | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/lisp/gnus/nnimap.el b/lisp/gnus/nnimap.el
index fc9304f..a92ed76 100644
--- a/lisp/gnus/nnimap.el
+++ b/lisp/gnus/nnimap.el
@@ -58,7 +58,7 @@ nnimap-server-port
 (defvoo nnimap-stream 'undecided
   "How nnimap talks to the IMAP server.
 The value should be either `undecided', `ssl' or `tls',
-`network', `starttls', `plain', or `shell'.
+`network', `starttls', `plain', `gssapi', or `shell'.
 
 If the value is `undecided', nnimap tries `ssl' first, then falls
 back on `network'.")
@@ -407,6 +407,10 @@ nnimap-open-connection-1
 	      (nnheader-message 7 "Opening connection to %s via shell..."
 				nnimap-address)
 	      '("imap"))
+	     ((eq nnimap-stream 'gssapi)
+	      (nnheader-message 7 "Opening connection to %s via GSSAPI..."
+				nnimap-address)
+	      '(143))
 	     ((memq nnimap-stream '(ssl tls))
 	      (nnheader-message 7 "Opening connection to %s via tls..."
 				nnimap-address)
@@ -460,7 +464,9 @@ nnimap-open-connection-1
 	    (setf (nnimap-capabilities nnimap-object)
 		  (mapcar #'upcase
 			  (split-string capabilities)))
-	    (unless (string-match-p "[*.] PREAUTH" greeting)
+	    (unless (or
+			(eq nnimap-stream 'gssapi)
+			(string-match-p "[*.] PREAUTH" greeting))
 	      (if (not (setq credentials
 			     (if (eq nnimap-authenticator 'anonymous)
 				 (list "anonymous"
-- 
2.1.4

[PATCH 4/4] update nnimap documentation for gssapi connections

[Jochen Hein]

       * doc/misc/gnus.texi: update nnimap docs for gssapi.
---
 doc/misc/gnus.texi | 24 +++++++++++++++---------
 1 file changed, 15 insertions(+), 9 deletions(-)

diff --git a/doc/misc/gnus.texi b/doc/misc/gnus.texi
index fa7cd09..a22ee3e 100644
--- a/doc/misc/gnus.texi
+++ b/doc/misc/gnus.texi
@@ -14237,7 +14237,7 @@ Customizing the IMAP Connection
 This is the default, and this first tries the @code{ssl} setting, and
 then tries the @code{network} setting.
 
-@item ssl
+@item ssl or tls
 This uses standard @acronym{TLS}/@acronym{SSL} connections.
 
 @item network
@@ -14257,6 +14257,11 @@ Customizing the IMAP Connection
 Non-encrypted and unsafe straight socket connection.
 @acronym{STARTTLS} will not be used even if it is available.
 
+@item gssapi
+Connect with GSSAPI by using the commands @code{gsasl} or
+@code{imtest}.  The commands can be customized by changing
+@code{gssapi-program}.
+
 @end table
 
 @item nnimap-authenticator
@@ -14981,14 +14986,15 @@ Mail Source Specifiers
 @item :stream
 What stream to use for connecting to the server, this is one of the
 symbols in @code{imap-stream-alist}.  Right now, this means
-@samp{gssapi}, @samp{kerberos4}, @samp{starttls}, @samp{tls},
-@samp{ssl}, @samp{shell} or the default @samp{network}.
-
-@item :authentication
-Which authenticator to use for authenticating to the server, this is
-one of the symbols in @code{imap-authenticator-alist}.  Right now,
-this means @samp{gssapi}, @samp{kerberos4}, @samp{digest-md5},
-@samp{cram-md5}, @samp{anonymous} or the default @samp{login}.
+@samp{undecided}, @samp{gssapi}, @samp{starttls}, @samp{tls},
+@samp{ssl}, @samp{shell}, @samp{plain} or the default @samp{network}.
+If the value is @samp{undecided}, nnimap tries @samp{ssl} first, then falls
+back on @samp{network}.
+
+@item :authenticator
+Which authenticator to use for authenticating to the server.  Right now,
+this means @samp{anonymous}, @samp{login}, @samp{plain},
+@samp{cram-md5}, or the default @samp{nil}.
 
 @item :program
 When using the @samp{shell} :stream, the contents of this variable is
-- 
2.1.4

Re: allow GSSAPI for imap access

[Lars Ingebrigtsen]

Jochen Hein <jochen@jochen.org> writes:

> These patches allow GSSAPI access to IMAP servers.  The patches are
> against current master.  Comments are welcome.

Look good, I think.  Does it work?  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: allow GSSAPI for imap access

[Adam Sjøgren]

Lars writes:

> Jochen Hein <jochen@jochen.org> writes:
>
>> These patches allow GSSAPI access to IMAP servers.  The patches are
>> against current master.  Comments are welcome.
>
> Look good, I think.  Does it work?  :-)

Has anyone tried it against Microsoft Exchange?


  Best regards,

    Adam

-- 
 "I always liked songs with parentheses in the title."        Adam Sjøgren
                                                         asjo@koldfront.dk

Re: allow GSSAPI for imap access

[Hein, Jochen]

Am 2016-02-15 08:59, schrieb Lars Ingebrigtsen:
> Jochen Hein <jochen@jochen.org> writes:
> 
>> These patches allow GSSAPI access to IMAP servers.  The patches are
>> against current master.  Comments are welcome.
> 
> Look good, I think.  Does it work?  :-)

Yes, that's the code I run on my system @home to access my kolab server.

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: allow GSSAPI for imap access

[Lars Ingebrigtsen]

"Hein, Jochen" <jochen@jochen.org> writes:

> Yes, that's the code I run on my system @home to access my kolab server.

Great, I'll apply it to the Emacs trunk tomorrow (Australian time).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: allow GSSAPI for imap access

[Jochen Hein]

asjo@koldfront.dk (Adam Sjøgren) writes:

> Has anyone tried it against Microsoft Exchange?

I only have Kolab here - cyrus-imapd is the IMAP server.  Kerberos is
MIT integrated in FreeIPA, both Kolab server and Emacs host are Domain
members in the FreeIPA domain.  No Windows domain here...

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: allow GSSAPI for imap access

[Adam Sjøgren]

Jochen writes:

> asjo@koldfront.dk (Adam Sjøgren) writes:
>
>> Has anyone tried it against Microsoft Exchange?
>
> I only have Kolab here - cyrus-imapd is the IMAP server.  Kerberos is
> MIT integrated in FreeIPA, both Kolab server and Emacs host are Domain
> members in the FreeIPA domain.  No Windows domain here...

I will try it against Microsoft Exchange at work at some point, when I
get time to build an Emacs from git...

Would be nice if it worked, then there would be one less place to keep
typing in my password :-)


  Best regards,

    Adam

-- 
 "Unserious to the end                                        Adam Sjøgren
  As if I had a choice"                                  asjo@koldfront.dk

Re: allow GSSAPI for imap access

[Jochen Hein]

asjo@koldfront.dk (Adam Sjøgren) writes:

> I will try it against Microsoft Exchange at work at some point, when I
> get time to build an Emacs from git...
>
> Would be nice if it worked, then there would be one less place to keep
> typing in my password :-)

You'll also need gsasl with kerberos enabled or imtest (from
cyrus-utils).  I have no idea if there are windows binaries available.
If there are, get a kerberos ticket and try "imtest -m gssapi -p imap
<hostname>" or "gsasl <hostname> imap --mechanism GSSAPI".

Please have a look at
http://www.gnu.org/software/gsasl/manual/html_node/Installing-under-Windows.html
http://www.gnu.org/software/gsasl/manual/html_node/Kerberos-on-Windows.html

If that works we'll see what gnus does with it.  I also have patches for
emacs 24.4 and the last Ma Gnus available.

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: allow GSSAPI for imap access

[Adam Sjøgren]

Jochen writes:

> asjo@koldfront.dk (Adam Sjøgren) writes:

>> I will try it against Microsoft Exchange at work at some point, when I
>> get time to build an Emacs from git...
>>
>> Would be nice if it worked, then there would be one less place to keep
>> typing in my password :-)

> You'll also need gsasl with kerberos enabled or imtest (from
> cyrus-utils).  I have no idea if there are windows binaries available.

I'm not running Windows, I just have to fetch email from Microsoft
Exchange at work.

Do you know if the gsasl in Ubuntu 14.04 works - it is v1.8.0?

It looks like imtest in Ubuntu 14.04 is in cyrus-clients v2.4.17.

> If there are, get a kerberos ticket and try "imtest -m gssapi -p imap
> <hostname>" or "gsasl <hostname> imap --mechanism GSSAPI".

I will try to give it a go when I get a chance, thanks!

> Please have a look at
> http://www.gnu.org/software/gsasl/manual/html_node/Installing-under-Windows.html
> http://www.gnu.org/software/gsasl/manual/html_node/Kerberos-on-Windows.html

Aiieee, no, never, ugh, no, no! :-)


  Thanks!

   Adam

-- 
 "It was called Tinderbox so we could make the joke,          Adam Sjøgren
  'the tree is on fire'. Puns are very important in      asjo@koldfront.dk
  naming tools."

Re: allow GSSAPI for imap access

[Jochen Hein]

asjo@koldfront.dk (Adam Sjøgren) writes:

> I'm not running Windows, I just have to fetch email from Microsoft
> Exchange at work.

I see.

> Do you know if the gsasl in Ubuntu 14.04 works - it is v1.8.0?

I didn't but since I have Ubuntu 14.04 installed on some machines, I had
a look:

$ apt-cache policy gsasl
gsasl:
  Installiert:           1.8.0-2ubuntu2
  Installationskandidat: 1.8.0-2ubuntu2
  Versionstabelle:
 *** 1.8.0-2ubuntu2 0
        500 http://de.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
        100 /var/lib/dpkg/status

> It looks like imtest in Ubuntu 14.04 is in cyrus-clients v2.4.17.
>
>> If there are, get a kerberos ticket and try "imtest -m gssapi -p imap
>> <hostname>" or "gsasl <hostname> imap --mechanism GSSAPI".
>
> I will try to give it a go when I get a chance, thanks!

gsasl on Ubuntu 14.04 worked here without problems.

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: allow GSSAPI for imap access

[Lars Ingebrigtsen]

Jochen Hein <jochen@jochen.org> writes:

> These patches allow GSSAPI access to IMAP servers.  The patches are
> against current master.  Comments are welcome.

I forgot to ask whether you have FSF copyright assignment papers on
file...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: allow GSSAPI for imap access

[Hein, Jochen]

Am 2016-02-16 06:57, schrieb Lars Ingebrigtsen:
> Jochen Hein <jochen@jochen.org> writes:
> 
>> These patches allow GSSAPI access to IMAP servers.  The patches are
>> against current master.  Comments are welcome.
> 
> I forgot to ask whether you have FSF copyright assignment papers on
> file...

I'm unsure - I have translator papers on file, and maybe some from the 
ninetys.
Shall I just file papers?

On the other hand the changes are either trivial or cut&paste 
(network-stream)...

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: allow GSSAPI for imap access

[Lars Ingebrigtsen]

"Hein, Jochen" <jochen@jochen.org> writes:

> I'm unsure - I have translator papers on file, and maybe some from the
> ninetys.

I've looked at the copyright assignment file, and there doesn't seem to
be anything that covers Emacs.

> Shall I just file papers?

Yes, please do.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: allow GSSAPI for imap access

[Adam Sjøgren]

Jochen writes:

> If there are, get a kerberos ticket and try "imtest -m gssapi -p imap
> <hostname>" or "gsasl <hostname> imap --mechanism GSSAPI".

Unfortunately it didn't work:

  $ imtest -m gssapi -p imap SERVER
  S: * OK Microsoft Exchange Server 2007 IMAP4 service ready
  C: C01 CAPABILITY
  S: * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI LOGINDISABLED STARTTLS IDLE NAMESPACE LITERAL+
  S: C01 OK CAPABILITY completed.
  Authentication failed. no mechanism available
  Security strength factor: 0

and:

  $ gsasl SERVER imap --mechanism GSSAPI
  Trying ‘SERVER’...
  * OK Microsoft Exchange Server 2007 IMAP4 service ready
  . CAPABILITY
  * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI LOGINDISABLED STARTTLS IDLE NAMESPACE LITERAL+
  . OK CAPABILITY completed.
  . STARTTLS
  . OK Begin TLS negotiation now.
  . CAPABILITY
  * CAPABILITY IMAP4 IMAP4rev1 AUTH=NTLM AUTH=GSSAPI AUTH=PLAIN IDLE NAMESPACE LITERAL+
  . OK CAPABILITY completed.
  . AUTHENTICATE GSSAPI
  +
  error: server did not return a token

Probably something on the Microsoft Exchange-side...


  Best regards,

    Adam

-- 
 "I used to feel Emacs' way strange. Maybe updating           Adam Sjøgren
  the copyright years of Walt Disney's works every year  asjo@koldfront.dk
  is nonsense. But Emacs' way is alike to it."

Re: allow GSSAPI for imap access

[Jochen Hein]

asjo@koldfront.dk (Adam Sjøgren) writes:

> Jochen writes:
>
>> If there are, get a kerberos ticket and try "imtest -m gssapi -p imap
>> <hostname>" or "gsasl <hostname> imap --mechanism GSSAPI".
>
> Unfortunately it didn't work:

Not nice.

>   $ imtest -m gssapi -p imap SERVER

imtest has an option "-v" - may be that has some more information.

>   $ gsasl SERVER imap --mechanism GSSAPI

Here you might try "--verbose".

> Probably something on the Microsoft Exchange-side...

Do you have access to the kerberos logs?  Does thunderbird work with
GSSAPI with your exchange server?

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.

Re: allow GSSAPI for imap access

[Adam Sjøgren]

Jochen writes:

> imtest has an option "-v" - may be that has some more information.

[gsasl]
> Here you might try "--verbose".

Output is exactly the same for both, unfortunately.

> Do you have access to the kerberos logs?

Nope, not even close.

> Does thunderbird work with GSSAPI with your exchange server?

Just tried, no it doesn't, so I guess that gets my hopes down...


  Best regards,

    Adam

-- 
 "I got into the driver’s seat                                Adam Sjøgren
  And I drove down 42nd Street                           asjo@koldfront.dk
  In my Cadillac
  Good car to drive after a war"

Re: allow GSSAPI for imap access

[Jochen Hein/Jochen Kellner]

Hello Lars,

Lars Ingebrigtsen <larsi@gnus.org> writes:

> "Hein, Jochen" <jochen@jochen.org> writes:
>
>> I'm unsure - I have translator papers on file, and maybe some from the
>> ninetys.
>
> I've looked at the copyright assignment file, and there doesn't seem to
> be anything that covers Emacs.
>
>> Shall I just file papers?
>
> Yes, please do.

Now my papers are on file. Anything else I should do?

Jochen

-- 
The only problem with troubleshooting is that the trouble shoots back.