smime-cert-by-ldap-1 problem

smime-cert-by-ldap-1 problem

[David S. Goldberg]

According to this bit from smime.el:

	  ;; Certificates on LDAP servers _should_ be in DER format,
	  ;; but there are some servers out there that distributes the
	  ;; certificates in PEM format (with or without
	  ;; header/footer) so we try to handle them anyway.
	  (if (or (string= (substring (cadaar ldapresult) 0 27)
			   "-----BEGIN CERTIFICATE-----")
		  (condition-case nil
		      (base64-decode-string (cadaar ldapresult))
		    (error nil)))
	      (setq cert
		    (smime-replace-in-string
		     (cadaar ldapresult)
		     (concat "\\(\n\\|\r\\|-----BEGIN CERTIFICATE-----\\|"
			     "-----END CERTIFICATE-----\\)")
		     "" t))
	    (setq cert (base64-encode-string (cadaar ldapresult) t)))

then the result of smime-cert-by-ldap should be a base64 encoded block
surrounded by the BEGIN and END CERTIFICATE lines.  Instead, I get the
DER with extra CRs inserted every 64 bytes, surrounded by the BEGIN
and END lines.  The problem is that base64-decode-string does not
throw an error, and from what I can see in base64.el, it's not
designed to.

My XEmacs is:

XEmacs 21.4 (patch 17) "Jumbo Shrimp" [Lucid] (i686-pc-cygwin, Mule)
of Wed Jun 1 2005 on mm117419-pc

with the 5/2005 sumo.

Interesting enough, the DER with the extra CRs seems to work OK though
I have to wonder if it's related to the CRs that in the magic number
issue I wrote about earlier.

I discovered this because I want to save the ldap search results to
the local file convention I've developed for myself but having the
slightly messed up DER format messes up my certificate verification
checks.

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: smime-cert-by-ldap-1 problem

[Arne Jørgensen]

david.goldberg6@verizon.net (David S. Goldberg) writes:

> According to this bit from smime.el:
>
> 	  ;; Certificates on LDAP servers _should_ be in DER format,
> 	  ;; but there are some servers out there that distributes the
> 	  ;; certificates in PEM format (with or without
> 	  ;; header/footer) so we try to handle them anyway.
> 	  (if (or (string= (substring (cadaar ldapresult) 0 27)
> 			   "-----BEGIN CERTIFICATE-----")
> 		  (condition-case nil
> 		      (base64-decode-string (cadaar ldapresult))
> 		    (error nil)))
> 	      (setq cert
> 		    (smime-replace-in-string
> 		     (cadaar ldapresult)
> 		     (concat "\\(\n\\|\r\\|-----BEGIN CERTIFICATE-----\\|"
> 			     "-----END CERTIFICATE-----\\)")
> 		     "" t))
> 	    (setq cert (base64-encode-string (cadaar ldapresult) t)))
>
> then the result of smime-cert-by-ldap should be a base64 encoded block
> surrounded by the BEGIN and END CERTIFICATE lines. 

Correct.

> Instead, I get the DER with extra CRs inserted every 64 bytes,
> surrounded by the BEGIN and END lines. The problem is that
> base64-decode-string does not throw an error, and from what I can
> see in base64.el, it's not designed to.

Is base64-decode-string on XEmacs different from the one in Emacs?

As I remember base64-decode-string does throw an error on Emacs. I
will look deeper in to this tomorrow or the day after.

> Interesting enough, the DER with the extra CRs seems to work OK though

That's weird I think.

> I have to wonder if it's related to the CRs that in the magic number
> issue I wrote about earlier.

I don't think so.

Kind regards,
-- 
Arne Jørgensen <http://arnested.dk/>

Re: smime-cert-by-ldap-1 problem

[Arne Jørgensen]

[-- Attachment #1: Type: text/plain, Size: 2464 bytes --]

Arne Jørgensen <arne@arnested.dk> writes:

> david.goldberg6@verizon.net (David S. Goldberg) writes:
>
>> According to this bit from smime.el:
>>
>> 	  ;; Certificates on LDAP servers _should_ be in DER format,
>> 	  ;; but there are some servers out there that distributes the
>> 	  ;; certificates in PEM format (with or without
>> 	  ;; header/footer) so we try to handle them anyway.
>> 	  (if (or (string= (substring (cadaar ldapresult) 0 27)
>> 			   "-----BEGIN CERTIFICATE-----")
>> 		  (condition-case nil
>> 		      (base64-decode-string (cadaar ldapresult))
>> 		    (error nil)))
>> 	      (setq cert
>> 		    (smime-replace-in-string
>> 		     (cadaar ldapresult)
>> 		     (concat "\\(\n\\|\r\\|-----BEGIN CERTIFICATE-----\\|"
>> 			     "-----END CERTIFICATE-----\\)")
>> 		     "" t))
>> 	    (setq cert (base64-encode-string (cadaar ldapresult) t)))
>>
>> then the result of smime-cert-by-ldap should be a base64 encoded block
>> surrounded by the BEGIN and END CERTIFICATE lines. 
>
> Correct.
>
>> Instead, I get the DER with extra CRs inserted every 64 bytes,
>> surrounded by the BEGIN and END lines. The problem is that
>> base64-decode-string does not throw an error, and from what I can
>> see in base64.el, it's not designed to.
>
> Is base64-decode-string on XEmacs different from the one in Emacs?

It is. The doc string for `base64-decode-string' on XEmacs in addition
says "Characters out of the base64 alphabet are ignored" and that
makes XEmacs a success on string that Emacs will fail on.

I still think it's quite surprising that your DER encoded certificate
is able to be base64 decoded in XEmacs though.

Well, one solution is to not rely on `base64-decode-string' only for
detecting if the certificate is in PEM format and instead base64
encode the decode string an compare the before and after strings.
That's what the first patch does.

Another solution is to look for a magic string ("MII") at the
beginning of the certificate. I have not been able to find any
authoritative documentation stating that a certificate must begin with
"MII" (in base64 encoding), but googling for "x509 certificate MII" is
not against it.

The second patch implements this (and is definitely more elegant).

(Just for the record: the case in which the problem is when we try to
detect if the retrieved certificate is in PEM format but without the
"-----BEGIN CERTIFICATE-----" header).

Kind regards,
-- 
Arne Jørgensen <http://arnested.dk/>


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: base64-encode-string after base64-decode-string --]
[-- Type: text/x-patch, Size: 1380 bytes --]

Index: lisp/ChangeLog
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/ChangeLog,v
retrieving revision 7.721
diff -u -p -r7.721 ChangeLog
--- lisp/ChangeLog	16 Jun 2005 06:01:42 -0000	7.721
+++ lisp/ChangeLog	16 Jun 2005 08:19:23 -0000
@@ -1,3 +1,9 @@
+2005-06-16  Arne J^[,Ax^[(Brgensen  <arne@arnested.dk>
+
+	* smime.el (smime-cert-by-ldap-1): Detect PEM format by doing a
+	`base64-encode-string' of the `base64-decode-string'ed string
+	because XEmacs doesn't always fail on `base64-decode-string'.
+
 2005-06-16  Miles Bader  <miles@gnu.org>
 
 	* gnus-xmas.el (gnus-xmas-group-startup-message):
Index: lisp/smime.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/smime.el,v
retrieving revision 7.14
diff -u -p -r7.14 smime.el
--- lisp/smime.el	31 May 2005 13:01:59 -0000	7.14
+++ lisp/smime.el	16 Jun 2005 08:19:23 -0000
@@ -600,7 +600,10 @@ A string or a list of strings is returne
 	  (if (or (string= (substring (cadaar ldapresult) 0 27)
 			   "-----BEGIN CERTIFICATE-----")
 		  (condition-case nil
-		      (base64-decode-string (cadaar ldapresult))
+		      (string=
+		       (cadaar ldapresult)
+		       (base64-encode-string
+			(base64-decode-string (cadaar ldapresult))))
 		    (error nil)))
 	      (setq cert
 		    (smime-replace-in-string

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #3: look for magic MII in certificate --]
[-- Type: text/x-patch, Size: 1320 bytes --]

Index: lisp/ChangeLog
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/ChangeLog,v
retrieving revision 7.721
diff -u -p -r7.721 ChangeLog
--- lisp/ChangeLog	16 Jun 2005 06:01:42 -0000	7.721
+++ lisp/ChangeLog	16 Jun 2005 08:27:12 -0000
@@ -1,3 +1,8 @@
+2005-06-16  Arne J^[,Ax^[(Brgensen  <arne@arnested.dk>
+
+	* smime.el (smime-cert-by-ldap-1): Detect PEM format without
+	header by looking for magic "MII" at the beginnig.
+
 2005-06-16  Miles Bader  <miles@gnu.org>
 
 	* gnus-xmas.el (gnus-xmas-group-startup-message):
Index: lisp/smime.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/smime.el,v
retrieving revision 7.14
diff -u -p -r7.14 smime.el
--- lisp/smime.el	31 May 2005 13:01:59 -0000	7.14
+++ lisp/smime.el	16 Jun 2005 08:27:13 -0000
@@ -599,9 +599,8 @@ A string or a list of strings is returne
 	  ;; header/footer) so we try to handle them anyway.
 	  (if (or (string= (substring (cadaar ldapresult) 0 27)
 			   "-----BEGIN CERTIFICATE-----")
-		  (condition-case nil
-		      (base64-decode-string (cadaar ldapresult))
-		    (error nil)))
+		  (string= (substring (cadaar ldapresult) 0 3)
+			   "MII"))
 	      (setq cert
 		    (smime-replace-in-string
 		     (cadaar ldapresult)

Re: smime-cert-by-ldap-1 problem

[Simon Josefsson]

Arne Jørgensen <arne@arnested.dk> writes:

> Another solution is to look for a magic string ("MII") at the
> beginning of the certificate. I have not been able to find any
> authoritative documentation stating that a certificate must begin with
> "MII" (in base64 encoding), but googling for "x509 certificate MII" is
> not against it.

I suspect only degenerative x.509 certs don't start in this way...

I applied your second patch.  Was that you think is best?

Re: smime-cert-by-ldap-1 problem

[Arne Jørgensen]

Simon Josefsson <jas@extundo.com> writes:

> Arne Jørgensen <arne@arnested.dk> writes:
>
>> Another solution is to look for a magic string ("MII") at the
>> beginning of the certificate. I have not been able to find any
>> authoritative documentation stating that a certificate must begin with
>> "MII" (in base64 encoding), but googling for "x509 certificate MII" is
>> not against it.
>
> I suspect only degenerative x.509 certs don't start in this way...

Probably.

> I applied your second patch.  Was that you think is best?

Yes.

Kind regards,
-- 
Arne Jørgensen <http://arnested.dk/>

Re: smime-cert-by-ldap-1 problem

[David S. Goldberg]

Works fine for me now on XEmacs.  Thanks!
-- 
Dave Goldberg
david.goldberg6@verizon.net