magic numbers in mm-view.el

magic numbers in mm-view.el

[David S. Goldberg]

If I use the #secure tag to send an encrypted s/mime email to others
and include a copy for myself, I am unable to read my copy in Gnus.  I
am able to decrypt and verify it (assuming it's signed as well) by
manually running Openssl on the message file and am also able to do so
in Thunderbird.  Oddly enough, if I build a multipart structure for
the message using #multipart instead of #secure I don't have this
problem.  But because the structure of the message is apparently not a
problem for OpenSSL or Thunderbird, I believe that the problem is that
the mm-pkcs7-enveloped-magic variable is not sufficient to cover all
cases.

Running (pp mm-pkcs7-enveloped-magic) gives

"0\\(€\\|.\\|‚..\\|ƒ...\\)\x06	\\*†H†÷\r\x01\a\x03"

So it's already a regular expression.  Here's what I see:

"0‚\nÍ\x06	*†H†÷\r\x01\a\x03"

The problem is that the '\n' after the \202 does not match the . in
the regular expression.  It seems it shouldn't be too hard to fix.
Just put [.\n] instead of just . in those locations.  However, in
mm-view.el we have

(defvar mm-pkcs7-enveloped-magic
  (mm-string-as-unibyte
   (mapconcat 'char-to-string
	      (list ?\x30 ?\x5c ?\x28 ?\x80 ?\x5c ?\x7c ?\x81 ?\x2e ?\x5c
		    ?\x7c ?\x82 ?\x2e ?\x2e ?\x5c ?\x7c ?\x83 ?\x2e ?\x2e
		    ?\x2e ?\x5c ?\x29 ?\x06 ?\x09 ?\x5c ?\x2a ?\x86 ?\x48
		    ?\x86 ?\xf7 ?\x0d ?\x01 ?\x07 ?\x03) "")))

And I am unsure how to work the fix into that construct.  I'd be glad
to do it but need a bit of guidance on how to translate the characters.

On a somewhat related note, at one point I seem to recall someone
posting a patch to support the use of the application/x-pkcs7-mime
type for signed messages (as opposed to encrypted and signed).  I'm
starting to see some of those show up more at work now that we're
migrating to Exchange.  Did that not work?

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: magic numbers in mm-view.el

[Simon Josefsson]

david.goldberg6@verizon.net (David S. Goldberg) writes:

> If I use the #secure tag to send an encrypted s/mime email to others
> and include a copy for myself, I am unable to read my copy in Gnus.  I
> am able to decrypt and verify it (assuming it's signed as well) by
> manually running Openssl on the message file and am also able to do so
> in Thunderbird.  Oddly enough, if I build a multipart structure for
> the message using #multipart instead of #secure I don't have this
> problem.  But because the structure of the message is apparently not a
> problem for OpenSSL or Thunderbird, I believe that the problem is that
> the mm-pkcs7-enveloped-magic variable is not sufficient to cover all
> cases.
>
> Running (pp mm-pkcs7-enveloped-magic) gives
>
> "0\\(€\\|.\\|‚..\\|ƒ...\\)\x06	\\*†H†÷\r\x01\a\x03"
>
> So it's already a regular expression.  Here's what I see:
>
> "0‚\nÍ\x06	*†H†÷\r\x01\a\x03"
>
> The problem is that the '\n' after the \202 does not match the . in
> the regular expression.  It seems it shouldn't be too hard to fix.
> Just put [.\n] instead of just . in those locations.  However, in
> mm-view.el we have
>
> (defvar mm-pkcs7-enveloped-magic
>   (mm-string-as-unibyte
>    (mapconcat 'char-to-string
> 	      (list ?\x30 ?\x5c ?\x28 ?\x80 ?\x5c ?\x7c ?\x81 ?\x2e ?\x5c
> 		    ?\x7c ?\x82 ?\x2e ?\x2e ?\x5c ?\x7c ?\x83 ?\x2e ?\x2e
> 		    ?\x2e ?\x5c ?\x29 ?\x06 ?\x09 ?\x5c ?\x2a ?\x86 ?\x48
> 		    ?\x86 ?\xf7 ?\x0d ?\x01 ?\x07 ?\x03) "")))
>
> And I am unsure how to work the fix into that construct.  I'd be glad
> to do it but need a bit of guidance on how to translate the characters.

Either you could simply replace those '.' with \(.\|\n\), but I think
maintainability suggest getting rid of the entire char-list and
replace it with something simpler.  Perhaps you can use `format' and
have the argument contain the printable characters, and regexp
keywords, and use %c or something for the escaped characters.

Re: magic numbers in mm-view.el

[Arne Jørgensen]

david.goldberg6@verizon.net (David S. Goldberg) writes:

> On a somewhat related note, at one point I seem to recall someone
> posting a patch to support the use of the application/x-pkcs7-mime
> type for signed messages (as opposed to encrypted and signed).  I'm
> starting to see some of those show up more at work now that we're
> migrating to Exchange.  Did that not work?

It was a patch from Milan Zamazal <pdm@brailcom.org> on gnu.emacs.gnus
back in March:

<http://groups-beta.google.com/group/gnu.emacs.gnus/browse_thread/thread/9dedf9f77031083/6cf5bbcc3d647610?q=87fyybk7lz.fsf@blackbird.zamazal.org&rnum=1#6cf5bbcc3d647610>

I mailed him a week or two ago and ask if he had signed papers or
would be willing to sign so we could apply the patch, but I haven't
heard from him yet.

I don't know if the patch works (I have not received that kind of
mails) but I have used the patch since back then and it has not broken
anything.

Kind regards,
-- 
Arne Jørgensen <http://arnested.dk/>

Re: magic numbers in mm-view.el

[David S. Goldberg]

>>>>> On Fri, 10 Jun 2005 22:31:19 +0200, Simon Josefsson <jas@extundo.com> said:

> Either you could simply replace those '.' with \(.\|\n\), but I think
> maintainability suggest getting rid of the entire char-list and
> replace it with something simpler.  Perhaps you can use `format' and
> have the argument contain the printable characters, and regexp
> keywords, and use %c or something for the escaped characters.

I'm not sure this is what you have in mind, but I now have the
follwing in my .gnus and am able to read all the smime encrypted mails
I've got:

(setq mm-pkcs7-enveloped-magic
      (format 
       "0\\(%c\\|%c\\(.\\|%c\\)\\|%c\\(.\\|%c\\)\\(.\\|%c\\)\\|%c\\(.\\|%c\\)\\(.\\|%c\\)\\(.\\|%c\\)\\)%c%c\\*%cH%c%c%c%c%c%c"
       ?\x80 ?\x81 ?\n ?\x82 ?\n ?\n ?\x83 ?\n ?\n ?\n ?\x06 ?\x09
       ?\x86 ?\x86 ?\xf7 ?\x0d ?\x01 ?\x07 ?\x03))

I don't know enough about coding systems to know how to properly apply
the current call to mm-string-as-unibyte.  According to my (now using
MULE) XEmacs it does nothing but that may not be a universal thing.

Thanks,
-- 
Dave Goldberg
david.goldberg6@verizon.net

Re: magic numbers in mm-view.el

[Roy Hashimoto]

David S. Goldberg <david.goldberg6 <at> verizon.net> writes:
> If I use the #secure tag to send an encrypted s/mime email to others
> and include a copy for myself, I am unable to read my copy in Gnus.  I
> am able to decrypt and verify it (assuming it's signed as well) by
> manually running Openssl on the message file and am also able to do so
> in Thunderbird.  Oddly enough, if I build a multipart structure for
> the message using #multipart instead of #secure I don't have this
> problem.  But because the structure of the message is apparently not a
> problem for OpenSSL or Thunderbird, I believe that the problem is that
> the mm-pkcs7-enveloped-magic variable is not sufficient to cover all
> cases.

This is a followup to an ancient thread but the bug is still present in the gnus
shipped with GNU Emacs 24.3.  To recap, sometimes valid S/MIME encrypted
messages will not be recognized with the error "Unknown or unimplemented
PKCS#7 type".  The cause was identified in the original message, which was
that the regular expression recognizing the binary PKCS7 envelope used a '.' 
to match any byte but that does not include 0x0a (\n).

Here's a patch that fixes that issue for both signed and encrypted messages
and makes the regexp a little easier to read:

Correct PKCS7 regexps to match messages with 0x0a bytes.

Signed-off-by: Roy Hashimoto <roy.hashimoto@gmail.com>
---

diff --git a/lisp/gnus/mm-view.el b/lisp/gnus/mm-view.el
index ac6170a..b1cba27 100644
--- a/lisp/gnus/mm-view.el
+++ b/lisp/gnus/mm-view.el
@@ -660,14 +660,26 @@ If MODE is not set, try to find mode automatically."
 ;;      id-signedData OBJECT IDENTIFIER ::= { iso(1) member-body(2)
 ;;          us(840) rsadsi(113549) pkcs(1) pkcs7(7) 2 }
 (defvar mm-pkcs7-signed-magic
-  "\x30\x5c\x28\x80\x5c\x7c\x81\x2e\x5c\x7c\x82\x2e\x2e\x5c\x7c\x83\x2e\x2e\
-\x2e\x5c\x29\x06\x09\x5c\x2a\x86\x48\x86\xf7\x0d\x01\x07\x02")
+  (concat
+    "0"
+    "\\(\\(\x80\\)"
+    "\\|\\(\x81\\(.\\|\n\\)\\{1\\}\\)"
+    "\\|\\(\x82\\(.\\|\n\\)\\{2\\}\\)"
+    "\\|\\(\x83\\(.\\|\n\\)\\{3\\}\\)"
+    "\\)"
+    "\x06\x09\\*\x86H\x86\xf7\x0d\x01\x07\x02"))
 
 ;;      id-envelopedData OBJECT IDENTIFIER ::= { iso(1) member-body(2)
 ;;          us(840) rsadsi(113549) pkcs(1) pkcs7(7) 3 }
 (defvar mm-pkcs7-enveloped-magic
-  "\x30\x5c\x28\x80\x5c\x7c\x81\x2e\x5c\x7c\x82\x2e\x2e\x5c\x7c\x83\x2e\x2e\
-\x2e\x5c\x29\x06\x09\x5c\x2a\x86\x48\x86\xf7\x0d\x01\x07\x03")
+  (concat
+    "0"
+    "\\(\\(\x80\\)"
+    "\\|\\(\x81\\(.\\|\n\\)\\{1\\}\\)"
+    "\\|\\(\x82\\(.\\|\n\\)\\{2\\}\\)"
+    "\\|\\(\x83\\(.\\|\n\\)\\{3\\}\\)"
+    "\\)"
+    "\x06\x09\\*\x86H\x86\xf7\x0d\x01\x07\x03"))
 
 (defun mm-view-pkcs7-get-type (handle)
   (mm-with-unibyte-buffer
--

Roy

Re: magic numbers in mm-view.el

[Daiki Ueno]

Roy Hashimoto <roy.hashimoto@gmail.com> writes:

> This is a followup to an ancient thread but the bug is still present
> in the gnus shipped with GNU Emacs 24.3.  To recap, sometimes valid
> S/MIME encrypted messages will not be recognized with the error
> "Unknown or unimplemented PKCS#7 type".  The cause was identified in
> the original message, which was that the regular expression
> recognizing the binary PKCS7 envelope used a '.'  to match any byte
> but that does not include 0x0a (\n).
>
> Here's a patch that fixes that issue for both signed and encrypted messages
> and makes the regexp a little easier to read:

Thanks; applied to Gnus git master.

Regards,
-- 
Daiki Ueno