default-enable-multibyte-characters in pgg-gpg

default-enable-multibyte-characters in pgg-gpg

[Daiki Ueno]

[-- Attachment #1: Type: text/plain, Size: 384 bytes --]

Hello,

Setting default-enable-multibyte-characters in pgg-gpg-start-process
makes GnuPG messages unreadable under multibyte locales.

I found the article which introduced this change:
http://article.gmane.org/gmane.emacs.gnus.general/50457
I think this case is rare, and doubt that this really fixed the problem.
Could anyone who can use a German keyboard check the attached patch?


[-- Attachment #2: pgg-gpg.el.diff --]
[-- Type: application/octet-stream, Size: 548 bytes --]

Index: lisp/pgg-gpg.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/pgg-gpg.el,v
retrieving revision 7.16
diff -u -r7.16 pgg-gpg.el
--- lisp/pgg-gpg.el	27 Mar 2006 09:34:18 -0000	7.16
+++ lisp/pgg-gpg.el	29 Mar 2006 05:51:07 -0000
@@ -82,7 +82,6 @@
 	 (coding-system-for-write 'binary)
 	 (process-connection-type nil)
 	 (orig-mode (default-file-modes))
-	 default-enable-multibyte-characters
 	 (buffer (generate-new-buffer " *pgg-gpg*"))
 	 process)
     (with-current-buffer buffer

[-- Attachment #3: Type: text/plain, Size: 25 bytes --]


Regards,
-- 
Daiki Ueno

Re: default-enable-multibyte-characters in pgg-gpg

[Reiner Steib]

On Wed, Mar 29 2006, Daiki Ueno wrote:

> Setting default-enable-multibyte-characters in pgg-gpg-start-process
> makes GnuPG messages unreadable under multibyte locales.
>
> I found the article which introduced this change:
> http://article.gmane.org/gmane.emacs.gnus.general/50457
> I think this case is rare, 

I don't think it is rare because people will simply use all characters
found on their keyboard.

> and doubt that this really fixed the problem.

At least Frank has confirmed that it does fix it for him in
<http://article.gmane.org/gmane.emacs.gnus.general/50877>.  Could you
suggest a better fix?  Maybe we could bind enable-multibyte-characters
only in some of the callers of `pgg-gpg-start-process', where
appropriate?

> Could anyone who can use a German keyboard check the attached patch?

As your patch just reverts this patch, I'd guess that it will
reintroduce the problem the other patch tried to solve.

Frank, could you try?  (Remove `default-enable-multibyte-characters'
from the function `pgg-gpg-start-process' in `pgg-gpg.el' and test
with a pass-prase containing non-ASCII characters.)

Bye, Reiner.

-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: default-enable-multibyte-characters in pgg-gpg

[Daiki Ueno]

Hello Reiner,

>>>>> In <v93bh1a3z1.fsf@marauder.physik.uni-ulm.de> 
>>>>>	Reiner Steib <reinersteib+gmane@imap.cc> wrote:
> > Setting default-enable-multibyte-characters in pgg-gpg-start-process
> > makes GnuPG messages unreadable under multibyte locales.
> >
> > I found the article which introduced this change:
> > http://article.gmane.org/gmane.emacs.gnus.general/50457
> > I think this case is rare, and doubt that this really fixed the problem.

> At least Frank has confirmed that it does fix it for him in
> <http://article.gmane.org/gmane.emacs.gnus.general/50877>.  Could you
> suggest a better fix?  Maybe we could bind enable-multibyte-characters
> only in some of the callers of `pgg-gpg-start-process', where
> appropriate?

> > Could anyone who can use a German keyboard check the attached patch?

> As your patch just reverts this patch, I'd guess that it will
> reintroduce the problem the other patch tried to solve.

Ah, I understand why this was necessary at that time.

When the patch was introduced, pgg-gpg used synchronous code and sent
a passphrase together with data on a temp buffer:

(defun pgg-gpg-process-region (start end passphrase program args)
...
          (let ((coding-system-for-write 'binary)
                (input (buffer-substring-no-properties start end))
                (default-enable-multibyte-characters nil))
            (with-temp-buffer
              (when passphrase
                (insert passphrase "\n"))
              (insert input)
              (setq exit-status
                    (apply #'call-process-region (point-min) (point-max) program
                           nil errors-buffer nil args))))

So I think setting default-enable-multibyte-characters is no longer
necessary because we switched to use asynchronous code and passphrases
are sent separately.

Regards,
-- 
Daiki Ueno

Re: default-enable-multibyte-characters in pgg-gpg

[Reiner Steib]

On Wed, Mar 29 2006, Daiki Ueno wrote:

>>>>>>	Reiner Steib <reinersteib+gmane@imap.cc> wrote:
>> As your patch just reverts this patch, I'd guess that it will
>> reintroduce the problem the other patch tried to solve.
>
> Ah, I understand why this was necessary at that time.
>
> When the patch was introduced, pgg-gpg used synchronous code and sent
> a passphrase together with data on a temp buffer:  [...]

Yes, revision 6.17 (in Gnus CVS) had this code.

> So I think setting default-enable-multibyte-characters is no longer
> necessary because we switched to use asynchronous code and passphrases
> are sent separately.

Thank you for this explanation.  I think we should clarify this in
the ChangeLog as follows:

--8<---------------cut here---------------start------------->8---
2006-03-29  Daiki Ueno  <ueno@unixuser.org>

	* pgg-gpg.el (pgg-gpg-start-process): Don't bind
	default-enable-multibyte-characters.  This reverts the change from
	revision 6.17 which is no longer necessary because the passphrase
	is sent separately now.  GnuPG messages are unreadable under
	multibyte locales with default-enable-multibyte-characters set to
	nil.
--8<---------------cut here---------------end--------------->8---

For testing, I changed my pass-phrase to start with `ä' (#x8e4,
U+00E4, LATIN SMALL LETTER A DIAERESIS) which I can enter using
MultiKey-a [1].  When sending a signed message I get (both, with and
without default-enable-multibyte-characters):

--8<---------------cut here---------------start------------->8---
Debugger entered--Lisp error: (error "Attempt to change byte length of a string")
  fillarray("\x8e4[...]" 0)
  pgg-gpg-status-BAD_PASSPHRASE(#<process pgg-gpg<2>> "BAD_PASSPHRASE E222255A3C6B854E")
  pgg-gpg-process-filter(#<process pgg-gpg<2>> "[GNUPG:] BAD_PASSPHRASE E222255A3C6B854E\n")
  accept-process-output(#<process pgg-gpg<2>> 1)
  pgg-gpg-wait-for-status(#<process pgg-gpg<2>> ("GOOD_PASSPHRASE"))
  pgg-gpg-sign-region(1 256 nil nil)
  funcall(pgg-gpg-sign-region 1 256 nil nil)
  apply(funcall pgg-gpg-sign-region (1 256 nil nil))
  pgg-invoke("sign-region" gpg 1 256 nil nil)
  pgg-sign-region(1 256)
  mml2015-pgg-sign((part (sign . "pgpmime") (tag-location . 1492) (contents . "[...]")))
  mml2015-sign((part (sign . "pgpmime") (tag-location . 1492) (contents . "[...]")))
  mml-generate-mime()
  message-encode-message-body()
  message-send-mail(nil)
  message-send-via-mail(nil)
  message-send(nil)
  message-send-and-exit(nil)
  call-interactively(message-send-and-exit)
--8<---------------cut here---------------end--------------->8---

Bye, Reiner.

[1] xmodmap -e 'keysym  a = a A adiaeresis Adiaeresis'
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: default-enable-multibyte-characters in pgg-gpg

[Simon Josefsson]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> Thank you for this explanation.  I think we should clarify this in
> the ChangeLog as follows:

Done.

> For testing, I changed my pass-phrase to start with `ä' (#x8e4,
> U+00E4, LATIN SMALL LETTER A DIAERESIS) which I can enter using
> MultiKey-a [1].  When sending a signed message I get (both, with and
> without default-enable-multibyte-characters):
>
> Debugger entered--Lisp error: (error "Attempt to change byte length of a string")
>   fillarray("\x8e4[...]" 0)
>   pgg-gpg-status-BAD_PASSPHRASE(#<process pgg-gpg<2>> "BAD_PASSPHRASE E222255A3C6B854E")

After fixing that minor issue, we would probably also have to make it
possible to configure which character set is used for the password
encoding.  Perhaps your terminal was using ISO-8859-1 and emacs used
UTF-8, then gpg likely won't accept the password.

Alternatively, gpg should convert the password received from the
terminal-encoded charset to the charset used when setting the
password, or always convert to UTF-8.

PGG: byte length problem in pass-phrase (was: default-enable-multibyte-characters in pgg-gpg)

[Reiner Steib]

On Wed, Mar 29 2006, Simon Josefsson wrote:

> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>> For testing, I changed my pass-phrase to start with `ä' (#x8e4,
>> U+00E4, LATIN SMALL LETTER A DIAERESIS) [...]
>> Debugger entered--Lisp error: (error "Attempt to change byte length
>> of a string")
>>   fillarray("\x8e4[...]" 0)
>>   pgg-gpg-status-BAD_PASSPHRASE(#<process pgg-gpg<2>>
>>   "BAD_PASSPHRASE E222255A3C6B854E")
>
> After fixing that minor issue, we would probably also have to make it
> possible to configure which character set is used for the password
> encoding.  Perhaps your terminal was using ISO-8859-1 and emacs used
> UTF-8, then gpg likely won't accept the password.
>
> Alternatively, gpg should convert the password received from the
> terminal-encoded charset to the charset used when setting the
> password, or always convert to UTF-8.

I didn't have any UTF-8 settings.  (The following setting are from a
different machine, but I think I have the same locale and Emacs
settings on both.)

--8<---------------cut here---------------start------------->8---
From M-x report-emacs-bug RET:

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: POSIX
  value of $LC_CTYPE: de_DE@euro
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: en_US
  locale-coding-system: iso-8859-15
  default-enable-multibyte-characters: t

Minor modes in effect:
[...]
  unify-8859-on-decoding-mode: t
  unify-8859-on-encoding-mode: t
  utf-translate-cjk-mode: t

M-x describe-current-coding-system RET:

Coding system for saving this buffer:
  Not set locally, use the default.
Default coding system (for new files):
  1 -- iso-latin-1 (alias: iso-8859-1 latin-1)

Coding system for keyboard input:
  nil
Coding system for terminal output:
  0 -- iso-8859-15 (alias of iso-latin-9)

Defaults for subprocess I/O:
  decoding: 1 -- iso-latin-1 (alias: iso-8859-1 latin-1)
  encoding: 1 -- iso-latin-1 (alias: iso-8859-1 latin-1)
--8<---------------cut here---------------end--------------->8---

Do you see anything odd there?

Bye, Reiner.
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo---  |  PGP key available  |  http://rsteib.home.pages.de/

Re: PGG: byte length problem in pass-phrase

[Daiki Ueno]

[-- Attachment #1: Type: text/plain, Size: 849 bytes --]

>>>>> In <v9bqvp58eq.fsf_-_@marauder.physik.uni-ulm.de> 
>>>>>	Reiner Steib <reinersteib+gmane@imap.cc> wrote:
> On Wed, Mar 29 2006, Simon Josefsson wrote:

> > Reiner Steib <reinersteib+gmane@imap.cc> writes:
> >> For testing, I changed my pass-phrase to start with `ä' (#x8e4,
> >> U+00E4, LATIN SMALL LETTER A DIAERESIS) [...]
> >> Debugger entered--Lisp error: (error "Attempt to change byte length
> >> of a string")
> >>   fillarray("\x8e4[...]" 0)
> >>   pgg-gpg-status-BAD_PASSPHRASE(#<process pgg-gpg<2>>
> >>   "BAD_PASSPHRASE E222255A3C6B854E")

I wonder why this does not happen in the read-passwd function since it
clears string in similar way.

Anyway, the primary issue (i.e. your passphrase string is not supplied
to gpg as it is) might be fixed by the attached patch.  However I think
this is not a right fix.


[-- Attachment #2: pgg-gpg.el.diff --]
[-- Type: application/octet-stream, Size: 942 bytes --]

Index: lisp/pgg-gpg.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/pgg-gpg.el,v
retrieving revision 7.17
diff -w -u -r7.17 pgg-gpg.el
--- lisp/pgg-gpg.el	29 Mar 2006 07:51:17 -0000	7.17
+++ lisp/pgg-gpg.el	30 Mar 2006 00:30:30 -0000
@@ -210,7 +210,7 @@
 
 (defun pgg-gpg-status-GET_HIDDEN (process line)
   (let ((entry (assoc pgg-gpg-key-id pgg-gpg-user-id-alist)))
-    (if (setq pgg-gpg-passphrase
+    (when (setq pgg-gpg-passphrase
 	      (if (eq pgg-gpg-key-id 'SYM)
 		  (pgg-read-passphrase
 		   "GnuPG passphrase for symmetric encryption: ")
@@ -222,6 +222,8 @@
 		 (if (eq pgg-gpg-key-id 'PIN)
 		     "PIN"
 		   pgg-gpg-key-id))))
+      (if (fboundp 'string-as-unibyte)
+	  (setq pgg-gpg-passphrase (string-as-unibyte pgg-gpg-passphrase)))
 	(process-send-string process (concat pgg-gpg-passphrase "\n")))))
 
 (defun pgg-gpg-status-GOOD_PASSPHRASE (process line)

[-- Attachment #3: Type: text/plain, Size: 25 bytes --]


Regards,
-- 
Daiki Ueno

Re: PGG: byte length problem in pass-phrase

[Simon Josefsson]

Reiner Steib <reinersteib+gmane@imap.cc> writes:

> On Wed, Mar 29 2006, Simon Josefsson wrote:
>
>> Reiner Steib <reinersteib+gmane@imap.cc> writes:
>>> For testing, I changed my pass-phrase to start with `ä' (#x8e4,
>>> U+00E4, LATIN SMALL LETTER A DIAERESIS) [...]
>>> Debugger entered--Lisp error: (error "Attempt to change byte length
>>> of a string")
>>>   fillarray("\x8e4[...]" 0)
>>>   pgg-gpg-status-BAD_PASSPHRASE(#<process pgg-gpg<2>>
>>>   "BAD_PASSPHRASE E222255A3C6B854E")
>>
>> After fixing that minor issue, we would probably also have to make it
>> possible to configure which character set is used for the password
>> encoding.  Perhaps your terminal was using ISO-8859-1 and emacs used
>> UTF-8, then gpg likely won't accept the password.
>>
>> Alternatively, gpg should convert the password received from the
>> terminal-encoded charset to the charset used when setting the
>> password, or always convert to UTF-8.
>
> I didn't have any UTF-8 settings.  (The following setting are from a
> different machine, but I think I have the same locale and Emacs
> settings on both.)
>
> From M-x report-emacs-bug RET:
>
> Important settings:
>   value of $LC_ALL: nil
>   value of $LC_COLLATE: POSIX
>   value of $LC_CTYPE: de_DE@euro
>   value of $LC_MESSAGES: nil
>   value of $LC_MONETARY: nil
>   value of $LC_NUMERIC: nil
>   value of $LC_TIME: nil
>   value of $LANG: en_US
>   locale-coding-system: iso-8859-15
>   default-enable-multibyte-characters: t
>
> Minor modes in effect:
> [...]
>   unify-8859-on-decoding-mode: t
>   unify-8859-on-encoding-mode: t
>   utf-translate-cjk-mode: t
>
> M-x describe-current-coding-system RET:
>
> Coding system for saving this buffer:
>   Not set locally, use the default.
> Default coding system (for new files):
>   1 -- iso-latin-1 (alias: iso-8859-1 latin-1)
>
> Coding system for keyboard input:
>   nil
> Coding system for terminal output:
>   0 -- iso-8859-15 (alias of iso-latin-9)
>
> Defaults for subprocess I/O:
>   decoding: 1 -- iso-latin-1 (alias: iso-8859-1 latin-1)
>   encoding: 1 -- iso-latin-1 (alias: iso-8859-1 latin-1)
>
> Do you see anything odd there?

No, Emacs should use ISO-8859-1(5).  Maybe your terminal was UTF-8?

I'm assuming you typed the same password in emacs as in the terminal.