Passwords, stored in Gnome Keyring and KWallet

Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Hi,

the GNOME Keyring and KDE Wallet developers have agreed a common API,
called Secret Services, to access their password repositories. This API
is D-Bus based, see
<http://www.freedesktop.org/wiki/Specifications/secret-storage-spec>.

I have written a small package, secrets.el, which uses this API to read
and write passwords. It is tested with GNOME Keyring 2.29.90; a public
KWallet implementation is not available yet. I intend to use it in
Tramp; it will also be proposed to be added to GNU Emacs after the 23.2
release. It might also be applicable in XEmacs, once D-Bus support has
been added there.

However, it might be useful to apply this package somewhere else as
well. One idea is to integrate it into auth-sources.el, as configurable
alternative to the ~/authinfo.gpg password store.

What do people think?

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Sat, 20 Feb 2010 15:47:01 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> the GNOME Keyring and KDE Wallet developers have agreed a common API,
MA> called Secret Services, to access their password repositories. This API
MA> is D-Bus based, see
MA> <http://www.freedesktop.org/wiki/Specifications/secret-storage-spec>.

MA> I have written a small package, secrets.el, which uses this API to read
MA> and write passwords. It is tested with GNOME Keyring 2.29.90; a public
MA> KWallet implementation is not available yet. I intend to use it in
MA> Tramp; it will also be proposed to be added to GNU Emacs after the 23.2
MA> release. It might also be applicable in XEmacs, once D-Bus support has
MA> been added there.

MA> However, it might be useful to apply this package somewhere else as
MA> well. One idea is to integrate it into auth-sources.el, as configurable
MA> alternative to the ~/authinfo.gpg password store.

That's a great idea.  auth-sources right now only has strings to
represent a source (implying a netrc-style file), but it's easy to
augment it to support something like 'secret-services or a more
complicated structure, e.g. '(secret-services parameter1 parameter2)

The only things we have to agree is how to pick an entry
(auth-source-pick, but this is probably OK as is) and how to get the
data out (auth-source-user-or-password, anything like
netrc-machine-user-or-password will work).

Sadly I don't see secrets.el in the Tramp repo currently.  Is it
available?

Perhaps it's better to name it sec-serv.el or something similarly more
descriptive?  It doesn't matter that much to me.

Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 918 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:

Hi Ted,

> The only things we have to agree is how to pick an entry
> (auth-source-pick, but this is probably OK as is) and how to get the
> data out (auth-source-user-or-password, anything like
> netrc-machine-user-or-password will work).

See the Commentary section. You can define lookup attributes, which are
supposed to do exactly this. The examples I have used there match this
case :-)

> Sadly I don't see secrets.el in the Tramp repo currently.  Is it
> available?

I have not decided yet, whether to store it in Tramp's archive. If it is
used as backend for auth-sources, it might be better to keep it close to
this. I have added the current version to this message.

> Perhaps it's better to name it sec-serv.el or something similarly more
> descriptive?  It doesn't matter that much to me.

It doesn't matter too much to me as well.

> Ted

Best regards, Michael.


[-- Attachment #2: secrets.el --]
[-- Type: application/emacs-lisp, Size: 28467 bytes --]

[-- Attachment #3: Type: text/plain, Size: 143 bytes --]

_______________________________________________
Tramp-devel mailing list
Tramp-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/tramp-devel

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Tue, 23 Feb 2010 14:38:50 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

>> The only things we have to agree is how to pick an entry
>> (auth-source-pick, but this is probably OK as is) and how to get the
>> data out (auth-source-user-or-password, anything like
>> netrc-machine-user-or-password will work).

MA> See the Commentary section. You can define lookup attributes, which are
MA> supposed to do exactly this. The examples I have used there match this
MA> case :-)

MA> I have not decided yet, whether to store it in Tramp's archive. If it is
MA> used as backend for auth-sources, it might be better to keep it close to
MA> this. I have added the current version to this message.

possible TODO list, then:

- secrets-debug should be structured like auth-source-debug to prevent
  leaking sensitive information.  Right now I don't see anything
  sensitive though.

- make part of Emacs (so both Tramp and Gnus can use it; since it
  requires a recent Emacs anyway).  I think it makes sense to keep it
  out of {gnus,tramp}/contrib/ or {gnus,tramp}/lisp/ because it depends
  on specific Emacs internals.  I don't know about XEmacs support but of
  course have no problem with it.

- possibly rename to secrets-service.el if you feel the urge or someone
  makes you do it :)

- change auth-source.el to take '(secrets "COLL") as a auth-sources spec
  naming collection COLL.  Then it will simply call

  (secrets-get-attribute "COLL" "hostname:port" :login)
  (secrets-get-attribute "COLL" "hostname:port" :password)

  to get the login and password spec.  Does that seem reasonable?  I
  think the host name + port should be the key for simplicity (rather
  than searching and listing collection items), but you should decide.

  The calls to secrets.el will be autoloaded so unless you use the
  'secrets spec in auth-sources, you won't load the library.

- write docs in auth.info on setting this up

I can do the last two in that order, but need your opinion on the item
key.

Thanks!
Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> possible TODO list, then:
>
> - secrets-debug should be structured like auth-source-debug to prevent
>   leaking sensitive information.  Right now I don't see anything
>   sensitive though.

OK. In fact, I haven't done something useful wrt secrets-debug yet :-)

> - make part of Emacs (so both Tramp and Gnus can use it; since it
>   requires a recent Emacs anyway).  I think it makes sense to keep it
>   out of {gnus,tramp}/contrib/ or {gnus,tramp}/lisp/ because it depends
>   on specific Emacs internals.  I don't know about XEmacs support but of
>   course have no problem with it.

Agreed. I will offer this, after the 23.2 release.

> - possibly rename to secrets-service.el if you feel the urge or someone
>   makes you do it :)

Yep. I don't feel the urge, but I will follow pressures ...

> - change auth-source.el to take '(secrets "COLL") as a auth-sources spec
>   naming collection COLL.  Then it will simply call
>
>   (secrets-get-attribute "COLL" "hostname:port" :login)
>   (secrets-get-attribute "COLL" "hostname:port" :password)
>
>   to get the login and password spec.  Does that seem reasonable?  I
>   think the host name + port should be the key for simplicity (rather
>   than searching and listing collection items), but you should decide.
>
>   The calls to secrets.el will be autoloaded so unless you use the
>   'secrets spec in auth-sources, you won't load the library.

I would let it to you. Allow "COLL" to be nil; then the (temporary)
collection "session" will be used. If "COLL" does not exist, you could
offer to create it on-the-fly.

However, the password is NOT an attribute. You must retrieve it with

  (secrets-get-secret "COLL" "hostname:port")

And I don't know, whether "hostname:port" is sufficient as secret item
name. You might want to connect to a given host under different
identities, for example via ssh as "ted@hostname" or "root@hostname". I
assume, you use different passwords then.

If you want to encode it in the secret item name, you shall use
"user@hostname:port". But I still believe it is more simple to use
attributes, because it is more convenient to *search* for an entry via
attributes, in order to know, whether it exists it already, or not.

See my test cases at the end of secrets.el. The item "albinus@magdalene"
was creted in the login keyring by another application; it is in fact
"user@host".

> - write docs in auth.info on setting this up
>
> I can do the last two in that order, but need your opinion on the item
> key.

Yes, please do. We can discuss pros and cons, once it is running :-)

> Thanks!
> Ted

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Wed, 24 Feb 2010 09:28:00 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> possible TODO list, then:
>> 
>> - secrets-debug should be structured like auth-source-debug to prevent
>> leaking sensitive information.  Right now I don't see anything
>> sensitive though.

MA> OK. In fact, I haven't done something useful wrt secrets-debug yet :-)

(sorry for the delay)

OK, if I need it while working on the support I'll add it.

>> - make part of Emacs (so both Tramp and Gnus can use it; since it
>> requires a recent Emacs anyway).  I think it makes sense to keep it
>> out of {gnus,tramp}/contrib/ or {gnus,tramp}/lisp/ because it depends
>> on specific Emacs internals.  I don't know about XEmacs support but of
>> course have no problem with it.

MA> Agreed. I will offer this, after the 23.2 release.

>> - possibly rename to secrets-service.el if you feel the urge or someone
>> makes you do it :)

MA> Yep. I don't feel the urge, but I will follow pressures ...

Consider me at 0 Newtons, so let's not worry about it.

>> - change auth-source.el to take '(secrets "COLL") as a auth-sources spec
>> naming collection COLL.  Then it will simply call
>> 
>> (secrets-get-attribute "COLL" "hostname:port" :login)
>> (secrets-get-attribute "COLL" "hostname:port" :password)
>> 
>> to get the login and password spec.  Does that seem reasonable?  I
>> think the host name + port should be the key for simplicity (rather
>> than searching and listing collection items), but you should decide.
>> 
>> The calls to secrets.el will be autoloaded so unless you use the
>> 'secrets spec in auth-sources, you won't load the library.

MA> I would let it to you. Allow "COLL" to be nil; then the (temporary)
MA> collection "session" will be used. If "COLL" does not exist, you could
MA> offer to create it on-the-fly.

We should make it Just Work.  Is there a reason not to create it on the
fly?  Can the collection be guessed from the system configuration by
secrets.el, if there's only one active collection?

MA> However, the password is NOT an attribute. You must retrieve it with

MA>   (secrets-get-secret "COLL" "hostname:port")

MA> And I don't know, whether "hostname:port" is sufficient as secret item
MA> name. You might want to connect to a given host under different
MA> identities, for example via ssh as "ted@hostname" or "root@hostname". I
MA> assume, you use different passwords then.

MA> If you want to encode it in the secret item name, you shall use
MA> "user@hostname:port". But I still believe it is more simple to use
MA> attributes, because it is more convenient to *search* for an entry via
MA> attributes, in order to know, whether it exists it already, or not.

MA> See my test cases at the end of secrets.el. The item "albinus@magdalene"
MA> was creted in the login keyring by another application; it is in fact
MA> "user@host".

Hmm, auth-source depends on the "host+protocol" key for everything.  The
assumption is always that there's only one login per host+protocol.

Can you show specifically how I would call secrets.el, given host X and
port Y, and use the first username found (the password comes from the
discovered object as you mentioned)?  The docs for secrets-search-items
are incorrect so I want to be sure (though from your examples at the end
it looks reasonably easy).

We can make the preferred username part of the auth-sources spec like
this (also including the secrets tag, and completely untested):

(defcustom auth-sources '((:source "~/.authinfo.gpg" :host t :protocol t))
  "List of authentication sources.

Each entry is the authentication type with optional properties."
  :group 'auth-source
  :version "23.1" ;; No Gnus
  :type `(repeat :tag "Authentication Sources"
		 (list :tag "Source definition"
		       (const :format "" :value :source)
		       (choice :tag "Authentication backend choice"
		               (string :tag "Authentication Source (file)")
		               (list :tag "secrets.el (Secret Service API/KWallet/GNOME KeyRing)" 
                                     (const secrets)
                                     (choice :tag "Collection to use"
                                             (string :tag "Collection name")
                                             (const :tag "Any" nil)
                                             (const :tag "Temporary" temp))))
		       (const :format "" :value :host)
		       (choice :tag "Host (machine) choice"
			       (const :tag "Any" t)
			       (regexp :tag "Host (machine) regular expression (TODO)")
			       (const :tag "Fallback" nil))
		       (const :format "" :value :protocol)
		       (choice :tag "Protocol"
			       (const :tag "Any" t)
			       (const :tag "Fallback" nil)
			       ,@auth-source-protocols-customize)
		       (const :format "" :value :preferred-username)
		       (choice :tag "Personality or username"
			       (const :tag "Any" t)
			       (const :tag "Fallback" nil)
			       (string :tag "Specific user name")))))

Like I said this is untested but the point is to show the structure
under the authentication backend choices and the optional personality
parameter (which can be extended to mean more than just username,
depending on the backend).  Do you think this is OK?

Then I'll simply pass the preferences as an optional list to
auth-source-pick and take them into account when searching (so the glue
to each backend would notice them).  I would like to make it completely
optional, though, because a single username is by far the most common
use case, so the above defcustom, which makes it mandatory, will
probably not be the final version.

>> - write docs in auth.info on setting this up
>> 
>> I can do the last two in that order, but need your opinion on the item
>> key.

MA> Yes, please do. We can discuss pros and cons, once it is running :-)

I'll work on it after we figure out the above.  Thanks so much for
working on secrets.el.

Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> MA> I would let it to you. Allow "COLL" to be nil; then the (temporary)
> MA> collection "session" will be used. If "COLL" does not exist, you could
> MA> offer to create it on-the-fly.
>
> We should make it Just Work.  Is there a reason not to create it on the
> fly?

Not from the functional pov. But one should restrict the number of
collections; if there are too many, it is harder to share the secrets
between different applications.

> Can the collection be guessed from the system configuration by
> secrets.el, if there's only one active collection?

In gnome-keyring, there must be at least the collection "login".
Furthermore, the Secret Service API contains an alias mechanism for
collections. The special alias "default" is mentioned explicitely, so
one could also simply use this collection. I don't know, what is more
useful; here we need some feedback for our default settings.

> MA> And I don't know, whether "hostname:port" is sufficient as secret item
> MA> name. You might want to connect to a given host under different
> MA> identities, for example via ssh as "ted@hostname" or "root@hostname". I
> MA> assume, you use different passwords then.
>
> MA> If you want to encode it in the secret item name, you shall use
> MA> "user@hostname:port". But I still believe it is more simple to use
> MA> attributes, because it is more convenient to *search* for an entry via
> MA> attributes, in order to know, whether it exists it already, or not.
>
> MA> See my test cases at the end of secrets.el. The item "albinus@magdalene"
> MA> was creted in the login keyring by another application; it is in fact
> MA> "user@host".
>
> Hmm, auth-source depends on the "host+protocol" key for everything.  The
> assumption is always that there's only one login per host+protocol.

Personally, I use different accounts on the remote hosts. Sometimes I
connect as albinus@host, sometimes as root@host etc, always via ssh.

> Can you show specifically how I would call secrets.el, given host X and
> port Y, and use the first username found (the password comes from the
> discovered object as you mentioned)?  The docs for secrets-search-items
> are incorrect so I want to be sure (though from your examples at the end
> it looks reasonably easy).

There are always lists, and you could use the car of a list as first hit.

Assume, you have created 4 items on host "remote-host":

(secrets-create-item nil "item" "geheim" :method "sudo" :user "joe" :host "remote-host")
(secrets-create-item nil "another item" "whatever" :method "sudo" :user "suzanne" :host "remote-host")
(secrets-create-item nil "now this" "42" :method "ssh" :user "ted" :host "remote-host")
(secrets-create-item nil "now that" "27" :method "ftp" :user "ted" :host "remote-host")

You can get all items for "remote-host" by this:

(secrets-search-items nil :host "remote-host")
=> ("now this" "item" "now that" "another item")

If you want to know ted's items on "remote-host", you apply

(secrets-search-items nil :user "ted" :host "remote-host")
=> ("now this" "now that")

And if you want the user names of all items on "remote-host" registered
for method (aka port) "sudo", you do:

(let (result)
  (dolist (item
	   (secrets-search-items nil :method "sudo" :host "remote-host")
	   result)
    (add-to-list 'result (secrets-get-attribute nil item :user))))
=> ("joe" "suzanne")

> We can make the preferred username part of the auth-sources spec like
> this (also including the secrets tag, and completely untested):
>
> (defcustom auth-sources '((:source "~/.authinfo.gpg" :host t :protocol t))
>   "List of authentication sources.
>
> Each entry is the authentication type with optional properties."
>   :group 'auth-source
>   :version "23.1" ;; No Gnus

This must be 23.2. There were some bug fixes in dbus.el since 23.1,
needed for secrets.el.

>   :type `(repeat :tag "Authentication Sources"
> 		 (list :tag "Source definition"
> 		       (const :format "" :value :source)
> 		       (choice :tag "Authentication backend choice"
> 		               (string :tag "Authentication Source (file)")
> 		               (list :tag "secrets.el (Secret Service API/KWallet/GNOME KeyRing)" 
>                                      (const secrets)
>                                      (choice :tag "Collection to use"
>                                              (string :tag "Collection name")
>                                              (const :tag "Any" nil)
>                                              (const :tag "Temporary" temp))))

I understand the intention. However, the temporary (session) collection
in secrets.el is coded either as "session" or as nil. Maybe we can
reflect this.

> 		       (const :format "" :value :host)
> 		       (choice :tag "Host (machine) choice"
> 			       (const :tag "Any" t)
> 			       (regexp :tag "Host (machine) regular expression (TODO)")
> 			       (const :tag "Fallback" nil))
> 		       (const :format "" :value :protocol)
> 		       (choice :tag "Protocol"
> 			       (const :tag "Any" t)
> 			       (const :tag "Fallback" nil)
> 			       ,@auth-source-protocols-customize)
> 		       (const :format "" :value :preferred-username)
> 		       (choice :tag "Personality or username"
> 			       (const :tag "Any" t)
> 			       (const :tag "Fallback" nil)
> 			       (string :tag "Specific user name")))))
>
> Like I said this is untested but the point is to show the structure
> under the authentication backend choices and the optional personality
> parameter (which can be extended to mean more than just username,
> depending on the backend).  Do you think this is OK?

Yes, this looks OK.

> Then I'll simply pass the preferences as an optional list to
> auth-source-pick and take them into account when searching (so the glue
> to each backend would notice them).  I would like to make it completely
> optional, though, because a single username is by far the most common
> use case, so the above defcustom, which makes it mandatory, will
> probably not be the final version.

Let's try it.

> Ted

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Steinar Bang]

>>>>> Ted Zlatanov <tzz@lifelogs.com>:

> On Wed, 24 Feb 2010 09:28:00 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Yep. I don't feel the urge, but I will follow pressures ...

> Consider me at 0 Newtons, so let's not worry about it.

0 Pascal perhaps...? (0 N/m²)

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Thu, 11 Mar 2010 12:46:28 +0100 Steinar Bang <sb@dod.no> wrote: 

>>>>>> Ted Zlatanov <tzz@lifelogs.com>:
>> On Wed, 24 Feb 2010 09:28:00 +0100 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Yep. I don't feel the urge, but I will follow pressures ...

>> Consider me at 0 Newtons, so let's not worry about it.

SB> 0 Pascal perhaps...? (0 N/m²)

Definitely, and 0 Watts as well ;)

Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

I tried to work on this but secrets.el was not usable on a recent Ubuntu
system (secrets-list-collections, for example, fails).  Can you explain
how to set it up or point me to a resource?
http://code.confuego.org/secrets-xdg-specs/ specified the protocol only
and I couldn't find a tutorial anywhere.

I comitted changes to the customization of auth-sources which would let
the user specify, in addition to the current "file" option expressed as
a string filename:

- a ':secrets' specifier, matching a host and a protocol and (optionally)
  a user name

- within each ':secrets' specifier, one of:

  - a collection name as a string

  - the 'default symbol (which we'll convert to a collection name with
    (secrets-get-alias "default") IIUC)

  - nil to indicate we don't have a valid collection and this specifier
    should be skipped (this helps with misconfigured specifiers)

  - t to indicate any collection is OK, we'll pick the first one
    returned by secrets-list-collections that matches the host, port,
    and username

  - the string "session" to indicate the temporary session-level
    collection

- optional parameters at the end of the spec, including a user name.  I
  may move the host and port to these optional parameters as well but
  I'm not sure if that's a good idea.  Comments welcome.

So this is about 30% done...

Thanks
Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

[-- Attachment #1: Type: text/plain, Size: 1379 bytes --]

The major problem so far is that secrets-search-items on a nil (matching
any) collection doesn't tell me what collection matched.  So then I
can't say

(secrets-get-attributes nil (car (secrets-search-items nil :user "joe")))

to find the attributes of that item (although 

(secrets-get-secret nil "my item") 

does work, so this only fails when I need attributes).

When I do know the session, things work.  Attached is a patch to show
how auth-source-pick and auth-source-user-or-password would change, on
top of the recent check-ins to Gnus.  Basically auth-source-pick bakes a
list of results that includes the found collection in :search and the
found item in :item.  Then auth-source-user-or-password looks up the
:user attribute for the "login" attribute and the secret for the
"password" attribute.

The default session is looked up as the "default" alias and if that
fails, as the "default" session (with a message).  Otherwise most of the
code converts between the nil "match all" session parameter in
secrets.el and the t "match all" in auth-source.el and massages the
results.

I also redid the auth-sources customization a little bit.

If you can follow this, great.  Otherwise if you think it's too
complicated, it's time to redo auth-source.el to be more flexible about
taking extra parameters.  As the author it's hard for me to say if that
time is now :)

Ted


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: auth-secrets.patch --]
[-- Type: text/x-diff, Size: 7652 bytes --]

Index: auth-source.el
===================================================================
RCS file: /usr/local/cvsroot/gnus/lisp/auth-source.el,v
retrieving revision 7.20
diff -u -r7.20 auth-source.el
--- auth-source.el	23 Mar 2010 02:54:14 -0000	7.20
+++ auth-source.el	23 Mar 2010 18:52:52 -0000
@@ -124,7 +124,6 @@
                                              (const :tag "Default" 'default)
                                              (const :tag "Any" t)
                                              (const :tag "Temporary" "session")
-                                             (string :tag "Specific session name")
                                              (const :tag "Fallback" nil))))
 		       (const :format "" :value :host)
 		       (choice :tag "Host (machine) choice"
@@ -139,7 +138,7 @@
 		       (repeat :tag "Extra Parameters" :inline t
 			       (choice :tag "Extra parameter"
 				       (list :tag "Preferred username" :inline t
-					     (const :format "" :value :preferred-username)
+					     (const :format "" :value :user)
 					     (choice :tag "Personality or username"
 						     (const :tag "Any" t)
 						     (const :tag "Fallback" nil)
@@ -154,7 +153,7 @@
 ;; (customize-variable 'auth-source-protocols)
 ;; (setq auth-source-protocols nil)
 ;; (format "%S" auth-source-protocols)
-;; (auth-source-pick "a" 'imap)
+;; (auth-source-pick "a" 'imap '())
 ;; (auth-source-user-or-password "login" "imap.myhost.com" 'imap)
 ;; (auth-source-user-or-password "password" "imap.myhost.com" 'imap)
 ;; (auth-source-user-or-password-imap "login" "imap.myhost.com")
@@ -174,25 +173,74 @@
 		    'message)))
       (apply logger msg))))
 
-(defun auth-source-pick (host protocol &optional fallback)
-  "Parse `auth-sources' for HOST, and PROTOCOL matches.
+(defun auth-source-pick (host protocol extras &optional fallback)
+  "Parse `auth-sources' for HOST and PROTOCOL (+EXTRAS) matches.
+
+Only :user is checked currently in the plist EXTRAS.
 
 Returns fallback choices (where PROTOCOL or HOST are nil) with FALLBACK t."
   (interactive "sHost: \nsProtocol: \n") ;for testing
   (let (choices)
     (dolist (choice auth-sources)
-      (let ((h (plist-get choice :host))
-	    (p (plist-get choice :protocol)))
-	(when (and
-	       (or (equal t h)
-		   (and (stringp h) (string-match h host))
-		   (and fallback (equal h nil)))
-	       (or (equal t p)
-		   (and (symbolp p) (equal p protocol))
-		   (and fallback (equal p nil))))
-	  (push choice choices))))
+      (let* ((h (plist-get choice :host))
+	     (p (plist-get choice :protocol))
+	     (s (plist-get choice :source))
+	     (u (or (plist-get choice :user) t)) ; the user spec in this choice, defaults to t
+	     (user (plist-get extras :user))	 ; the passed user
+	     ;; this is only set for Secret Service API specs (through secrets.el)
+	     (coll (plist-get s :secrets)))
+	(cond
+	 (coll				; use secrets.el here
+	  (when (eq coll 'default)
+	    (setq coll (secrets-get-alias "default"))
+	    (unless coll 
+	      (auth-source-do-debug
+	       "Default alias: no '%s' alias.  Trying collection '%s'."
+	       "default" "default")
+	      (setq coll "default")))
+	  (let* ((coll-search (cond
+			       ((stringp coll) coll)
+			       ((eq coll t) nil) ; t = any collection
+			       
+			       ;; when the collection is nil:
+			       ;; in fallback mode, accept it as any
+			       ;; otherwise, hope to fail
+			       ((null coll) (if fallback
+						nil
+					      " *fallback-fail*"))))
+		 ;; this will have the other secrets-search-items parameters
+		 (other-search (append
+				(when (stringp h) `(:host ,h))
+				(when (stringp u) `(:user ,u))
+				(when (stringp p) `(:port ,p))))
+		 (results (apply 'secrets-search-items
+				 coll-search
+				 other-search)))
+	    (auth-source-do-debug
+	     "auth-source-pick: got items %s in collection search %s + %s"
+	     results coll-search other-search)
+	    (dolist (result results)
+	      (push `(:source secrets
+			      :item ,result
+			      :collection ,coll
+			      :search ,coll-search
+			      ,@other-search)
+		    choices))))
+	 (t				; this is any non-secrets spec
+	  (when (and
+		 (or (equal t h)
+		     (and (stringp h) (string-match h host))
+		     (and fallback (equal h nil)))
+		 (or (equal t p)
+		     (and (symbolp p) (equal p protocol))
+		     (and fallback (equal p nil)))
+		 (or (equal t u)	; note that u defaults to t
+		     (and (stringp u) (string-match u user))))
+	    (push choice choices))))))
     (if choices
 	choices
+      ;; when there were no matches, do a second pass but this time
+      ;; fallbacks (nil values) are acceptable
       (unless fallback
 	(auth-source-pick host protocol t)))))
 
@@ -205,44 +253,58 @@
   (interactive)
   (setq auth-source-cache (make-hash-table :test 'equal)))
 
-(defun auth-source-user-or-password (mode host protocol)
+(defun auth-source-user-or-password (mode host protocol &optional username)
   "Find MODE (string or list of strings) matching HOST and PROTOCOL.
 MODE can be \"login\" or \"password\" for example."
   (auth-source-do-debug
-   "auth-source-user-or-password: get %s for %s (%s)"
-   mode host protocol)
+   "auth-source-user-or-password: get %s for %s (%s) + user=%s"
+   mode host protocol username)
   (let* ((listy (listp mode))
 	 (mode (if listy mode (list mode)))
-	 (cname (format "%s %s:%s" mode host protocol))
+	 (extras (when username `(:user ,username)))
+	 (cname (format "%s %s:%s %s" mode host protocol extras))
 	 (found (gethash cname auth-source-cache)))
     (if found
 	(progn
 	  (auth-source-do-debug
-	   "auth-source-user-or-password: cached %s=%s for %s (%s)"
+	   "auth-source-user-or-password: cached %s=%s for %s (%s) + %s"
 	   mode
 	   ;; don't show the password
-	   (if (and (member "password" mode) auth-source-hide-passwords) "SECRET" found)
-	   host protocol)
-	  found)
-      (dolist (choice (auth-source-pick host protocol))
-	(setq found (netrc-machine-user-or-password
-		     mode
-		     (plist-get choice :source)
-		     (list host)
-		     (list (format "%s" protocol))
-		     (auth-source-protocol-defaults protocol)))
+	   (if (and (member "password" mode) auth-source-hide-passwords)
+	       "SECRET"
+	     found)
+	   host protocol extras)
+	  found)			; return the found data
+      (dolist (choice (auth-source-pick host protocol extras))
+	(setq found (cond
+		     ;; the secrets.el spec
+		     ((eq (plist-get choice :source) 'secrets)
+		      (let ((coll (plist-get choice :search))
+			    (item (plist-get choice :item)))
+			(mapcar (lambda (m)
+				  (if (equal "password" m)
+				      (secrets-get-secret coll item)
+				    (secrets-get-attribute coll item :user)))
+				mode)))
+		     (t		; anything else is netrc
+		      (netrc-machine-user-or-password
+		       mode
+		       (plist-get choice :source)
+		       (list host)
+		       (list (format "%s" protocol))
+		       (auth-source-protocol-defaults protocol)))))
 	(when found
 	  (auth-source-do-debug
-	   "auth-source-user-or-password: found %s=%s for %s (%s)"
+	   "auth-source-user-or-password: found %s=%s for %s (%s) + %s"
 	   mode
 	   ;; don't show the password
 	   (if (and (member "password" mode) auth-source-hide-passwords) "SECRET" found)
-	   host protocol)
+	   host protocol extras)
 	  (setq found (if listy found (car-safe found)))
 	  (when auth-source-do-cache
 	    (puthash cname found auth-source-cache)))
 	(return found)))))
-
+  
 (defun auth-source-protocol-defaults (protocol)
   "Return a list of default ports and names for PROTOCOL."
   (cdr-safe (assoc protocol auth-source-protocols)))

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

I forgot to mention how I got secrets.el to work.  It requires the
latest Emacs with D-Bus support, plus the latest gnome-keyring.  With
Ubuntu the gnome-keyring install was a pain so I upgraded my whole
system to 10.x from 9.x and it works there.  The passwords can be seen
and modified through secrets.el or through the `seahorse' application.

I did not try KWallet.

Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

I've checked in a new version of auth-source.el that supports secrets.el
and should be backwards compatible with the old auth-sources format.
The manual auth.texi was also updated.

Michael, I left a TODO in the manual for explaining secrets.el.

Thanks
Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

>>>>> Ted Zlatanov wrote:
> I've checked in a new version of auth-source.el that supports secrets.el
> and should be backwards compatible with the old auth-sources format.
> The manual auth.texi was also updated.

> Michael, I left a TODO in the manual for explaining secrets.el.

Now `auth-sources' defaults to `((:source "~/.authinfo.gpg"))',
which causes `auth-source-pick' to perform:

(plist-get "~/.authinfo.gpg" :secrets)

Emacs 22/23/24 returns nil for it, however it makes Emacs 21 and
XEmacs cause an error and prevents Gnus from starting.  This
workaround makes it work:


[-- Attachment #2: Type: text/x-patch, Size: 404 bytes --]

--- auth-source.el~	2010-03-28 21:49:05 +0000
+++ auth-source.el	2010-03-29 04:58:50 +0000
@@ -214,7 +214,7 @@
 	(dolist (choice auth-sources)
 	  (let* ((s (plist-get choice :source))
 		 ;; this is only set for Secret Service API specs (see secrets.el)
-		 (coll (plist-get s :secrets))
+		 (coll (and (consp s) (plist-get s :secrets)))
 		 (score 0))
 	    (cond
 	     (coll				; use secrets.el here

Re: Passwords, stored in Gnome Keyring and KWallet

[Norbert Koch]

* Katsumi Yamaoka <yamaoka@jpl.org>:

Hi!

> Now `auth-sources' defaults to `((:source "~/.authinfo.gpg"))',
> which causes `auth-source-pick' to perform:
>
> (plist-get "~/.authinfo.gpg" :secrets)

Apart from this, it - again - breaks the login to my imap server without
any authinfo files given :-(

Unable to open server nnimap+mmlab due to: Malformed property list:
~/.authinfo.gpg, "~/.authinfo.gpg"
Loading auth-source...done
Loading auth-source...
Parsing authinfo file `~/.authinfo'.
Waiting for response from imap.mmlab.de...done
imap: Connecting with STARTTLS...done



emacs-version
"21.5  (beta29) \"garbanzo\" 8bf9e67e06ec+ XEmacs Lucid"
gnus-version
"No Gnus v0.11"

norbert.
-- 

Re: Passwords, stored in Gnome Keyring and KWallet

[Katsumi Yamaoka]

>>>>> Norbert Koch wrote:
> * Katsumi Yamaoka <yamaoka@jpl.org>:

> Hi!

>> Now `auth-sources' defaults to `((:source "~/.authinfo.gpg"))',
>> which causes `auth-source-pick' to perform:
>>
>> (plist-get "~/.authinfo.gpg" :secrets)

> Apart from this, it - again - breaks the login to my imap server without
> any authinfo files given :-(

> Unable to open server nnimap+mmlab due to: Malformed property list:
> ~/.authinfo.gpg, "~/.authinfo.gpg"

That's just what I got in XEmacs.  Doesn't the workaround patch
I posted solve it?

Re: Passwords, stored in Gnome Keyring and KWallet

[Norbert Koch]

* Katsumi Yamaoka <yamaoka@jpl.org>:

Hi!

> That's just what I got in XEmacs.  Doesn't the workaround patch
> I posted solve it?

Of course it does :-) Sorry, I read your message wrong ...

Thanks,
norbert.
-- 

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Mon, 29 Mar 2010 14:00:22 +0900 Katsumi Yamaoka <yamaoka@jpl.org> wrote: 

KY> Now `auth-sources' defaults to `((:source "~/.authinfo.gpg"))',
KY> which causes `auth-source-pick' to perform:

KY> (plist-get "~/.authinfo.gpg" :secrets)

KY> Emacs 22/23/24 returns nil for it, however it makes Emacs 21 and
KY> XEmacs cause an error and prevents Gnus from starting.  This
KY> workaround makes it work:
...

Thanks for fixing this.  I checked in your patch.

Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Michael, I left a TODO in the manual for explaining secrets.el.

I'm sitting with this, it is not so easy.

The crucial point is how to populate the Gnome Keyring with valid
entries. For .authinfo files it is obvious, "edit the file".

One option would be to explain `secrets-create-item'. But that's not
convenient; a user who wants to access her imap account via gnus would
dislike it.

And it is not the intention of a password database like Gnome Keyring
(or KDE Wallet, or Firefox Password Manager, ...). Those databases let
their clients store passwords on the fly.

Consequently, auth-source.el shall offer such an interface. Something
like

(defun auth-source-create (host password &optional user protocol)
  "Create a new entry in the password database for accessing HOST.
PASSWORD is the credential needed for authentication.  If USER is
nil, PASSWORD will be applied for all users.  PROTOCOL can
restrict the authentication for access methods, like \"imap\"
or \"ssh\".
The password databases as defined in `auth-sources' will be
checked in order to find the best fit according to the arguments.
Note, that only password databases of the Secret Service API will be
taken into account, authinfo like flat files will not be used for
creation of new entries.")

A typical client library would have code like this:

(let ((password (auth-source-user-or-password "password" server port user)))
  (unless password
    (setq password (read-passwd "Password: "))
    (auth-source-create server password user port))
  password)

What do you think about?

> Thanks
> Ted

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Wed, 12 May 2010 14:07:54 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Michael, I left a TODO in the manual for explaining secrets.el.

MA> I'm sitting with this, it is not so easy.

btw, sorry I haven't updated auth-source.el, I've been swamped with work
(until next week).  I know there's at least one issue with the Secrets
API support there.

MA> Consequently, auth-source.el shall offer such an interface. Something
MA> like

MA> (defun auth-source-create (host password &optional user protocol)
MA>   "Create a new entry in the password database for accessing HOST.
MA> PASSWORD is the credential needed for authentication.  If USER is
MA> nil, PASSWORD will be applied for all users.  PROTOCOL can
MA> restrict the authentication for access methods, like \"imap\"
MA> or \"ssh\".
MA> The password databases as defined in `auth-sources' will be
MA> checked in order to find the best fit according to the arguments.
MA> Note, that only password databases of the Secret Service API will be
MA> taken into account, authinfo like flat files will not be used for
MA> creation of new entries.")

I agree 100% that this is useful.  Why not allow authinfo editing too?
I think auth-source-create should just be a wrapper for

;;; 'create-if-missing should be supplemented by a global
;;; auth-source-create-if-missing which could be 'never, 'always, or nil
(auth-source-user-or-password "password" server port user 'create-if-missing)

and it would (theoretically) work for any editable backend.  At least
for the two backends we've discussed (Secrets API and flat netrc files)
this would work.  That would give us a stateless way to ensure that a
password exists, prompting the user if necessary without interrupting
the code flow, and allowing the user to specify their preference as well
if they want to always or never create entries.

MA> A typical client library would have code like this:

MA> (let ((password (auth-source-user-or-password "password" server port user)))
MA>   (unless password
MA>     (setq password (read-passwd "Password: "))
MA>     (auth-source-create server password user port))
MA>   password)

All of this could live inside auth-source-user-or-password and get
triggered by 'create-if-missing.  Thus the auth-source API would remain
the same as now except for a new optional flag.  No new user-visible
functions need be introduced and clients only need to specify that flag
if they have a preference.

Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I agree 100% that this is useful.  Why not allow authinfo editing too?
> I think auth-source-create should just be a wrapper for
>
> ;;; 'create-if-missing should be supplemented by a global
> ;;; auth-source-create-if-missing which could be 'never, 'always, or nil
> (auth-source-user-or-password "password" server port user 'create-if-missing)

D'accord. Maybe we dont't need `auth-source-create-if-missing'; the last
arg `create-if' missing could be t or nil.

> and it would (theoretically) work for any editable backend.  At least
> for the two backends we've discussed (Secrets API and flat netrc files)
> this would work.  That would give us a stateless way to ensure that a
> password exists, prompting the user if necessary without interrupting
> the code flow, and allowing the user to specify their preference as well
> if they want to always or never create entries.

OK. We have a long weekend here in Germany (next 4 days); maybe I try an
implementation. I want to understand your code anyway :-)

> Ted

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

[-- Attachment #1: Type: text/plain, Size: 1810 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:

Hi Ted,

> ;;; 'create-if-missing should be supplemented by a global
> ;;; auth-source-create-if-missing which could be 'never, 'always, or nil
> (auth-source-user-or-password "password" server port user 'create-if-missing)
>
> and it would (theoretically) work for any editable backend.  At least
> for the two backends we've discussed (Secrets API and flat netrc files)
> this would work.  That would give us a stateless way to ensure that a
> password exists, prompting the user if necessary without interrupting
> the code flow, and allowing the user to specify their preference as well
> if they want to always or never create entries.

I've tried to implement it last days. What I've obeserved is that we
also need a `delete-existing' argument. The reason behind is, that
people might add wrong passwords, which will be detected by the client
only. The client must be able to overwrite this password. So I have
changed the function too

(defun auth-source-user-or-password (mode host protocol &optional username create-missing delete-existing)

Maybe `create-missing' and `delete-existing' could be put into one
argument; but this would be more complex then.

I found your implementation with two passes inconvenient for the two new
arguments. Therefore, I have rewritten `auth-source-user-or-password'
and `auth-source-pick'. The latter one just identifies applicable
entries from `auth-sources according' to the spec compiled from
`auth-source-user-or-password' arguments.

There are also the new functions `auth-get-source',
`auth-source-retrieve', `auth-source-create' and
`auth-source-delete'. These functions are introduced in order to make
`auth-source-user-or-password' readable.

The new code is appended; please comment.

> Ted

Best regards, Michael.


[-- Attachment #2: auth-source.el --]
[-- Type: application/emacs-lisp, Size: 17534 bytes --]

[-- Attachment #3: Type: text/plain, Size: 143 bytes --]

_______________________________________________
Tramp-devel mailing list
Tramp-devel@gnu.org
http://lists.gnu.org/mailman/listinfo/tramp-devel

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Michael Albinus <michael.albinus@gmx.de> writes:

Hi Ted,

> The new code is appended; please comment.

For your convenience, I've just added a new command to secrets.el, which
shows all collections, items, and attributes in a buffer. Try "M-x
secrets-show-secrets" on a fresh build of Emacs' trunk. Maybe you find
it useful.

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Sun, 16 May 2010 20:39:49 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

>> ;;; 'create-if-missing should be supplemented by a global
>> ;;; auth-source-create-if-missing which could be 'never, 'always, or nil
>> (auth-source-user-or-password "password" server port user 'create-if-missing)
>> 
>> and it would (theoretically) work for any editable backend.  At least
>> for the two backends we've discussed (Secrets API and flat netrc files)
>> this would work.  That would give us a stateless way to ensure that a
>> password exists, prompting the user if necessary without interrupting
>> the code flow, and allowing the user to specify their preference as well
>> if they want to always or never create entries.

MA> I've tried to implement it last days. What I've obeserved is that we
MA> also need a `delete-existing' argument. The reason behind is, that
MA> people might add wrong passwords, which will be detected by the client
MA> only. The client must be able to overwrite this password. So I have
MA> changed the function too

Wonderful, I didn't think of that.  auth-source.el started life as
basically a wrapper around a Customize statement and it shows.

MA> (defun auth-source-user-or-password (mode host protocol &optional username create-missing delete-existing)

MA> Maybe `create-missing' and `delete-existing' could be put into one
MA> argument; but this would be more complex then.

MA> I found your implementation with two passes inconvenient for the two new
MA> arguments.

Yes, it was very much a quick hack.

MA> Therefore, I have rewritten `auth-source-user-or-password' and
MA> `auth-source-pick'. The latter one just identifies applicable
MA> entries from `auth-sources according' to the spec compiled from
MA> `auth-source-user-or-password' arguments.

MA> There are also the new functions `auth-get-source',
MA> `auth-source-retrieve', `auth-source-create' and
MA> `auth-source-delete'. These functions are introduced in order to make
MA> `auth-source-user-or-password' readable.

MA> The new code is appended; please comment.

Your version is much more readable and extensible than mine (I should
have rewritten like you did long ago).  I'm 100% in favor of using it.
Do you want to use the Gnus repo as a testbed or would you rather push
straight into Emacs (from where it will be synchronized back into Gnus)?

Thanks
Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Wed, 12 May 2010 14:07:54 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Michael, I left a TODO in the manual for explaining secrets.el.

MA> I'm sitting with this, it is not so easy.

MA> The crucial point is how to populate the Gnome Keyring with valid
MA> entries. For .authinfo files it is obvious, "edit the file".

Maybe assistant.el could be useful?  This is a good simple task: set up
auth-source entries for various types of accounts.  The manual doesn't
have to explain as much then, and the .ast file can describe the
secrets.el steps to create and populate a Secrets API backend.

If not, it becomes a pretty complicated explanation as you saw.  What
direction would you prefer?

MA> Consequently, auth-source.el shall offer such an interface. Something
MA> like (auth-source-create)
...
MA> What do you think about?

I agree 100% with your changes to auth-source.el you posted previously
to make this interface a possibility.  Deleting and creating
authinfo/netrc entries, which is missing in your version, should not be
too hard: we just need to remember the original line for every parsed
entry in netrc.el (to avoid cases where the parser might create
identical parses for two different lines).

Thank you
Ted

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> MA> The new code is appended; please comment.
>
> Your version is much more readable and extensible than mine (I should
> have rewritten like you did long ago).  I'm 100% in favor of using it.
> Do you want to use the Gnus repo as a testbed or would you rather push
> straight into Emacs (from where it will be synchronized back into Gnus)?

I've committed it to the Emacs repository. Until now, it is the most
convenient way to work for me, given that such changes are synchronized
to the Gnus repository.

If requested, I could also commit directly to the Gnus repository. I
would like to avoid double commits, 'tho.

> Thanks
> Ted

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> MA> The crucial point is how to populate the Gnome Keyring with valid
> MA> entries. For .authinfo files it is obvious, "edit the file".
>
> Maybe assistant.el could be useful?  This is a good simple task: set up
> auth-source entries for various types of accounts.  The manual doesn't
> have to explain as much then, and the .ast file can describe the
> secrets.el steps to create and populate a Secrets API backend.

It is not part of GNU Emacs anymore:

2007-11-17  Richard Stallman  <rms@gnu.org>

	* assistant.el: Remove file.

> If not, it becomes a pretty complicated explanation as you saw.  What
> direction would you prefer?

Don't know. I will check.

> Thank you
> Ted

Best regards, Michael.

Re: Passwords, stored in Gnome Keyring and KWallet

[Ted Zlatanov]

On Tue, 08 Jun 2010 15:56:14 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
MA> The crucial point is how to populate the Gnome Keyring with valid
MA> entries. For .authinfo files it is obvious, "edit the file".
>> 
>> Maybe assistant.el could be useful?  This is a good simple task: set up
>> auth-source entries for various types of accounts.  The manual doesn't
>> have to explain as much then, and the .ast file can describe the
>> secrets.el steps to create and populate a Secrets API backend.

MA> It is not part of GNU Emacs anymore:

MA> 2007-11-17  Richard Stallman  <rms@gnu.org>

MA> 	* assistant.el: Remove file.

RMS removed it because it had no use cases.  I don't think it will be a
problem to bring it back if we are using it.  It's certainly the only
such interface (for guided setup) for Emacs that I know.

Ted