auth-sources asking for password 2 or 3 times

auth-sources asking for password 2 or 3 times

[Sivaram Neelakantan]

Hi,

I'm using the latest git pull of gnus and things seem to work as
expected with authinfo.gpg.  But it keeps asking for the symmetric
password far too many times.

/home/sivaramn/.auth: 0% (0/136)
/home/sivaramn/.auth: 100% (136/136)
/home/sivaramn/.auth: 0% (0/136)
/home/sivaramn/.auth: 100% (136/136)
/home/sivaramn/.auth: 0% (0/136)
/home/sivaramn/.auth: 100% (136/136)

235 2.7.0 Accepted
250 2.1.0 OK p436848wfc.17
250 2.1.5 OK p436848wfc.17
354  Go ahead p436848wfc.17
250 2.0.0 OK 1298225550 p436848wfc.17
221 2.0.0 closing connection p436848wfc.17
Sending...done

I looked up the info manual and simply ended up adding

(setq epa-file-cache-passphrase-for-symmetric-encryption t)

Sometimes it asks twice, sometimes thrice in a row in the act of
hitting C-C C-c.

Other than that, things work as expected.

 sivaram
 -- 

Re: auth-sources asking for password 2 or 3 times

[Lars Ingebrigtsen]

Sivaram Neelakantan <nsivaram.net@gmail.com> writes:

> I'm using the latest git pull of gnus and things seem to work as
> expected with authinfo.gpg.  But it keeps asking for the symmetric
> password far too many times.

I think the obvious solution here is to just add the same
~/.authinfo.gpg caching code to auth-source as I added to netrc.el.

Otherwise Gnus just isn't usable out-of-the-box if you're using a .gpg
file.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: auth-sources asking for password 2 or 3 times

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I think the obvious solution here is to just add the same
> ~/.authinfo.gpg caching code to auth-source as I added to netrc.el.
>
> Otherwise Gnus just isn't usable out-of-the-box if you're using a .gpg
> file.

I've now done this, so you should only be queried for the .gpg password
once.

This is, of course, unsafe, but until we get a better solution into
auth-source, it's the only viable solution.  Feel free to remove it
after something better is in place.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: auth-sources asking for password 2 or 3 times

[Ted Zlatanov]

On Sun, 20 Feb 2011 17:35:57 -0800 Lars Ingebrigtsen <larsi@gnus.org> wrote: 

LI> Lars Ingebrigtsen <larsi@gnus.org> writes:
>> I think the obvious solution here is to just add the same
>> ~/.authinfo.gpg caching code to auth-source as I added to netrc.el.
>> 
>> Otherwise Gnus just isn't usable out-of-the-box if you're using a .gpg
>> file.

LI> I've now done this, so you should only be queried for the .gpg password
LI> once.

LI> This is, of course, unsafe, but until we get a better solution into
LI> auth-source, it's the only viable solution.  Feel free to remove it
LI> after something better is in place.  :-)

I put a change for this to use lexical-bind and obfuscated data stored
inside the lambda function.  I think it's as safe as we can get.  IMHO
EPA/EPG are not going to do the caching for us so you were right to move
it to the auth-source level.

Ted

Re: auth-sources asking for password 2 or 3 times

[Daiki Ueno]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I put a change for this to use lexical-bind and obfuscated data stored
> inside the lambda function.  I think it's as safe as we can get.  IMHO
> EPA/EPG are not going to do the caching for us so you were right to move
> it to the auth-source level.

I have been always unhappy to see that you complain "EPA/EPG are not
going to do the caching" again and again, although I see a pain in the
neck is in auth-source/netrc rather than EPA/EPG.

Why auth-source/netrc tries to visit ~/.authinfo.gpg multiple times even
for only one connection?  My guess is that, auth-source/netrc tries to
open that file for each parameter (e.g. user, host, port, password),
right?  If so, it looks to me superfluous, since user/host/port are
generally not a secret information.

How about splitting ~/.authinfo.gpg into 2 files, one is for non-secret
information and another is for secret information?  The non-secret file
would be a plain text compatible with netrc, while the secret file would
be encrypted and the decrypted content is a simple 1:1 mapping from ID
(auth-source token?) to password.

Just a thought.

Regards,
-- 
Daiki Ueno

Re: auth-sources asking for password 2 or 3 times

[Ted Zlatanov]

On Wed, 23 Feb 2011 11:14:01 +0900 Daiki Ueno <ueno@unixuser.org> wrote: 

DU> Ted Zlatanov <tzz@lifelogs.com> writes:
>> I put a change for this to use lexical-bind and obfuscated data stored
>> inside the lambda function.  I think it's as safe as we can get.  IMHO
>> EPA/EPG are not going to do the caching for us so you were right to move
>> it to the auth-source level.

DU> I have been always unhappy to see that you complain "EPA/EPG are not
DU> going to do the caching" again and again, although I see a pain in the
DU> neck is in auth-source/netrc rather than EPA/EPG.

Sorry if it seems like I'm complaining.  My point was just that EPA/EPG
shouldn't have to do caching to accomodate auth-source.el usage (which
is very different from the user-level interactions).  It works well and
I appreciate how much work you've done on it.

DU> Why auth-source/netrc tries to visit ~/.authinfo.gpg multiple times even
DU> for only one connection?  My guess is that, auth-source/netrc tries to
DU> open that file for each parameter (e.g. user, host, port, password),
DU> right?  If so, it looks to me superfluous, since user/host/port are
DU> generally not a secret information.

I don't think that's the case, at least not anymore (I changed quite a
bit today).  You can see in *Messages* (if you set `auth-source-debug'
to 'trivia) one of these messages:

"auth-source-netrc-parse: using CACHED file data for %s"

or one EPA/EPG decode message like this:

/home/tzz/autodist/f: 0% (0/1949)
/home/tzz/autodist/f: 100% (1949/1949)

per file per search.  If I'm wrong, please let me know so I can fix the
search.

DU> How about splitting ~/.authinfo.gpg into 2 files, one is for non-secret
DU> information and another is for secret information?  The non-secret file
DU> would be a plain text compatible with netrc, while the secret file would
DU> be encrypted and the decrypted content is a simple 1:1 mapping from ID
DU> (auth-source token?) to password.

That's exactly why `auth-sources' defaults to the list "~/.authinfo.gpg"
"~/.authinfo" "~/.netrc".  I'm not sure why I'd make the encrypted file
in a different format, though.  That would make it hard to move entries
between the two formats and would confuse users.  Can you explain if I
misunderstood?

Don't forget auth-source.el supports the Secrets API as well, which has
a completely different way to search and expand results.  I'll work on
the 'secrets backend to make it connect with Chrome password entries,
for instance.

Thanks
Ted

Re: auth-sources asking for password 2 or 3 times

[Daiki Ueno]

Ted Zlatanov <tzz@lifelogs.com> writes:

> DU> Why auth-source/netrc tries to visit ~/.authinfo.gpg multiple times even
> DU> for only one connection?  My guess is that, auth-source/netrc tries to
> DU> open that file for each parameter (e.g. user, host, port, password),
> DU> right?  If so, it looks to me superfluous, since user/host/port are
> DU> generally not a secret information.
>
> I don't think that's the case, at least not anymore (I changed quite a
> bit today).

Then, that's good.  I will try later.

> DU> How about splitting ~/.authinfo.gpg into 2 files, one is for non-secret
> DU> information and another is for secret information?  The non-secret file
> DU> would be a plain text compatible with netrc, while the secret file would
> DU> be encrypted and the decrypted content is a simple 1:1 mapping from ID
> DU> (auth-source token?) to password.
>
> That's exactly why `auth-sources' defaults to the list "~/.authinfo.gpg"
> "~/.authinfo" "~/.netrc".  I'm not sure why I'd make the encrypted file
> in a different format, though.  That would make it hard to move entries
> between the two formats and would confuse users.  Can you explain if I
> misunderstood?

I agree with that it might be hard for users to maintain two files.

However, you seem to be missing the point of my idea, FWIW, here is the
detail:

If auth-source.el looks for several parameters (say,
user/host/port/password) to establish a connection, it needs to decrypt
~/.authinfo.gpg (at least) 4 times if cache is disabled (right?).

However, if we store user/host/port/token in a plain text file (say,
~/.netrc), and store token/password mapping in an encrypted file (say,
~/.passwords.gpg), auth-source.el needs to decrypt the latter file only
once.

In other words, my idea is to delay decryption until password is really
necessary.  This is useful when accessing password-less news servers
(e.g. gmane).  Currently, if I start Gnus with M-x gnus-no-server and
open news.gmane.org, it asks a password for ~/.authinfo.gpg.

> Don't forget auth-source.el supports the Secrets API as well, which has
> a completely different way to search and expand results.  I'll work on
> the 'secrets backend to make it connect with Chrome password entries,
> for instance.

After brief look at the secrets API, it also seems to consider lookup
attributes as non-secret information, and only passwords have to be
encrypted on the disk.

Regards,
-- 
Daiki Ueno

Re: auth-sources asking for password 2 or 3 times

[Lars Ingebrigtsen]

Daiki Ueno <ueno@unixuser.org> writes:

> In other words, my idea is to delay decryption until password is really
> necessary.  This is useful when accessing password-less news servers
> (e.g. gmane).  Currently, if I start Gnus with M-x gnus-no-server and
> open news.gmane.org, it asks a password for ~/.authinfo.gpg.

I think both the user name and the password can be considered secret.

The reason Gnus needs to read the ~/.authinfo file even for servers that
may not demand a password is that if you do give a user name and a
password to (for instance) Gmane, you get other privileges/groups.  So
Gnus doesn't really know before it opens the file whether it needs to.

But in that instance, having the password be in a separate secret file
would certainly help, since most people (except the Gmane admins) do not
use a user name/password when contacting news.gmane.org.

However, I think the train has left when it comes to the ~/.authinfo
format.  It's always been that way, and users are used to it, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: auth-sources asking for password 2 or 3 times

[Daiki Ueno]

Lars Ingebrigtsen <larsi@gnus.org> writes:

>> In other words, my idea is to delay decryption until password is really
>> necessary.  This is useful when accessing password-less news servers
>> (e.g. gmane).  Currently, if I start Gnus with M-x gnus-no-server and
>> open news.gmane.org, it asks a password for ~/.authinfo.gpg.
>
> I think both the user name and the password can be considered secret.

OK.

> But in that instance, having the password be in a separate secret file
> would certainly help, since most people (except the Gmane admins) do not
> use a user name/password when contacting news.gmane.org.
>
> However, I think the train has left when it comes to the ~/.authinfo
> format.  It's always been that way, and users are used to it, I think.

Do you mean it is not feasible to change the format anymore?  Though I'm
not familiar with the history of the netrc format, I was thinking of the
following extension introducing a new keyword "credential", which takes
an ID associated with a password along with hidden attributes:

$ cat ~/.authinfo # plain text
machine example.org credential my-home-imap port imap

$ gpg < ~/.secrets.org.gpg
|--------------+----------+------------|
| id           | password | attributes |
|--------------+----------+------------|
| my-home-imap | PaSSwoRd | user=foo   |
|--------------+----------+------------|

If an entry in ~/.authinfo has neither "login", "password", nor
"credential", Gnus could consider the entry password-less and would not
try to decrypt ~/.secrets.org.gpg.

I think there will be no compatibility issue, except the netrc format
extension.  If a user want to try this new feature, he could just
customize auth-sources so that it points to ~/.authinfo instead of
~/.authinfo.gpg.

Regards,
-- 
Daiki Ueno

Re: auth-sources asking for password 2 or 3 times

[Ted Zlatanov]

On Wed, 23 Feb 2011 21:25:57 +0900 Daiki Ueno <ueno@unixuser.org> wrote: 

DU> Do you mean it is not feasible to change the format anymore?  Though I'm
DU> not familiar with the history of the netrc format, I was thinking of the
DU> following extension introducing a new keyword "credential", which takes
DU> an ID associated with a password along with hidden attributes:

DU> $ cat ~/.authinfo # plain text
DU> machine example.org credential my-home-imap port imap

DU> $ gpg < ~/.secrets.org.gpg
DU> |--------------+----------+------------|
DU> | id           | password | attributes |
DU> |--------------+----------+------------|
DU> | my-home-imap | PaSSwoRd | user=foo   |
DU> |--------------+----------+------------|

DU> If an entry in ~/.authinfo has neither "login", "password", nor
DU> "credential", Gnus could consider the entry password-less and would not
DU> try to decrypt ~/.secrets.org.gpg.

DU> I think there will be no compatibility issue, except the netrc format
DU> extension.  If a user want to try this new feature, he could just
DU> customize auth-sources so that it points to ~/.authinfo instead of
DU> ~/.authinfo.gpg.

Yes, this could certainly be workable.  Could the line be:

machine example.org port imap credential my-home-imap credential-file "~/.secrets.org.gpg"

so that a) the netrc file can hold many such pointers, and b) we don't
have to change the file name spec to "fileA+fileB" as I proposed?

It's a little more verbose but IMO that's not a big deal in a small file
like netrc.  It's also backwards compatible so the users don't have to
change their existing auth-sources or their authinfo/netrc files.

Ted

Re: auth-sources asking for password 2 or 3 times

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> DU> I think there will be no compatibility issue, except the netrc format
> DU> extension.  If a user want to try this new feature, he could just
> DU> customize auth-sources so that it points to ~/.authinfo instead of
> DU> ~/.authinfo.gpg.
>
> Yes, this could certainly be workable.  Could the line be:
>
> machine example.org port imap credential my-home-imap credential-file "~/.secrets.org.gpg"
>
> so that a) the netrc file can hold many such pointers, and b) we don't
> have to change the file name spec to "fileA+fileB" as I proposed?

I think it sounds like a good idea, but I'm not quite sure that this is
really needed (in the Gnus use case, at least).

I mean, if you do have a .authinfo.gpg file, then it's very likely that
you have some passwords in there, and Gnus will need them at some
point.  As the file is cached, it doesn't really matter that connecting
to news.gmane.org queries the file, since it's already in memory.

Conversely, splitting the file up into two files does require more work
for the user if the user wants to edit the file(s).

So while it seems like a workable idea, I have a feeling that there's
(a) not a real use case there, and (b) it makes things more awkward for
the user generally.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: auth-sources asking for password 2 or 3 times

[Daiki Ueno]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> So while it seems like a workable idea, I have a feeling that there's
> (a) not a real use case there, and (b) it makes things more awkward
> for the user generally.

Right, now I changed my mind :) Maybe better approach would be to extend
secrets.el to have GPG backend as a fallback.  It could then manage
mappings across two files internally/automatically (well, though I think
gnome-keyring is way to go, some people care portability and want to
manage their password collections in Emacs editable files).

Regards,
-- 
Daiki Ueno

Re: auth-sources asking for password 2 or 3 times

[Michael Albinus]

Daiki Ueno <ueno@unixuser.org> writes:

> Right, now I changed my mind :) Maybe better approach would be to extend
> secrets.el to have GPG backend as a fallback.  It could then manage
> mappings across two files internally/automatically (well, though I think
> gnome-keyring is way to go, some people care portability and want to
> manage their password collections in Emacs editable files).

I do not understand. secrets.el is a package offering functions for the
D-Bus Secret Service API "org.freedesktop.secrets". How would GPG fit
into this?

> Regards,

Best regards, Michael.

Re: auth-sources asking for password 2 or 3 times

[Daiki Ueno]

Michael Albinus <michael.albinus@gmx.de> writes:

> Daiki Ueno <ueno@unixuser.org> writes:
>
>> Right, now I changed my mind :) Maybe better approach would be to extend
>> secrets.el to have GPG backend as a fallback.  It could then manage
>> mappings across two files internally/automatically (well, though I think
>> gnome-keyring is way to go, some people care portability and want to
>> manage their password collections in Emacs editable files).
>
> I do not understand. secrets.el is a package offering functions for the
> D-Bus Secret Service API "org.freedesktop.secrets". How would GPG fit
> into this?

Yes, I know.  However its Elisp interface could be implemented using GPG
files, without access to D-Bus service in theory?  For example, having
non-secret portion of items in ~/.emacs.d/secrets/collection.org and
secret portion of items in ~/.emacs.d/secrets/collection.org.gpg.

Regards,
-- 
Daiki Ueno

Re: auth-sources asking for password 2 or 3 times

[Michael Albinus]

Daiki Ueno <ueno@unixuser.org> writes:

> Yes, I know.  However its Elisp interface could be implemented using GPG
> files, without access to D-Bus service in theory?  For example, having
> non-secret portion of items in ~/.emacs.d/secrets/collection.org and
> secret portion of items in ~/.emacs.d/secrets/collection.org.gpg.

Anything goes. But wouldn't it be rather a new auth-sources backend?

> Regards,

Best regards, Michael.

Re: auth-sources asking for password 2 or 3 times

[Daiki Ueno]

Michael Albinus <michael.albinus@gmx.de> writes:

>> Yes, I know.  However its Elisp interface could be implemented using GPG
>> files, without access to D-Bus service in theory?
>
> Anything goes. But wouldn't it be rather a new auth-sources backend?

Or, a drop-in replacement of secrets.el, like ls-lisp.el for
insert-directory.  Anyway I probably understand that you would like to
to keep secrets.el simple and clean :)

Regards,
-- 
Daiki Ueno

Re: auth-sources asking for password 2 or 3 times

[Ted Zlatanov]

On Fri, 25 Feb 2011 16:17:47 +0900 Daiki Ueno <ueno@unixuser.org> wrote: 

DU> Lars Ingebrigtsen <larsi@gnus.org> writes:
>> So while it seems like a workable idea, I have a feeling that there's
>> (a) not a real use case there, and (b) it makes things more awkward
>> for the user generally.

DU> Right, now I changed my mind :) Maybe better approach would be to extend
DU> secrets.el to have GPG backend as a fallback.  It could then manage
DU> mappings across two files internally/automatically (well, though I think
DU> gnome-keyring is way to go, some people care portability and want to
DU> manage their password collections in Emacs editable files).

You mean if the Secrets API is not available, secrets.el should emulate
it with a file-based backend?  That would be useful.  Hmm.  But it
depends on the platform, too.  Often they have their own OS-level
mechanisms (for Mac OS X it's the keychain, for example).  So it may be
useful to also add more backends to auth-source.el in addition to
extending secrets.el.

If you or anyone want me to implement something specific in
auth-source.el, let me know.  I already have the Mac OS X keychain
support on my TODO list.  For secrets.el work you and Michael should
decide what's useful.

Incidentally, do you like the way I use lexical-bind to hide the secret
data in auth-source?  Is there a better way?  Lars and I were thinking
that maybe a Emacs-level C API would be better to hide secret data but I
don't think we ever formalized a proposal.

Ted

Re: auth-sources asking for password 2 or 3 times

[Ted Zlatanov]

On Wed, 23 Feb 2011 16:20:48 +0900 Daiki Ueno <ueno@unixuser.org> wrote: 

DU> I agree with that it might be hard for users to maintain two files.

DU> However, you seem to be missing the point of my idea, FWIW, here is the
DU> detail:

DU> If auth-source.el looks for several parameters (say,
DU> user/host/port/password) to establish a connection, it needs to decrypt
DU> ~/.authinfo.gpg (at least) 4 times if cache is disabled (right?).

Not anymore.  If someone uses the old API
(`auth-source-user-or-password') 4 times then yes.

DU> However, if we store user/host/port/token in a plain text file (say,
DU> ~/.netrc), and store token/password mapping in an encrypted file (say,
DU> ~/.passwords.gpg), auth-source.el needs to decrypt the latter file only
DU> once.

I see.  That seems to me a bit inconvenient: now the user has to manage
two files and keep them in sync.  But I think I understand you're trying
to separate connection parameters (everything but the :secret token)
from the secrets themselves.  Hmm.  How about a new spec in auth-sources
like this:

"~/.netrc+~/.authinfo.gpg"

which would look in netrc for all the non-secret things and then in the
second file for the secrets?

DU> In other words, my idea is to delay decryption until password is really
DU> necessary.  This is useful when accessing password-less news servers
DU> (e.g. gmane).  Currently, if I start Gnus with M-x gnus-no-server and
DU> open news.gmane.org, it asks a password for ~/.authinfo.gpg.

I think this has to be fixed in the nntp.el code.  `auth-source-search'
is called so it has to look for credentials, which means opening files.

Ted

Re: auth-sources asking for password 2 or 3 times

[Lars Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I put a change for this to use lexical-bind and obfuscated data stored
> inside the lambda function.  I think it's as safe as we can get.

Yup; nice.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen