Password protection

Password protection

[Lars Magne Ingebrigtsen]

I find it sort of puzzling that we have to jump through all these hoops
to get at credentials.  I mean, Firefox users don't have to set up a gpg
agent or type their passwords a gazillion times, so why should users?

But then I thought about it, and it is rather complicated.  It's
acceptable to store the passwords in memory (that's what Firefox does),
but it's not acceptable that any Lisp phrase can say
(get-stored-password ...), and then get the password.  That's too
unsafe.

So here's my thought:  If there was a C-level function that would slurp
in your ~/.authinfo.gpg data, and then let you use it, but without
actually ever letting a Lisp-level function see the passwords --
wouldn't that be nice?

Here's how I see it working:

1) Gnus calls (authinfo-store-tokens "~/.authinfo.gpg"), and the user is
(probably) prompted for a password.

2) The data is stored in the C layer, probably obfuscated in some way.

3) A new C function is added:

(process-send-auth process "LOGIN larsi %p\n\r"
                   '((:hosts ("imap.gmail.com"))
                     (:ports ("imaps" "imap" 443))
                     (:user ("larsi"))))

This function would then work just like `process-send-string', only that
it roots out the first matching password from the auth info first, and
expand the string sent.

That way the Lisp application layer will never actually see the
password, but it will be able to control what's otherwise being sent,
and what credentials to use in a flexible manner.

This should be as safe as the Firefox model.  That is, if you read
/proc/mem, you can get at the passwords, but it's not trivially
available from the Lisp layer.  Well, unless you set up a loopback
server or a proxy or something, but the same is the case with Firefox.

Am I missing something obvious here?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Richard Riley]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I find it sort of puzzling that we have to jump through all these hoops
> to get at credentials.  I mean, Firefox users don't have to set up a gpg
> agent or type their passwords a gazillion times, so why should users?

You need the agent as its secure and talks to the back end to use the
same keys which are used for other authentications such as ssh.

Firefox doesnt.


>
> But then I thought about it, and it is rather complicated.  It's
> acceptable to store the passwords in memory (that's what Firefox does),
> but it's not acceptable that any Lisp phrase can say
> (get-stored-password ...), and then get the password.  That's too
> unsafe.
>
> So here's my thought:  If there was a C-level function that would slurp
> in your ~/.authinfo.gpg data, and then let you use it, but without
> actually ever letting a Lisp-level function see the passwords --
> wouldn't that be nice?
>
> Here's how I see it working:
>
> 1) Gnus calls (authinfo-store-tokens "~/.authinfo.gpg"), and the user is
> (probably) prompted for a password.
>
> 2) The data is stored in the C layer, probably obfuscated in some way.
>
> 3) A new C function is added:
>
> (process-send-auth process "LOGIN larsi %p\n\r"
>                    '((:hosts ("imap.gmail.com"))
>                      (:ports ("imaps" "imap" 443))
>                      (:user ("larsi"))))
>
> This function would then work just like `process-send-string', only that
> it roots out the first matching password from the auth info first, and
> expand the string sent.
>
> That way the Lisp application layer will never actually see the
> password, but it will be able to control what's otherwise being sent,
> and what credentials to use in a flexible manner.
>
> This should be as safe as the Firefox model.  That is, if you read
> /proc/mem, you can get at the passwords, but it's not trivially
> available from the Lisp layer.  Well, unless you set up a loopback
> server or a proxy or something, but the same is the case with Firefox.
>
> Am I missing something obvious here?

-- 
☘ http://www.shamrockirishbar.com, http://splash-of-open-sauce.blogspot.com/ http://www.richardriley.net

Re: Password protection

[Julien Danjou]

[-- Attachment #1: Type: text/plain, Size: 287 bytes --]

On Tue, Sep 28 2010, Lars Magne Ingebrigtsen wrote:

> Am I missing something obvious here?

If the process run by the Lisp code is `cat > file', Lisp code reading
file will get the password, won't it?

-- 
Julien Danjou
// ᐰ <julien@danjou.info>   http://julien.danjou.info

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Password protection

[Ted Zlatanov]

On Tue, 28 Sep 2010 16:17:57 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> I find it sort of puzzling that we have to jump through all these hoops
LMI> to get at credentials.  I mean, Firefox users don't have to set up a gpg
LMI> agent or type their passwords a gazillion times, so why should users?

Look at it the other way: why shouldn't they set up a GPG agent or enter
a passphrase?  (assuming that entering the passphrase more than once is
a bug I plan to fix)

With the Secrets API they don't need even that much (you should really
try it), but there's a baseline of "I am authorized to open a secure
file."  By comparison, Chrome and Firefox store passwords in a format
that's pretty easy to decrypt and only requires reading files (e.g. see
http://www.blogsdna.com/12040/how-to-recover-stored-google-chrome-passwords.htm)

The user can always set up an unencrypted file if he doesn't like
encryption.

So I'd argue that Emacs has, practically speaking, better security
*externally* than Firefox, Chrome, or most other web browsers with a
authinfo.gpg file.  Now from the inside, yes, it's a candy store of
passwords, and that's a concern.  But Doing It Right requires a lot of
infrastructure that Emacs Lisp doesn't have.  And Firefox and Chrome
extensions can get at your passwords too AFAIK.

LMI> So here's my thought:  If there was a C-level function that would slurp
LMI> in your ~/.authinfo.gpg data, and then let you use it, but without
LMI> actually ever letting a Lisp-level function see the passwords --
LMI> wouldn't that be nice?

LMI> Here's how I see it working:

LMI> 1) Gnus calls (authinfo-store-tokens "~/.authinfo.gpg"), and the user is
LMI> (probably) prompted for a password.

LMI> 2) The data is stored in the C layer, probably obfuscated in some way.

LMI> 3) A new C function is added:

LMI> (process-send-auth process "LOGIN larsi %p\n\r"
LMI>                    '((:hosts ("imap.gmail.com"))
LMI>                      (:ports ("imaps" "imap" 443))
LMI>                      (:user ("larsi"))))

LMI> This function would then work just like `process-send-string', only that
LMI> it roots out the first matching password from the auth info first, and
LMI> expand the string sent.

LMI> That way the Lisp application layer will never actually see the
LMI> password, but it will be able to control what's otherwise being sent,
LMI> and what credentials to use in a flexible manner.

You're basically describing the Secrets API, which does this over D-Bus,
allows saving a password, and has many other features.  So we could have
(auth-source-upload-secrets "authinfo.gpg") to do (1) differentially or
as an overwrite and then (2) and (3) would Just Work persistently.

Ted

Re: Password protection

[Ted Zlatanov]

On Tue, 28 Sep 2010 16:46:32 +0200 Julien Danjou <julien@danjou.info> wrote: 

JD> On Tue, Sep 28 2010, Lars Magne Ingebrigtsen wrote:
>> Am I missing something obvious here?

JD> If the process run by the Lisp code is `cat > file', Lisp code reading
JD> file will get the password, won't it?

Going in the direction of signing packages is an interesting way to
solve this.  ELPA-style repos could deliver packages signed with the
author's key and you, the user, would authorize that key.  Emacs would,
of course, come authorized for its own internal packages.  But this is
really a can of worms better spilled in emacs-devel IMO.

Ted

Re: Password protection

[David Engster]

Lars Magne Ingebrigtsen writes:
> So here's my thought:  If there was a C-level function that would slurp
> in your ~/.authinfo.gpg data, and then let you use it, but without
> actually ever letting a Lisp-level function see the passwords --
> wouldn't that be nice?

As long as this is optional I don't see much of a problem with it,
though I think that having one framework like gpg-agent is actually a
good thing. I'm using gpg-agent for passphrases, certificates (gpgsm)
and ssh-agent, and it's working really well.

-David

Re: Password protection

[Daniel Pittman]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> I find it sort of puzzling that we have to jump through all these hoops
> to get at credentials.  I mean, Firefox users don't have to set up a gpg
> agent or type their passwords a gazillion times, so why should users?

I just shove mine into ~/.netrc, set appropriate permissions, and let the
whole thing be.  I did my risk assessment, encrypted my entire disk, and was
satisfied that by the time someone could steal my password through there the
could equally steal it behind the back of any encryption.

So, yeah, I agree with you.

        Daniel

-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

Re: Password protection

[Richard Riley]

Daniel Pittman <daniel@rimspace.net> writes:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> I find it sort of puzzling that we have to jump through all these hoops
>> to get at credentials.  I mean, Firefox users don't have to set up a gpg
>> agent or type their passwords a gazillion times, so why should users?
>
> I just shove mine into ~/.netrc, set appropriate permissions, and let the
> whole thing be.  I did my risk assessment, encrypted my entire disk, and was
> satisfied that by the time someone could steal my password through there the
> could equally steal it behind the back of any encryption.
>
> So, yeah, I agree with you.

Nice. But your case is a total fringe case so doesn't justify not using
a proper agent for gpg key retrieval. To argue that its unnecessary
because your entire disk is already encrypted is probably not a strong
case ;)

Re: Password protection

[Daniel Pittman]

Richard Riley <rileyrg@googlemail.com> writes:
> Daniel Pittman <daniel@rimspace.net> writes:
>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>
>>> I find it sort of puzzling that we have to jump through all these hoops
>>> to get at credentials.  I mean, Firefox users don't have to set up a gpg
>>> agent or type their passwords a gazillion times, so why should users?
>>
>> I just shove mine into ~/.netrc, set appropriate permissions, and let the
>> whole thing be.  I did my risk assessment, encrypted my entire disk, and was
>> satisfied that by the time someone could steal my password through there the
>> could equally steal it behind the back of any encryption.
>>
>> So, yeah, I agree with you.
>
> Nice. But your case is a total fringe case so doesn't justify not using
> a proper agent for gpg key retrieval. To argue that its unnecessary
> because your entire disk is already encrypted is probably not a strong
> case ;)

You are right: the disk encryption is an attractive nuisance in my argument.

By the time someone can bypass the security precautions of having a file owned
by me, on my single user machine, they have already won plenty of other ways.

        Daniel

...besides, isn't using Gnus already a fringe activity?
-- 
✣ Daniel Pittman            ✉ daniel@rimspace.net            ☎ +61 401 155 707
               ♽ made with 100 percent post-consumer electrons

Re: Password protection

[Richard Riley]

Daniel Pittman <daniel@rimspace.net> writes:

> Richard Riley <rileyrg@googlemail.com> writes:
>> Daniel Pittman <daniel@rimspace.net> writes:
>>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>>
>>>> I find it sort of puzzling that we have to jump through all these hoops
>>>> to get at credentials.  I mean, Firefox users don't have to set up a gpg
>>>> agent or type their passwords a gazillion times, so why should users?
>>>
>>> I just shove mine into ~/.netrc, set appropriate permissions, and let the
>>> whole thing be.  I did my risk assessment, encrypted my entire disk, and was
>>> satisfied that by the time someone could steal my password through there the
>>> could equally steal it behind the back of any encryption.
>>>
>>> So, yeah, I agree with you.
>>
>> Nice. But your case is a total fringe case so doesn't justify not using
>> a proper agent for gpg key retrieval. To argue that its unnecessary
>> because your entire disk is already encrypted is probably not a strong
>> case ;)
>
> You are right: the disk encryption is an attractive nuisance in my argument.
>
> By the time someone can bypass the security precautions of having a file owned
> by me, on my single user machine, they have already won plenty of other ways.
>
>         Daniel
>
> ...besides, isn't using Gnus already a fringe activity?

Indeed! Which makes your case a fringe in a fringe ;)

The agent is used for many other things too in addition to gnus gpg file
decryption.

I log in once when emacs opens my "rgr.gpg" file and have a long timeout
set in the gpg-agent.conf.  Since the authinfo.gpg is encrypted with the
same key the agent takes care of it. No hardship at all.

The real benefits of course are the ability to use public (and free!)
git repos for ones dotemacs collection including the authinfo file
which, being encrypted as a .gpg, is pretty useless to anyone else. 


-- 
☘ http://www.shamrockirishbar.com, http://splash-of-open-sauce.blogspot.com/ http://www.richardriley.net

Re: Password protection

[Gijs Hillenius]

On 29 Sep 2010, Daniel Pittman wrote:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> I find it sort of puzzling that we have to jump through all these hoops
>> to get at credentials.  I mean, Firefox users don't have to set up a gpg
>> agent or type their passwords a gazillion times, so why should users?
>
> I just shove mine into ~/.netrc, set appropriate permissions, and let the
> whole thing be.  I did my risk assessment, encrypted my entire disk, and was
> satisfied that by the time someone could steal my password through there the
> could equally steal it behind the back of any encryption.

With the kind help of Daiku, who talked me through this, I just managed
to fix the 'having a .authinfo.gpg and getting asked six times for the
password' issue here. At gnus start-up, I'm now asked just one time:

The trick is doing this

gpg --symmetric .authinfo

Re: Password protection

[Richard Riley]

Gijs Hillenius <gijs@hillenius.net> writes:

> On 29 Sep 2010, Daniel Pittman wrote:
>
>> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>>
>>> I find it sort of puzzling that we have to jump through all these hoops
>>> to get at credentials.  I mean, Firefox users don't have to set up a gpg
>>> agent or type their passwords a gazillion times, so why should users?
>>
>> I just shove mine into ~/.netrc, set appropriate permissions, and let the
>> whole thing be.  I did my risk assessment, encrypted my entire disk, and was
>> satisfied that by the time someone could steal my password through there the
>> could equally steal it behind the back of any encryption.
>
> With the kind help of Daiku, who talked me through this, I just managed
> to fix the 'having a .authinfo.gpg and getting asked six times for the
> password' issue here. At gnus start-up, I'm now asked just one time:
>
> The trick is doing this
>
> gpg --symmetric .authinfo
>

Emacs and EPA should do that for you when you first save the gpg
file. It should prompt you for which key to use or you can select
symmetric if I remember rightly.

My authinfo.gpg is signed with a key and I am only prompted once. The
only thing I did with the command line was to create the initial key.

regards

r.


-- 
☘ http://www.shamrockirishbar.com, http://splash-of-open-sauce.blogspot.com/ http://www.richardriley.net

Re: Password protection

[Tassilo Horn]

Gijs Hillenius <gijs@hillenius.net> writes:

Hi!

> With the kind help of Daiku, who talked me through this, I just
> managed to fix the 'having a .authinfo.gpg and getting asked six times
> for the password' issue here. At gnus start-up, I'm now asked just one
> time:
>
> The trick is doing this
>
> gpg --symmetric .authinfo

Hm, I wrote the file with emacs and when saving it the first time, it
asked me to choose either a pulic key or to use symmetric encryption (if
I don't choose one).  I did the latter, provided a password, and that's
all...

I've already reported an emacs bug.

--8<---------------cut here---------------start------------->8---
From: Tassilo Horn <tassilo@member.fsf.org>
Subject: 24.0.50; EPA queries twice and doesn't cache passwords when decrypting files
To: bug-gnu-emacs@gnu.org
Date: Wed, 29 Sep 2010 09:33:04 +0200
Message-ID: <87eicdqa1b.fsf@thinkpad.tsdh.de>
User-Agent: Gnus/5.110011 (No Gnus v0.11) Emacs/24.0.50 (gnu/linux)
--8<---------------cut here---------------end--------------->8---

Feel free to add additional information there.

Bye,
Tassilo

Re: Password protection

[Gijs Hillenius]

On 29 Sep 2010, Richard Riley wrote:


[...]

>> The trick is doing this
>>
>> gpg --symmetric .authinfo
>>
>
> Emacs and EPA should do that for you when you first save the gpg
> file. It should prompt you for which key to use or you can select
> symmetric if I remember rightly.

Richard, you are right. I did not realise I had to select NONE, to enable
symmetric encryption.

Till now, I had selected my own key before hitting OK. That is how I
understood what I should do, following this document:

http://www.emacswiki.org/emacs/GnusEncryptedAuthInfo

Re: Password protection

[Lars Magne Ingebrigtsen]

Richard Riley <rileyrg@googlemail.com> writes:

> You need the agent as its secure and talks to the back end to use the
> same keys which are used for other authentications such as ssh.
>
> Firefox doesnt.

I know.  What's good enough for Firefox is good enough for us.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Lars Magne Ingebrigtsen]

Julien Danjou <julien@danjou.info> writes:

> If the process run by the Lisp code is `cat > file', Lisp code reading
> file will get the password, won't it?

Sure.  It's as easy to get passwords out of Firefox.  (Well, almost.
Everything is easier with Lisp.  :-)

Probably the main gain from pushing the passwords down into the C layer
is that the passwords will never appear in any Lisp backtraces, or any
logs, or anything like that.  Inadvertent password disclosure in the
current Emacs is a real problem.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Look at it the other way: why shouldn't they set up a GPG agent or enter
> a passphrase?  (assuming that entering the passphrase more than once is
> a bug I plan to fix)

Entering a pass phrase (once) is fine if they've chosen to set things up
like that.  Firefox also allows the passwords to be, er,
password-protected.  :-)

But setting up a GPG agent is way too much for a normal user.  I think.

> So I'd argue that Emacs has, practically speaking, better security
> *externally* than Firefox, Chrome, or most other web browsers with a
> authinfo.gpg file.  Now from the inside, yes, it's a candy store of
> passwords, and that's a concern.  But Doing It Right requires a lot of
> infrastructure that Emacs Lisp doesn't have.  And Firefox and Chrome
> extensions can get at your passwords too AFAIK.

Yeah.  A `process-send-password' function makes the candy somewhat less
tasty, I think...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Charles Philip Chan]

[-- Attachment #1: Type: text/plain, Size: 486 bytes --]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> But setting up a GPG agent is way too much for a normal user.  I
> think.

It is not hard to setup at all with gpg1. AFAIK, all distros that uses
gpg1 have it setup out of the box. Also, gpg1 is because obsolete, gpg2
uses the agent by default- there is no way to turn off agent use and
pinentry.

Charles

-- 
"The world is beating a path to our door"

  -- Bruce Perens, (Open Sources, 1999 O'Reilly and Associates)

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Password protection

[Ted Zlatanov]

On Wed, 29 Sep 2010 16:07:03 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Look at it the other way: why shouldn't they set up a GPG agent or enter
>> a passphrase?  (assuming that entering the passphrase more than once is
>> a bug I plan to fix)

LMI> Entering a pass phrase (once) is fine if they've chosen to set things up
LMI> like that.  Firefox also allows the passwords to be, er,
LMI> password-protected.  :-)

Right.  So the usability issues are the same.

LMI> But setting up a GPG agent is way too much for a normal user.  I think.

I really don't think you can get reasonable security without a
passphrase entry at some point.  The GPG agent is securely written to do
this.  The Secrets API (about which I wrote some proposals you skipped)
is also much more secure than Emacs can be and makes it unnecessary to
enter any passphrases, being keyed to your user login.

>> So I'd argue that Emacs has, practically speaking, better security
>> *externally* than Firefox, Chrome, or most other web browsers with a
>> authinfo.gpg file.  Now from the inside, yes, it's a candy store of
>> passwords, and that's a concern.  But Doing It Right requires a lot of
>> infrastructure that Emacs Lisp doesn't have.  And Firefox and Chrome
>> extensions can get at your passwords too AFAIK.

LMI> Yeah.  A `process-send-password' function makes the candy somewhat less
LMI> tasty, I think...

As I said, the Secrets API implements exactly this and is available
through auth-source.el without any C code required.  What doesn't it do?

Ted

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I really don't think you can get reasonable security without a
> passphrase entry at some point.  The GPG agent is securely written to do
> this.  The Secrets API (about which I wrote some proposals you skipped)
> is also much more secure than Emacs can be and makes it unnecessary to
> enter any passphrases, being keyed to your user login.

Yes, I agree that a passphrase has to be entered at some point.

> As I said, the Secrets API implements exactly this and is available
> through auth-source.el without any C code required.  What doesn't it do?

Doesn't the Secrets API return a password back to Emacs Lisp land?

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Ted Zlatanov]

On Wed, 29 Sep 2010 21:27:54 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

>> As I said, the Secrets API implements exactly this and is available
>> through auth-source.el without any C code required.  What doesn't it do?

LMI> Doesn't the Secrets API return a password back to Emacs Lisp land?

Yes, because `secrets-get-secret' is an ELisp function.  But we can have
`secrets-send-secret-to-process' and
`auth-source-send-user-or-password-to-process' at the C level.

Ted

Re: Password protection

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Wed, 29 Sep 2010 21:27:54 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>
> LMI> Doesn't the Secrets API return a password back to Emacs Lisp land?
>
> Yes, because `secrets-get-secret' is an ELisp function.  But we can have
> `secrets-send-secret-to-process' and

You mean calling Fdbus_call_method on C level?

> `auth-source-send-user-or-password-to-process' at the C level.
>
> Ted

Best regards, Michael.

Re: Password protection

[Ted Zlatanov]

On Wed, 29 Sep 2010 22:35:33 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
>> On Wed, 29 Sep 2010 21:27:54 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>> 
LMI> Doesn't the Secrets API return a password back to Emacs Lisp land?
>> 
>> Yes, because `secrets-get-secret' is an ELisp function.  But we can have
>> `secrets-send-secret-to-process' and

MA> You mean calling Fdbus_call_method on C level?

I think Lars wants it to be more specific to passwords, not a general
D-Bus call.  But yes, it would eventually call out to that function.

Ted

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Yes, because `secrets-get-secret' is an ELisp function.  But we can have
> `secrets-send-secret-to-process' and
> `auth-source-send-user-or-password-to-process' at the C level.

Right.

Well, we can see what we're doing after you've done the auth-source
rework to make sure that people don't need to type the ~/.authinfo.pgp
password so many times...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> LMI> Doesn't the Secrets API return a password back to Emacs Lisp land?
>>> 
>>> Yes, because `secrets-get-secret' is an ELisp function.  But we can have
>>> `secrets-send-secret-to-process' and
>
> MA> You mean calling Fdbus_call_method on C level?
>
> I think Lars wants it to be more specific to passwords, not a general
> D-Bus call.  But yes, it would eventually call out to that function.

Hmm. Just a rough idea: what a about a new lisp object "password"? It
can be created only via an (interactive) C-level function, and another
C-level function returns the carried string. On lisp level, the value
is available only as hash string of the password itself. By this,
functions like `eq' or `string-equal' could still be applied.

C-level functions, visible in lisp (like `process-send-string' or
`dbus-call-method'), could be tought to accept a password *or* a string
as parameter.

> Ted

Best regards, Michael.

Re: Password protection

[Ted Zlatanov]

On Thu, 30 Sep 2010 05:32:53 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Ted Zlatanov <tzz@lifelogs.com> writes:
LMI> Doesn't the Secrets API return a password back to Emacs Lisp land?
>>>> 
>>>> Yes, because `secrets-get-secret' is an ELisp function.  But we can have
>>>> `secrets-send-secret-to-process' and
>> 
MA> You mean calling Fdbus_call_method on C level?
>> 
>> I think Lars wants it to be more specific to passwords, not a general
>> D-Bus call.  But yes, it would eventually call out to that function.

MA> Hmm. Just a rough idea: what a about a new lisp object "password"? It
MA> can be created only via an (interactive) C-level function, and another
MA> C-level function returns the carried string. On lisp level, the value
MA> is available only as hash string of the password itself. By this,
MA> functions like `eq' or `string-equal' could still be applied.

MA> C-level functions, visible in lisp (like `process-send-string' or
MA> `dbus-call-method'), could be tought to accept a password *or* a string
MA> as parameter.

So only a process or C code can see or set the real value of a secret
token?  That seems like a great idea.

Ted

Re: Password protection

[Lars Magne Ingebrigtsen]

Michael Albinus <michael.albinus@gmx.de> writes:

> Hmm. Just a rough idea: what a about a new lisp object "password"? It
> can be created only via an (interactive) C-level function, and another
> C-level function returns the carried string. On lisp level, the value
> is available only as hash string of the password itself. By this,
> functions like `eq' or `string-equal' could still be applied.

It sounds nice, but I think introducing an entirely new object type just
for passwords will be a hard sell with the Emacs maintainers.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Ted Zlatanov]

On Thu, 30 Sep 2010 18:25:57 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Michael Albinus <michael.albinus@gmx.de> writes:
>> Hmm. Just a rough idea: what a about a new lisp object "password"? It
>> can be created only via an (interactive) C-level function, and another
>> C-level function returns the carried string. On lisp level, the value
>> is available only as hash string of the password itself. By this,
>> functions like `eq' or `string-equal' could still be applied.

LMI> It sounds nice, but I think introducing an entirely new object type just
LMI> for passwords will be a hard sell with the Emacs maintainers.

If it doesn't break existing code, doesn't slow Emacs down, and provides
needed functionality we can't get in any other way, they'll probably OK
it.  Do you or Michael want to make a proposal or should I?

Ted

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> If it doesn't break existing code, doesn't slow Emacs down, and provides
> needed functionality we can't get in any other way, they'll probably OK
> it.  Do you or Michael want to make a proposal or should I?

I'm not quite sure how it's going to be used, either.  Today, nnimap says

(process-send-string ... (format "LOGIN %S %S" user password))

That, obviously, can't work any more.  So in what circumstances will
this appear?  If it's only in new functions like process-send-password,
then it kinda seems like the type isn't needed, because we could just
load it into the C layer without ever exporting it back to the Lisp
layer at all...

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Ted Zlatanov]

On Thu, 30 Sep 2010 18:47:58 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> If it doesn't break existing code, doesn't slow Emacs down, and provides
>> needed functionality we can't get in any other way, they'll probably OK
>> it.  Do you or Michael want to make a proposal or should I?

LMI> I'm not quite sure how it's going to be used, either.  Today, nnimap says

LMI> (process-send-string ... (format "LOGIN %S %S" user password))

LMI> That, obviously, can't work any more.  So in what circumstances will
LMI> this appear?  If it's only in new functions like process-send-password,
LMI> then it kinda seems like the type isn't needed, because we could just
LMI> load it into the C layer without ever exporting it back to the Lisp
LMI> layer at all...

But then you won't be able to pass the secret tokens around or examine
their hashes.  Those are valuable tools for debugging and building more
functionality around the secret tokens.  Generally I'd rather
encapsulate secrets safely than make them inaccessible.

Your example would not change.  I think it could be:

(let ((password (make-secret "hello")))
   (format "%s" password) ; #SECRET#abc123 is the unique one-way hash
   (process-send-string ... password) ; sends the password
   (process-send-string ... #SECRET#abc123) ; sends the password also
   (process-send-string ... (format "%s" #SECRET#abc123)) ; sends the externally useless hash
   (debug password)) ; shows #SECRET#abc123

So only process-send-string and C code would be able to look inside a
secret token.  It will complicate the example code a tiny bit.

Ted

Re: Password protection

[Michael Albinus]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> If it doesn't break existing code, doesn't slow Emacs down, and provides
>> needed functionality we can't get in any other way, they'll probably OK
>> it.  Do you or Michael want to make a proposal or should I?
>
> I'm not quite sure how it's going to be used, either.  Today, nnimap says
>
> (process-send-string ... (format "LOGIN %S %S" user password))
>
> That, obviously, can't work any more.  So in what circumstances will
> this appear?  If it's only in new functions like process-send-password,
> then it kinda seems like the type isn't needed, because we could just
> load it into the C layer without ever exporting it back to the Lisp
> layer at all...

The object might be used whereever a string is allowed, even as
substring of another string. Similar to objects handled by
number-or-marker-p, which accepts a number or a marker object.

process-send-string would recognize, that the string returned by (format
...) contains a password object, and it would expand it.

Best regards, Michael.

Re: Password protection

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 30 Sep 2010 18:25:57 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 
>
> LMI> Michael Albinus <michael.albinus@gmx.de> writes:
>>> Hmm. Just a rough idea: what a about a new lisp object "password"? It
>>> can be created only via an (interactive) C-level function, and another
>>> C-level function returns the carried string. On lisp level, the value
>>> is available only as hash string of the password itself. By this,
>>> functions like `eq' or `string-equal' could still be applied.
>
> LMI> It sounds nice, but I think introducing an entirely new object type just
> LMI> for passwords will be a hard sell with the Emacs maintainers.

Yes, it is a high hurdle to jump over.

> If it doesn't break existing code, doesn't slow Emacs down, and provides
> needed functionality we can't get in any other way, they'll probably OK
> it.  Do you or Michael want to make a proposal or should I?

I'll be happy to support you. My English might not be fluid enough for
the discussion (often, I have the impression Stefan simply does not
understand what I'm saying. Because of me).

> Ted

Best regards, Michael.

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> But then you won't be able to pass the secret tokens around or examine
> their hashes.  Those are valuable tools for debugging and building more
> functionality around the secret tokens.  Generally I'd rather
> encapsulate secrets safely than make them inaccessible.

It has debugging value, but since you're not going to be able to ever
actually see their real value in the Lisp layer, but have to write
special C functions to do anything with them, I think it has low value
otherwise.  So it mainly has all the drawbacks connected with
introducing a new type.

If you wish to have some debuggability, you can just have a function
like `(secret-credential-hash "imap.gmail.com" "imaps" "password")'
to return a hash of the "password" secret that's stashed, and you don't
need any new type.

> Your example would not change.  I think it could be:
>
> (let ((password (make-secret "hello")))
>    (format "%s" password) ; #SECRET#abc123 is the unique one-way hash
>    (process-send-string ... password) ; sends the password
>    (process-send-string ... #SECRET#abc123) ; sends the password also
>    (process-send-string ... (format "%s" #SECRET#abc123)) ; sends the externally useless hash
>    (debug password)) ; shows #SECRET#abc123
>
> So only process-send-string and C code would be able to look inside a
> secret token.  It will complicate the example code a tiny bit.

No, I don't think that'd actually work.  To take an example: Passwords
in IMAP needs to be utf7-imap-encoded and quote-treated.  (I think.)  So if
you have the password "hello "&" goodbye", you need to do the following
transform:

(format "%S" (utf7-encode password) t)

and what gets sent will be

"hello \"&utf7-thing\" goodbye"

Oh.  Hm.  But that won't work with my simpler process-send-password
thing either.  I had thought of the transform (because if you send basic
auth, you need to base64-encode the stuff first), but
Fbase64_encode_string is a C string, so I though you could just send a
transform down, too.  But here you have Lisp-level functions like
utf7-encode being called...

I don't think this is as simple as any of us thought.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Michael Albinus]

Ted Zlatanov <tzz@lifelogs.com> writes:

> MA> Hmm. Just a rough idea: what a about a new lisp object "password"? It
> MA> can be created only via an (interactive) C-level function, and another
> MA> C-level function returns the carried string. On lisp level, the value
> MA> is available only as hash string of the password itself. By this,
> MA> functions like `eq' or `string-equal' could still be applied.
>
> MA> C-level functions, visible in lisp (like `process-send-string' or
> MA> `dbus-call-method'), could be tought to accept a password *or* a string
> MA> as parameter.
>
> So only a process or C code can see or set the real value of a secret
> token?  That seems like a great idea.

Btw: we should not overestimate the kind of protection. Everybody can
install an asynchronous process, send the password object via
process-send-string, and read the plain password in the process
buffer. Everything on Elisp level.

> Ted

Best regards, Michael.

Re: Password protection

[Ted Zlatanov]

On Thu, 30 Sep 2010 19:17:46 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> It has debugging value, but since you're not going to be able to ever
LMI> actually see their real value in the Lisp layer, but have to write
LMI> special C functions to do anything with them, I think it has low value
LMI> otherwise.  So it mainly has all the drawbacks connected with
LMI> introducing a new type.

On Thu, 30 Sep 2010 19:19:07 +0200 Michael Albinus <michael.albinus@gmx.de> wrote: 

MA> Btw: we should not overestimate the kind of protection. Everybody can
MA> install an asynchronous process, send the password object via
MA> process-send-string, and read the plain password in the process
MA> buffer. Everything on Elisp level.

"So, other than that, Mrs. Lincoln, how did you like the play?" :)

I still say float it in emacs-devel and let them pick it apart.
Something useful may come out.  None of us are security experts.

Ted

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> I still say float it in emacs-devel and let them pick it apart.
> Something useful may come out.  None of us are security experts.

People over here are more chatty, so its easier to float ideas and get
feedback.  :-)

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Michael Albinus]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> Ted Zlatanov <tzz@lifelogs.com> writes:
>
>> I still say float it in emacs-devel and let them pick it apart.
>> Something useful may come out.  None of us are security experts.
>
> People over here are more chatty, so its easier to float ideas and get
> feedback.  :-)

I agree with both of you :-) I would say let's keep it 2 or 3 days in
ding, and when discussion becomes lame, let's go to emacs-devel.

Best regards, Michael.

Re: Password protection

[Ted Zlatanov]

On Wed, 29 Sep 2010 23:38:42 +0200 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Yes, because `secrets-get-secret' is an ELisp function.  But we can have
>> `secrets-send-secret-to-process' and
>> `auth-source-send-user-or-password-to-process' at the C level.

LMI> Right.

LMI> Well, we can see what we're doing after you've done the auth-source
LMI> rework to make sure that people don't need to type the ~/.authinfo.pgp
LMI> password so many times...

Any interest in pursuing this on emacs-devel now that I've done the
above with your (copious) help?

Ted

Re: Password protection

[Lars Magne Ingebrigtsen]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Any interest in pursuing this on emacs-devel now that I've done the
> above with your (copious) help?

I don't really have any ideas as to how to usably only expose passwords
on the C level and not on the Lisp level, since you usually need to do
transforms on the string (base64, etc) before sending it.

I think, perhaps, the most workable scheme for not exposing passwords in
backtraces and the like is the one you've come up with -- the lexical
bindings.

The other scheme that may work is the "have a password data type" think,
which would be oblique to the normal printing routines, but where you'd
have a special accessor to get the real string.  But it seems kinda
heavyweight, especially since the lexical binding thing seems to work
just fine.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: Password protection

[Ted Zlatanov]

On Sat, 05 Mar 2011 12:53:57 +0100 Lars Magne Ingebrigtsen <larsi@gnus.org> wrote: 

LMI> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Any interest in pursuing this on emacs-devel now that I've done the
>> above with your (copious) help?

LMI> I think, perhaps, the most workable scheme for not exposing passwords in
LMI> backtraces and the like is the one you've come up with -- the lexical
LMI> bindings.

OK, we'll leave it alone then.  It seems to work OK in every way.

Ted