magic constants in some startup code

magic constants in some startup code

[Richard Gorton]

We're using musl for our processor architecture; as part of doing the bring-up work, I need to fully understand process launching and thread creation.

As I'm reading through the source code, I see (more than one) 'magic' constants that I do not fully understand (musl 1.1.5), and would like to know 'how and why':


src/env/__init_tls.c:
	static long long builtin_tls[(sizeof(struct pthread) + 64)/sizeof(long long)];

I'm guessing that 64 is an arbitrary 'small' default amount of TLS?  Or is this to hold another specific bit of data?

----

src/env/__stack_chk_fail.c
	else __stack_chk_guard = (uintptr_t)&__stack_chk_guard * 1103515245;
 
the number equates to 0x41c64e6d.
Called from __init_libc as:
	 __init_ssp((void *)aux[AT_RANDOM]); 
The kernel is putting a random number into aux[AT_RANDOM] at process initialization.
Why not just put a predictable arbitrary number into __stack_chk_guard?


Regards,
	Richard
	rcgorton@cog-e.com

Re: magic constants in some startup code

[Rich Felker]

On Fri, Oct 31, 2014 at 09:31:37AM -0400, Richard Gorton wrote:
> We're using musl for our processor architecture; as part of doing
> the bring-up work, I need to fully understand process launching and
> thread creation.

Great!

> As I'm reading through the source code, I see (more than one)
> 'magic' constants that I do not fully understand (musl 1.1.5), and
> would like to know 'how and why':
> 
> 
> src/env/__init_tls.c:
> 	static long long builtin_tls[(sizeof(struct pthread) + 64)/sizeof(long long)];
> 
> I'm guessing that 64 is an arbitrary 'small' default amount of TLS?
> Or is this to hold another specific bit of data?

The main idea is to have some minimal amount of default TLS for
programs that just use a couple TLS variables, so that they don't have
an extra syscall and possible failure path in the startup code. It's
also possible (I'd have to re-check) that some of the code that aligns
pointers for TLS use is sub-optimal with respect to space and might
waste a little bit of overhead in the process of doing alignment. This
will not overflow (sizes are checked) but might be able to result in a
buffer of the exact needed size being rejected (falling back to
allocation) whereas having a little extra room avoids the issue.
Something like this was an issue at one time, or at least I didn't
demonstrate it not to be an issue, but I'm not sure if it still is or
not.

> ----
> 
> src/env/__stack_chk_fail.c
> 	else __stack_chk_guard = (uintptr_t)&__stack_chk_guard * 1103515245;
>  
> the number equates to 0x41c64e6d.
> Called from __init_libc as:
> 	 __init_ssp((void *)aux[AT_RANDOM]); 
> The kernel is putting a random number into aux[AT_RANDOM] at process initialization.
> Why not just put a predictable arbitrary number into __stack_chk_guard?

The reason you don't want a predictable arbitrary number for the stack
guard canary is that it makes it easy to bypass stack-protector by
including the known number in your overflow payload.

The idea in the above code, which really deserves a comment, is to
attempt to recover _some_ entropy from the address at which libc is
mapped (which hopefully was affected by ASLR) when AT_RANDOM is not
available. Modern Linux kernels always give you AT_RANDOM, so this
code path would only be taken on an ancient Linux version or a
non-Linux host.

The magic number 1103515245 is just a LCG, the same as what's used in
musl's rand_r() and in the C standard's sample rand(). It serves to
mix the bits somewhat, accounting for the likelihood that the mapping
address is not very random in some of its bits.

None of this is really very effective, but I've left it there because
it seems "better than nothing".

Rich

Re: magic constants in some startup code

[Richard Gorton]

Thank you (and a follow up question) - what code looks at this canary?  It is assigned to pthread_self()->canary, but I do not see any code inside musl itself that checks that value?  A work in progress?  Or does other code check this value?

Regards,
	Richard

On Oct 31, 2014, at 10:18 AM, Rich Felker <dalias@libc.org> wrote:
> 
>> ----
>> 
>> src/env/__stack_chk_fail.c
>> 	else __stack_chk_guard = (uintptr_t)&__stack_chk_guard * 1103515245;
>> 
>> the number equates to 0x41c64e6d.
>> Called from __init_libc as:
>> 	 __init_ssp((void *)aux[AT_RANDOM]); 
>> The kernel is putting a random number into aux[AT_RANDOM] at process initialization.
>> Why not just put a predictable arbitrary number into __stack_chk_guard?
> 
> The reason you don't want a predictable arbitrary number for the stack
> guard canary is that it makes it easy to bypass stack-protector by
> including the known number in your overflow payload.
> 
> The idea in the above code, which really deserves a comment, is to
> attempt to recover _some_ entropy from the address at which libc is
> mapped (which hopefully was affected by ASLR) when AT_RANDOM is not
> available. Modern Linux kernels always give you AT_RANDOM, so this
> code path would only be taken on an ancient Linux version or a
> non-Linux host.
> 
> The magic number 1103515245 is just a LCG, the same as what's used in
> musl's rand_r() and in the C standard's sample rand(). It serves to
> mix the bits somewhat, accounting for the likelihood that the mapping
> address is not very random in some of its bits.
> 
> None of this is really very effective, but I've left it there because
> it seems "better than nothing".
> 
> Rich

Re: magic constants in some startup code

[Rich Felker]

On Fri, Oct 31, 2014 at 10:31:45AM -0400, Richard Gorton wrote:
> Thank you (and a follow up question) - what code looks at this
> canary? It is assigned to pthread_self()->canary, but I do not see
> any code inside musl itself that checks that value? A work in
> progress? Or does other code check this value?

It's part of the stack-protector feature at the compiler level. gcc,
clang, and any other compilers that implement this feature generate
code to read the canary at the start of a function protected by stack
protector, store it between the saved return address and local
buffers, and check that it hasn't been clobbered before returning.

Rich

Re: magic constants in some startup code

[Andy Lutomirski]

On 10/31/2014 09:09 AM, Rich Felker wrote:
> On Fri, Oct 31, 2014 at 10:31:45AM -0400, Richard Gorton wrote:
>> Thank you (and a follow up question) - what code looks at this
>> canary? It is assigned to pthread_self()->canary, but I do not see
>> any code inside musl itself that checks that value? A work in
>> progress? Or does other code check this value?
> 
> It's part of the stack-protector feature at the compiler level. gcc,
> clang, and any other compilers that implement this feature generate
> code to read the canary at the start of a function protected by stack
> protector, store it between the saved return address and local
> buffers, and check that it hasn't been clobbered before returning.

I'm a bit confused by the code now.  Is the canary intended to be
per-thread or global?  There's a copy in struct pthread.

Also, would it make sense for musl to implement getauxval?  If so, it
might be nice to do something to avoid inadvertent misuse of the part of
AT_RANDOM value used here.

For example, musl could implement a trivial DRBG seeded by AT_RANDOM and
replace the AT_RANDOM data with the first output from the DRBG at
startup.  Then getauxval users are safe and musl can also have a stream
of decent random numbers for internal use.

If you think this is a good idea, I could implement it.  The main
downside would be that it'll require some crypto primitive.  There's
already a SHA-256 implementation in musl that could be reused, but it
would be a bit unfortunate to pull it in to all musl-linked static binaries.

--Andy

Re: Re: magic constants in some startup code

[Rich Felker]

On Fri, Oct 31, 2014 at 01:19:47PM -0700, Andy Lutomirski wrote:
> On 10/31/2014 09:09 AM, Rich Felker wrote:
> > On Fri, Oct 31, 2014 at 10:31:45AM -0400, Richard Gorton wrote:
> >> Thank you (and a follow up question) - what code looks at this
> >> canary? It is assigned to pthread_self()->canary, but I do not see
> >> any code inside musl itself that checks that value? A work in
> >> progress? Or does other code check this value?
> > 
> > It's part of the stack-protector feature at the compiler level. gcc,
> > clang, and any other compilers that implement this feature generate
> > code to read the canary at the start of a function protected by stack
> > protector, store it between the saved return address and local
> > buffers, and check that it hasn't been clobbered before returning.
> 
> I'm a bit confused by the code now.  Is the canary intended to be
> per-thread or global?  There's a copy in struct pthread.

That's a matter of matching the ABI the compiler expects/imposes. For
some archs where accessing globals is expensive and accessing TLS is
cheap, GCC reads the canary from a fixed thread-pointer-relative
address. For others, it accesses the global.

> Also, would it make sense for musl to implement getauxval?  If so, it
> might be nice to do something to avoid inadvertent misuse of the part of
> AT_RANDOM value used here.

musl does provide getauxval.

> For example, musl could implement a trivial DRBG seeded by AT_RANDOM and
> replace the AT_RANDOM data with the first output from the DRBG at
> startup.  Then getauxval users are safe and musl can also have a stream
> of decent random numbers for internal use.

This imposes a large code size cost in the mandatory startup code even
on programs that have no interest in AT_RANDOM (99% or more). Instead,
the first call to getauxval could do this, though, but I'm not sure
it's a good approach anyway. Linux has added the getrandom syscall
which can provide the BSD getentropy function or the more featureful
getrandom API, so using getauxval(AT_RANDOM) seems like a bad idea.
Even if we avoided reuse of the same data that went into the canary,
there's no way for callers using getauxval(AT_RANDOM) to tell whether
some other library code in the same process has already consumed
entropy from AT_RANDOM, so using it is not library-safe. It seems like
we should try to discourage use of getauxval(AT_RANDOM) as an entropy
source rather than giving false hope that it's safe.

> If you think this is a good idea, I could implement it.  The main
> downside would be that it'll require some crypto primitive.  There's
> already a SHA-256 implementation in musl that could be reused, but it
> would be a bit unfortunate to pull it in to all musl-linked static binaries.

Yes, code size is a concern, but it could be tucked away as a
dependency of other functions instead of being a dependency of the
startup code.

Rich

Re: magic constants in some startup code

[Andy Lutomirski]

On 10/31/2014 02:05 PM, Rich Felker wrote:
> On Fri, Oct 31, 2014 at 01:19:47PM -0700, Andy Lutomirski wrote:
>> On 10/31/2014 09:09 AM, Rich Felker wrote:
>>> On Fri, Oct 31, 2014 at 10:31:45AM -0400, Richard Gorton wrote:
>>>> Thank you (and a follow up question) - what code looks at this
>>>> canary? It is assigned to pthread_self()->canary, but I do not see
>>>> any code inside musl itself that checks that value? A work in
>>>> progress? Or does other code check this value?
>>>
>>> It's part of the stack-protector feature at the compiler level. gcc,
>>> clang, and any other compilers that implement this feature generate
>>> code to read the canary at the start of a function protected by stack
>>> protector, store it between the saved return address and local
>>> buffers, and check that it hasn't been clobbered before returning.
>>
>> I'm a bit confused by the code now.  Is the canary intended to be
>> per-thread or global?  There's a copy in struct pthread.
> 
> That's a matter of matching the ABI the compiler expects/imposes. For
> some archs where accessing globals is expensive and accessing TLS is
> cheap, GCC reads the canary from a fixed thread-pointer-relative
> address. For others, it accesses the global.
> 
>> Also, would it make sense for musl to implement getauxval?  If so, it
>> might be nice to do something to avoid inadvertent misuse of the part of
>> AT_RANDOM value used here.
> 
> musl does provide getauxval.

That'll teach me to look at the wrong version of musl.

> 
>> For example, musl could implement a trivial DRBG seeded by AT_RANDOM and
>> replace the AT_RANDOM data with the first output from the DRBG at
>> startup.  Then getauxval users are safe and musl can also have a stream
>> of decent random numbers for internal use.
> 
> This imposes a large code size cost in the mandatory startup code even
> on programs that have no interest in AT_RANDOM (99% or more). Instead,
> the first call to getauxval could do this, though, but I'm not sure
> it's a good approach anyway. Linux has added the getrandom syscall
> which can provide the BSD getentropy function or the more featureful
> getrandom API, so using getauxval(AT_RANDOM) seems like a bad idea.
> Even if we avoided reuse of the same data that went into the canary,
> there's no way for callers using getauxval(AT_RANDOM) to tell whether
> some other library code in the same process has already consumed
> entropy from AT_RANDOM, so using it is not library-safe. It seems like
> we should try to discourage use of getauxval(AT_RANDOM) as an entropy
> source rather than giving false hope that it's safe.

getrandom(2) has the annoying problem that you can't ask it for
best-effort entropy.  This caused systemd to add a /dev/urandom fallback
a few days ago (sigh).

Maybe I'll try to get a GRND_BESTEFFORT flag for getrandom into the
kernel.  I suppose that a musl getrandom wrapper could emulate that flag
(only) or something on older kernels.  Or maybe glibc and musl could
both agree to add some get_sort_of_decent_entropy function based on
AT_RANDOM.

> 
>> If you think this is a good idea, I could implement it.  The main
>> downside would be that it'll require some crypto primitive.  There's
>> already a SHA-256 implementation in musl that could be reused, but it
>> would be a bit unfortunate to pull it in to all musl-linked static binaries.
> 
> Yes, code size is a concern, but it could be tucked away as a
> dependency of other functions instead of being a dependency of the
> startup code.

Most or all existing getauxval users are unlikely to be using AT_RANDOM,
so doing this without any bloat might be hard.

--Andy

Re: Re: magic constants in some startup code

[Rich Felker]

On Fri, Oct 31, 2014 at 02:29:32PM -0700, Andy Lutomirski wrote:
> >> For example, musl could implement a trivial DRBG seeded by AT_RANDOM and
> >> replace the AT_RANDOM data with the first output from the DRBG at
> >> startup.  Then getauxval users are safe and musl can also have a stream
> >> of decent random numbers for internal use.
> > 
> > This imposes a large code size cost in the mandatory startup code even
> > on programs that have no interest in AT_RANDOM (99% or more). Instead,
> > the first call to getauxval could do this, though, but I'm not sure
> > it's a good approach anyway. Linux has added the getrandom syscall
> > which can provide the BSD getentropy function or the more featureful
> > getrandom API, so using getauxval(AT_RANDOM) seems like a bad idea.
> > Even if we avoided reuse of the same data that went into the canary,
> > there's no way for callers using getauxval(AT_RANDOM) to tell whether
> > some other library code in the same process has already consumed
> > entropy from AT_RANDOM, so using it is not library-safe. It seems like
> > we should try to discourage use of getauxval(AT_RANDOM) as an entropy
> > source rather than giving false hope that it's safe.
> 
> getrandom(2) has the annoying problem that you can't ask it for
> best-effort entropy.  This caused systemd to add a /dev/urandom fallback
> a few days ago (sigh).

Is best-effort ever useful? My feeling is that either you need
cryptographic quality entropy, in which case it's not acceptable to
get something fake, or you don't, in which case you can use something
like the clock. Maybe I'm misunderstanding what you mean by
best-effort. My impression was that getrandom was equivalent to
/dev/urandom, not the tin-foil-hattery that is /dev/random.

> Maybe I'll try to get a GRND_BESTEFFORT flag for getrandom into the
> kernel.  I suppose that a musl getrandom wrapper could emulate that flag
> (only) or something on older kernels.  Or maybe glibc and musl could
> both agree to add some get_sort_of_decent_entropy function based on
> AT_RANDOM.

Really you can provide perfecty good random numbers for cryptographic
purposes with just AT_RANDOM as a seen and a proper csprng. My
understanding of the motivation for fancier stuff is a (misguided,
IMO) idea that sequences in the parent and child should be independent
after fork.

> >> If you think this is a good idea, I could implement it.  The main
> >> downside would be that it'll require some crypto primitive.  There's
> >> already a SHA-256 implementation in musl that could be reused, but it
> >> would be a bit unfortunate to pull it in to all musl-linked static binaries.
> > 
> > Yes, code size is a concern, but it could be tucked away as a
> > dependency of other functions instead of being a dependency of the
> > startup code.
> 
> Most or all existing getauxval users are unlikely to be using AT_RANDOM,
> so doing this without any bloat might be hard.

Yes, this is a good point.

Rich

Re: magic constants in some startup code

[Andy Lutomirski]

On 10/31/2014 02:39 PM, Rich Felker wrote:
> On Fri, Oct 31, 2014 at 02:29:32PM -0700, Andy Lutomirski wrote:
>>>> For example, musl could implement a trivial DRBG seeded by AT_RANDOM and
>>>> replace the AT_RANDOM data with the first output from the DRBG at
>>>> startup.  Then getauxval users are safe and musl can also have a stream
>>>> of decent random numbers for internal use.
>>>
>>> This imposes a large code size cost in the mandatory startup code even
>>> on programs that have no interest in AT_RANDOM (99% or more). Instead,
>>> the first call to getauxval could do this, though, but I'm not sure
>>> it's a good approach anyway. Linux has added the getrandom syscall
>>> which can provide the BSD getentropy function or the more featureful
>>> getrandom API, so using getauxval(AT_RANDOM) seems like a bad idea.
>>> Even if we avoided reuse of the same data that went into the canary,
>>> there's no way for callers using getauxval(AT_RANDOM) to tell whether
>>> some other library code in the same process has already consumed
>>> entropy from AT_RANDOM, so using it is not library-safe. It seems like
>>> we should try to discourage use of getauxval(AT_RANDOM) as an entropy
>>> source rather than giving false hope that it's safe.
>>
>> getrandom(2) has the annoying problem that you can't ask it for
>> best-effort entropy.  This caused systemd to add a /dev/urandom fallback
>> a few days ago (sigh).
> 
> Is best-effort ever useful? My feeling is that either you need
> cryptographic quality entropy, in which case it's not acceptable to
> get something fake, or you don't, in which case you can use something
> like the clock. Maybe I'm misunderstanding what you mean by
> best-effort. My impression was that getrandom was equivalent to
> /dev/urandom, not the tin-foil-hattery that is /dev/random.

The clock really sucks for entropy.  There are systems on which it's
entirely plausible that two different processes will start in rapid
succession and get exactly the same value out of the clock.

> 
>> Maybe I'll try to get a GRND_BESTEFFORT flag for getrandom into the
>> kernel.  I suppose that a musl getrandom wrapper could emulate that flag
>> (only) or something on older kernels.  Or maybe glibc and musl could
>> both agree to add some get_sort_of_decent_entropy function based on
>> AT_RANDOM.
> 
> Really you can provide perfecty good random numbers for cryptographic
> purposes with just AT_RANDOM as a seen and a proper csprng. My
> understanding of the motivation for fancier stuff is a (misguided,
> IMO) idea that sequences in the parent and child should be independent
> after fork.

The problems with AT_RANDOM and with getrandom(2) involve early boot.
Newer kernels (especially on ARM, apparently) can boot quickly enough
that the RNG is in terribly shape when userspace starts.  AT_RANDOM will
contain something, regardless, but it might have rather little entropy.
 getrandom(2) will refuse to operate at all until the kernel thinks it
has 128 bits or so of entropy.

So, if you want entropy at process start, AT_RANDOM is the best you can
do.  But you should seed a per-process csprng with it if you can avoid
it, or at least you should reseed with getrandom, since the kernel RNG
will eventually end up being cryptographically secure.

IOW, there isn't really a great solution here.

I am, however, quite convinced that different sequences after fork is
important.  Otherwise you can have catastrophic failures, e.g. if you do
fork(); compute_dsa_signature();

--Andy

> 
>>>> If you think this is a good idea, I could implement it.  The main
>>>> downside would be that it'll require some crypto primitive.  There's
>>>> already a SHA-256 implementation in musl that could be reused, but it
>>>> would be a bit unfortunate to pull it in to all musl-linked static binaries.
>>>
>>> Yes, code size is a concern, but it could be tucked away as a
>>> dependency of other functions instead of being a dependency of the
>>> startup code.
>>
>> Most or all existing getauxval users are unlikely to be using AT_RANDOM,
>> so doing this without any bloat might be hard.
> 
> Yes, this is a good point.
> 
> Rich
> 

Re: Re: magic constants in some startup code

[Rich Felker]

On Fri, Oct 31, 2014 at 03:27:37PM -0700, Andy Lutomirski wrote:
> > Is best-effort ever useful? My feeling is that either you need
> > cryptographic quality entropy, in which case it's not acceptable to
> > get something fake, or you don't, in which case you can use something
> > like the clock. Maybe I'm misunderstanding what you mean by
> > best-effort. My impression was that getrandom was equivalent to
> > /dev/urandom, not the tin-foil-hattery that is /dev/random.
> 
> The clock really sucks for entropy.  There are systems on which it's
> entirely plausible that two different processes will start in rapid
> succession and get exactly the same value out of the clock.

Are you sure? AFAIK Linux has at least microsecond resolution on all
major archs and nanosecond on x86 and some other important ones, and
fork takes over 1000ns and exec a lot more still. So I don't see how
you could get duplicates.

> >> Maybe I'll try to get a GRND_BESTEFFORT flag for getrandom into the
> >> kernel.  I suppose that a musl getrandom wrapper could emulate that flag
> >> (only) or something on older kernels.  Or maybe glibc and musl could
> >> both agree to add some get_sort_of_decent_entropy function based on
> >> AT_RANDOM.
> > 
> > Really you can provide perfecty good random numbers for cryptographic
> > purposes with just AT_RANDOM as a seen and a proper csprng. My
> > understanding of the motivation for fancier stuff is a (misguided,
> > IMO) idea that sequences in the parent and child should be independent
> > after fork.
> 
> The problems with AT_RANDOM and with getrandom(2) involve early boot.
> Newer kernels (especially on ARM, apparently) can boot quickly enough
> that the RNG is in terribly shape when userspace starts.  AT_RANDOM will
> contain something, regardless, but it might have rather little entropy.
>  getrandom(2) will refuse to operate at all until the kernel thinks it
> has 128 bits or so of entropy.
> 
> So, if you want entropy at process start, AT_RANDOM is the best you can
> do.  But you should seed a per-process csprng with it if you can avoid
> it, or at least you should reseed with getrandom, since the kernel RNG
> will eventually end up being cryptographically secure.
> 
> IOW, there isn't really a great solution here.

Well, this sounds like a good reason not to have code that depends on
entropy in pid 1.... :-)

Maybe there are situations here where you'd want best-effort, but I
can't think of any except initializing the stack protector canary
(which already uses AT_RANDOM) for init or other very-early processes
which should not be exposed to any input, much less untrusted input.
And on a machine with insufficient initial entropy, the first thing
the init sequence does should be getting the hardware to get you some
entropy, no?

> I am, however, quite convinced that different sequences after fork is
> important.  Otherwise you can have catastrophic failures, e.g. if you do
> fork(); compute_dsa_signature();

I agree that this is a catastrophic failure; I just disagree on the
cause. I think it's a bug to be doing anything after fork execept exec
(or something to prepare for exec). By keeping a copy of the parent's
address space while continuing to run, you expose yourself to all
sorts of possible information-leak issues. It's really hard to arrange
for the address space not to contain secrets at the time of fork. And
in multithreaded programs, it's simply UB to do anything but calling
async-signal-safe functions in the child after fork.

I'd still like to avoid duplicate sequences just from a standpoint of
hardening, but I the programs depending on this are already failing to
follow good practices.

Rich

Re: Re: magic constants in some startup code

[Szabolcs Nagy]

* Rich Felker <dalias@libc.org> [2014-10-31 19:14:58 -0400]:
> On Fri, Oct 31, 2014 at 03:27:37PM -0700, Andy Lutomirski wrote:
> > > Is best-effort ever useful? My feeling is that either you need
> > > cryptographic quality entropy, in which case it's not acceptable to
> > > get something fake, or you don't, in which case you can use something
> > > like the clock. Maybe I'm misunderstanding what you mean by
> > > best-effort. My impression was that getrandom was equivalent to
> > > /dev/urandom, not the tin-foil-hattery that is /dev/random.
> > 
> > The clock really sucks for entropy.  There are systems on which it's
> > entirely plausible that two different processes will start in rapid
> > succession and get exactly the same value out of the clock.
> 
> Are you sure? AFAIK Linux has at least microsecond resolution on all
> major archs and nanosecond on x86 and some other important ones, and
> fork takes over 1000ns and exec a lot more still. So I don't see how
> you could get duplicates.
> 

i'm interested in why clock might be bad source of entropy too
(it is not usable for key generation and there are some devices with
broken clock, but otherwise it should be a good source to avoid collisions
in some name space)

> > The problems with AT_RANDOM and with getrandom(2) involve early boot.
> > Newer kernels (especially on ARM, apparently) can boot quickly enough
> > that the RNG is in terribly shape when userspace starts.  AT_RANDOM will
> > contain something, regardless, but it might have rather little entropy.
> >  getrandom(2) will refuse to operate at all until the kernel thinks it
> > has 128 bits or so of entropy.
> > 
> > So, if you want entropy at process start, AT_RANDOM is the best you can
> > do.  But you should seed a per-process csprng with it if you can avoid
> > it, or at least you should reseed with getrandom, since the kernel RNG
> > will eventually end up being cryptographically secure.
> > 
> > IOW, there isn't really a great solution here.
> 
> Well, this sounds like a good reason not to have code that depends on
> entropy in pid 1.... :-)
> 
> Maybe there are situations here where you'd want best-effort, but I
> can't think of any except initializing the stack protector canary
> (which already uses AT_RANDOM) for init or other very-early processes
> which should not be exposed to any input, much less untrusted input.
> And on a machine with insufficient initial entropy, the first thing
> the init sequence does should be getting the hardware to get you some
> entropy, no?
> 

if early boot here means init then i dont think it's reasonable to assume
good entropy at that point

anyway entropy quality should not be libc resposibility at all, libc uses
whatever the kernel gives

it used to be infeasible to get new entropy from the kernel on demand
without introducing difficult to handle failure paths, but in new kernels
eg the temp file name generation could easily use getrandom and then the
quailty is kernel side issue

Re: magic constants in some startup code

[Andy Lutomirski]

On 11/02/2014 09:17 AM, Szabolcs Nagy wrote:
> * Rich Felker <dalias@libc.org> [2014-10-31 19:14:58 -0400]:
>> On Fri, Oct 31, 2014 at 03:27:37PM -0700, Andy Lutomirski wrote:
>>>> Is best-effort ever useful? My feeling is that either you need
>>>> cryptographic quality entropy, in which case it's not acceptable to
>>>> get something fake, or you don't, in which case you can use something
>>>> like the clock. Maybe I'm misunderstanding what you mean by
>>>> best-effort. My impression was that getrandom was equivalent to
>>>> /dev/urandom, not the tin-foil-hattery that is /dev/random.
>>>
>>> The clock really sucks for entropy.  There are systems on which it's
>>> entirely plausible that two different processes will start in rapid
>>> succession and get exactly the same value out of the clock.
>>
>> Are you sure? AFAIK Linux has at least microsecond resolution on all
>> major archs and nanosecond on x86 and some other important ones, and
>> fork takes over 1000ns and exec a lot more still. So I don't see how
>> you could get duplicates.
>>
> 
> i'm interested in why clock might be bad source of entropy too
> (it is not usable for key generation and there are some devices with
> broken clock, but otherwise it should be a good source to avoid collisions
> in some name space)

I don't know the details, but I've heard of this being a real problem on
MIPS routers, for example.

> 
>>> The problems with AT_RANDOM and with getrandom(2) involve early boot.
>>> Newer kernels (especially on ARM, apparently) can boot quickly enough
>>> that the RNG is in terribly shape when userspace starts.  AT_RANDOM will
>>> contain something, regardless, but it might have rather little entropy.
>>>  getrandom(2) will refuse to operate at all until the kernel thinks it
>>> has 128 bits or so of entropy.
>>>
>>> So, if you want entropy at process start, AT_RANDOM is the best you can
>>> do.  But you should seed a per-process csprng with it if you can avoid
>>> it, or at least you should reseed with getrandom, since the kernel RNG
>>> will eventually end up being cryptographically secure.
>>>
>>> IOW, there isn't really a great solution here.
>>
>> Well, this sounds like a good reason not to have code that depends on
>> entropy in pid 1.... :-)
>>
>> Maybe there are situations here where you'd want best-effort, but I
>> can't think of any except initializing the stack protector canary
>> (which already uses AT_RANDOM) for init or other very-early processes
>> which should not be exposed to any input, much less untrusted input.
>> And on a machine with insufficient initial entropy, the first thing
>> the init sequence does should be getting the hardware to get you some
>> entropy, no?
>>
> 
> if early boot here means init then i dont think it's reasonable to assume
> good entropy at that point
> 
> anyway entropy quality should not be libc resposibility at all, libc uses
> whatever the kernel gives
> 
> it used to be infeasible to get new entropy from the kernel on demand
> without introducing difficult to handle failure paths, but in new kernels
> eg the temp file name generation could easily use getrandom and then the
> quailty is kernel side issue
> 

Actually, temp file name generation is probably a decent example of
wanting best effort.  And getrandom either blocks or fails in very early
boot.

--Andy