use of rand() in zsh

use of rand() in zsh

[Clinton Bunch]

Is there a reason that zsh goes out of the way (srand_deterministic) to 
use the least random "random" function on modern Unix?  I would think a 
zrandom function that used getrandom() or arc4random() or /dev/urandom 
would serve better. But before I write one, I wanted to know if there 
was a reason that isn't obvious to me.


(I have an idea for a new module builtin that solved the mktemp race in 
zsh scripts and it should use a better random source than rand())

Re: use of rand() in zsh

[Bart Schaefer]

On Mon, Oct 10, 2022 at 4:38 PM Clinton Bunch <cdb_zsh@zentaur.org> wrote:
>
> Is there a reason that zsh goes out of the way (srand_deterministic) to
> use the least random "random" function on modern Unix?

     The values of RANDOM form an intentionally-repeatable pseudo-random
     sequence; subshells that reference RANDOM will result in identical
     pseudo-random values unless the value of RANDOM is referenced or
     seeded in the parent shell in between subshell invocations.

This is e.g. so you aren't surprised by referencing $RANDOM inside
$(...) and getting a different result.

Re: use of rand() in zsh

[Clinton Bunch]

On 10/10/2022 8:14 PM, Bart Schaefer wrote:
> On Mon, Oct 10, 2022 at 4:38 PM Clinton Bunch <cdb_zsh@zentaur.org> wrote:
>> Is there a reason that zsh goes out of the way (srand_deterministic) to
>> use the least random "random" function on modern Unix?
>       The values of RANDOM form an intentionally-repeatable pseudo-random
>       sequence; subshells that reference RANDOM will result in identical
>       pseudo-random values unless the value of RANDOM is referenced or
>       seeded in the parent shell in between subshell invocations.
>
> This is e.g. so you aren't surprised by referencing $RANDOM inside
> $(...) and getting a different result.

Well, I guess that explains why it uses rand, but doesn't really explain 
to me why you'd *want* this behavior.  I would expect to get a different 
value in a subshell than I got in the parent shell.  I'm supposedly 
looking for random numbers.

  I guess I side with OpenBSD who changed the default behavior of rand().

Re: use of rand() in zsh

[Clinton Bunch]

On 10/10/2022 8:27 PM, Clinton Bunch wrote:
> On 10/10/2022 8:14 PM, Bart Schaefer wrote:
>> On Mon, Oct 10, 2022 at 4:38 PM Clinton Bunch <cdb_zsh@zentaur.org> 
>> wrote:
>>> Is there a reason that zsh goes out of the way (srand_deterministic) to
>>> use the least random "random" function on modern Unix?
>>       The values of RANDOM form an intentionally-repeatable 
>> pseudo-random
>>       sequence; subshells that reference RANDOM will result in identical
>>       pseudo-random values unless the value of RANDOM is referenced or
>>       seeded in the parent shell in between subshell invocations.
>>
>> This is e.g. so you aren't surprised by referencing $RANDOM inside
>> $(...) and getting a different result.
>
> Well, I guess that explains why it uses rand, but doesn't really 
> explain to me why you'd *want* this behavior.  I would expect to get a 
> different value in a subshell than I got in the parent shell.  I'm 
> supposedly looking for random numbers.
>
>  I guess I side with OpenBSD who changed the default behavior of rand().
>
>
I wonder if it wouldn't be worthwhile to create a module that gave 
access to better sources of random numbers on the systems that support them.

Re: use of rand() in zsh

[Bart Schaefer]

On Mon, Oct 10, 2022 at 6:32 PM Clinton Bunch <cdbunch@zentaur.org> wrote:
>
> I wonder if it wouldn't be worthwhile to create a module that gave
> access to better sources of random numbers on the systems that support them.

zmodload zsh/mathfunc
print $(( rand48() ))

Re: use of rand() in zsh

[Clinton Bunch]

On 10/10/2022 10:34 PM, Bart Schaefer wrote:
> On Mon, Oct 10, 2022 at 6:32 PM Clinton Bunch <cdbunch@zentaur.org> wrote:
>> I wonder if it wouldn't be worthwhile to create a module that gave
>> access to better sources of random numbers on the systems that support them.
> zmodload zsh/mathfunc
> print $(( rand48() ))

Yeah, I know about  e/rand48 but that's only slightly better than rand().

I was suggesting something that accessed the kernel entropy pool like 
bash's SRANDOM variable.    I know not every Unix has such a thing, but 
it would be useful on those that do.

Re: use of rand() in zsh

[Roman Perepelitsa]

On Tue, Oct 11, 2022 at 5:44 AM Clinton Bunch <cdb_zsh@zentaur.org> wrote:
>
> I was suggesting something that accessed the kernel entropy pool like
> bash's SRANDOM variable.

I would use it, and I think most uses of $RANDOM would be better off
as $SRANDOM.

In the meantime I'm using this:
https://gist.github.com/romkatv/a6cede40714ec77d4da73605c5ddb36a

Roman.

Re: use of rand() in zsh

[Clinton Bunch]

On 10/11/2022 1:31 AM, Roman Perepelitsa wrote:
>
> I would use it, and I think most uses of $RANDOM would be better off
> as $SRANDOM.
>
> In the meantime I'm using this:
> https://gist.github.com/romkatv/a6cede40714ec77d4da73605c5ddb36a
>
> Roman.
>
The question then is how should it be implemented:

1. Implement a shopt that changed the behavior of RANDOM (Pleasing 
because of the elegance of using RANDOM for good random values, but 
probably too confusing)

2. Simply implement a new parameter called SRANDOM (simplest, but change 
to core zsh interface)

3. Write a module (e.g. zsh/random) that introduced a new special 
parameter when loaded (more complicated, but isolated to those who want 
it and possible to fail on systems that don't have a kernel entropy pool)

Re: use of rand() in zsh

[Peter Stephenson]

> On 11/10/2022 13:51 Clinton Bunch <cdb_zsh@zentaur.org> wrote:
> 1. Implement a shopt that changed the behavior of RANDOM (Pleasing 
> because of the elegance of using RANDOM for good random values, but 
> probably too confusing)
> 
> 2. Simply implement a new parameter called SRANDOM (simplest, but change 
> to core zsh interface)
> 
> 3. Write a module (e.g. zsh/random) that introduced a new special 
> parameter when loaded (more complicated, but isolated to those who want 
> it and possible to fail on systems that don't have a kernel entropy pool)

An autoloadable parameter gives you something that works basically as if
it's part of the shell but can be turned off as well as giving an error
if the module isn't available, so might be the best bet.  So the
.mdd file would have

autofeatures=p:SRANDOM

pws

Re: use of rand() in zsh

[Clinton Bunch]

On 10/11/2022 8:23 AM, Peter Stephenson wrote:
>> On 11/10/2022 13:51 Clinton Bunch <cdb_zsh@zentaur.org> wrote:
>>
>> 3. Write a module (e.g. zsh/random) that introduced a new special
>> parameter when loaded (more complicated, but isolated to those who want
>> it and possible to fail on systems that don't have a kernel entropy pool)
> An autoloadable parameter gives you something that works basically as if
> it's part of the shell but can be turned off as well as giving an error
> if the module isn't available, so might be the best bet.  So the
> .mdd file would have
>
> autofeatures=p:SRANDOM
>
> pws
>
That's a neat trick, but as someone who has had to hack to build 
software built and primarily tested on Linux, on a Unix system stuck in 
the '00's (if not the last century), I'd rather have an explicit 
load/test for the existence of the module than have it blow up on a 
system where SRANDOM didn't work. (8 years as an HP-UX admin)

My $.02 for whatever that's worth. :)

Re: use of rand() in zsh

[Peter Stephenson]

> On 11/10/2022 15:15 Clinton Bunch <cdbunch@zentaur.org> wrote:
> 
>  
> On 10/11/2022 8:23 AM, Peter Stephenson wrote:
> >> On 11/10/2022 13:51 Clinton Bunch <cdb_zsh@zentaur.org> wrote:
> >>
> >> 3. Write a module (e.g. zsh/random) that introduced a new special
> >> parameter when loaded (more complicated, but isolated to those who want
> >> it and possible to fail on systems that don't have a kernel entropy pool)
> > An autoloadable parameter gives you something that works basically as if
> > it's part of the shell but can be turned off as well as giving an error
> > if the module isn't available, so might be the best bet.  So the
> > .mdd file would have
> >
> > autofeatures=p:SRANDOM
> >
> > pws
> >
> That's a neat trick, but as someone who has had to hack to build 
> software built and primarily tested on Linux, on a Unix system stuck in 
> the '00's (if not the last century), I'd rather have an explicit 
> load/test for the existence of the module than have it blow up on a 
> system where SRANDOM didn't work. (8 years as an HP-UX admin)
> 
> My $.02 for whatever that's worth. :)

You can do that too: the autoload only triggers if you refer to
$SRANDOM first, at which point the module will fail to load ---
quite possibly in an inconvenient location.  But if you write
your code to do the load explicitly it will check at that point.
So you should have the best of all worlds.

pws

Re: use of rand() in zsh

[Clinton Bunch]

On 10/10/2022 8:14 PM, Bart Schaefer wrote:
> On Mon, Oct 10, 2022 at 4:38 PM Clinton Bunch <cdb_zsh@zentaur.org> wrote:
>> Is there a reason that zsh goes out of the way (srand_deterministic) to
>> use the least random "random" function on modern Unix?
>       The values of RANDOM form an intentionally-repeatable pseudo-random
>       sequence; subshells that reference RANDOM will result in identical
>       pseudo-random values unless the value of RANDOM is referenced or
>       seeded in the parent shell in between subshell invocations.
>
> This is e.g. so you aren't surprised by referencing $RANDOM inside
> $(...) and getting a different result.
>
Doesn't zsleep_random's use of rand() break this predictability, or have 
I just not studied the use of zsleep_random enough?

Re: use of rand() in zsh

[Peter Stephenson]

> On 11/10/2022 13:35 Clinton Bunch <cdb_zsh@zentaur.org> wrote:
> On 10/10/2022 8:14 PM, Bart Schaefer wrote:
> > On Mon, Oct 10, 2022 at 4:38 PM Clinton Bunch <cdb_zsh@zentaur.org> wrote:
> >> Is there a reason that zsh goes out of the way (srand_deterministic) to
> >> use the least random "random" function on modern Unix?
> >
> > This is e.g. so you aren't surprised by referencing $RANDOM inside
> > $(...) and getting a different result.
> >
> Doesn't zsleep_random's use of rand() break this predictability, or have 
> I just not studied the use of zsleep_random enough?

Yes, looks like any locking of history files will disturb the sequence.

That could be fairly easily fixed on a modern system by using rand_r here
--- history locking doesn't need high quality randomness, just
some sort of variation, so probably isn't worth reimplementing fully.

pws