Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Jon Oster]

In 5.8.1, $RANDOM seems not to update its state when the command $RANDOM
was used in is piped somewhere. For example:

# echo $RANDOM
18163
# echo $RANDOM
23472
# echo $RANDOM
27685
# echo $RANDOM|cat
10178
# echo $RANDOM|cat
10178
# echo $RANDOM
10178
# echo $RANDOM
17862

This is new behaviour since I upgraded from 5.4.2 to 5.8.1. At first I
thought that it might be somehow related to multios, but the result is
the same with multios on or off. So, I did a bisect, and discovered that
faf0035e532cde45528806e7a05ad28a0ab7c0fb[1] is the commit that introduced
this behaviour.

Is this considered a bug? Or is it intended behaviour? Personally, I'd
argue it's a bug. Unfortunately, my C isn't good enough to try to fix and
submit a patch myself, but I tried to make this bug report as helpful as
possible.

[1] https://sourceforge.net/p/zsh/code/ci/faf0035e532cde45528806e7a05ad28a0ab7c0fb/

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Eric Cook]

On 6/28/23 16:10, Jon Oster wrote:
> In 5.8.1, $RANDOM seems not to update its state when the command $RANDOM
> was used in is piped somewhere. For example:
>
> Is this considered a bug? Or is it intended behaviour? Personally, I'd
> argue it's a bug. Unfortunately, my C isn't good enough to try to fix and
> submit a patch myself, but I tried to make this bug report as helpful as
> possible.
>
> [1] https://sourceforge.net/p/zsh/code/ci/faf0035e532cde45528806e7a05ad28a0ab7c0fb/


RANDOM <S>
        A  pseudo-random integer from 0 to 32767, newly generated each time this parame‐
        ter is referenced.  The random number generator can be seeded by assigning a nu‐
        meric value to RANDOM.

        The  values  of  RANDOM form an intentionally-repeatable pseudo-random sequence;
        subshells that reference RANDOM will result in  identical  pseudo-random  values
        unless  the  value  of RANDOM is referenced or seeded in the parent shell in be‐
        tween subshell invocations.


The LHS of the pipe should be a subshell, so the behavior is intentional.
RANDOM has be documented to work that way since it's introduction and this is commonly
misreported as a bug.

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Bart Schaefer]

On Wed, Jun 28, 2023 at 7:46 PM Eric Cook <llua@gmx.com> wrote:
>
> On 6/28/23 16:10, Jon Oster wrote:
> > In 5.8.1, $RANDOM seems not to update its state when the command $RANDOM
> > was used in is piped somewhere. For example:
> >
> > Is this considered a bug? Or is it intended behaviour?
> >
> > [1] https://sourceforge.net/p/zsh/code/ci/faf0035e532cde45528806e7a05ad28a0ab7c0fb/
>
> The LHS of the pipe should be a subshell, so the behavior is intentional.

This was, however, an unanticipated side-effect of the referenced
commit.  It changed behavior (that, before, did not conform to the
documentation) without that fact being called out in any of our files
that serve as release notes.

It opens the question of whether anything else that previously was
"pre-fork" might have changed behavior at that point.

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Ray Andrews]

On 2023-06-28 22:42, Bart Schaefer wrote:
> The LHS of the pipe should be a subshell, so the behavior is intentional.
> This was, however, an unanticipated side-effect of the referenced
> commit.

So it feels like a bug but is theoretically correct even tho it was 
'fixed' by accident?  I'd speculate that nine out of ten people would 
prefer the 'bug' since it's the intuitive behavior. If one wanted to 
preserve a value, would one not do that explicitly?  Just my two cents.

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Roman Perepelitsa]

On Thu, Jun 29, 2023 at 7:43 AM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Wed, Jun 28, 2023 at 7:46 PM Eric Cook <llua@gmx.com> wrote:
> >
> > On 6/28/23 16:10, Jon Oster wrote:
> > > In 5.8.1, $RANDOM seems not to update its state when the command $RANDOM
> > > was used in is piped somewhere. For example:
> > >
> > > Is this considered a bug? Or is it intended behaviour?
> > >
> > > [1] https://sourceforge.net/p/zsh/code/ci/faf0035e532cde45528806e7a05ad28a0ab7c0fb/
> >
> > The LHS of the pipe should be a subshell, so the behavior is intentional.
>
> This was, however, an unanticipated side-effect of the referenced
> commit.  It changed behavior (that, before, did not conform to the
> documentation) without that fact being called out in any of our files
> that serve as release notes.
>
> It opens the question of whether anything else that previously was
> "pre-fork" might have changed behavior at that point.

Perhaps things of this sort?

    % echo ${foo::=bar} | cat
    % zmodload zsh/system; echo $sysparams[pid] | cat

I cannot reproduce the 5.4.2 behavior reported by Jon, so I cannot
verify whether the identified commit affects these test cases.

    % docker run --rm zshusers/zsh:5.4.2 zsh -c 'repeat 2 echo $RANDOM | cat'
    15988
    9706

Roman.

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Mikael Magnusson]

On 6/29/23, Roman Perepelitsa <roman.perepelitsa@gmail.com> wrote:
> On Thu, Jun 29, 2023 at 7:43 AM Bart Schaefer <schaefer@brasslantern.com>
> wrote:
>>
>> On Wed, Jun 28, 2023 at 7:46 PM Eric Cook <llua@gmx.com> wrote:
>> >
>> > On 6/28/23 16:10, Jon Oster wrote:
>> > > In 5.8.1, $RANDOM seems not to update its state when the command
>> > > $RANDOM
>> > > was used in is piped somewhere. For example:
>> > >
>> > > Is this considered a bug? Or is it intended behaviour?
>> > >
>> > > [1]
>> > > https://sourceforge.net/p/zsh/code/ci/faf0035e532cde45528806e7a05ad28a0ab7c0fb/
>> >
>> > The LHS of the pipe should be a subshell, so the behavior is
>> > intentional.
>>
>> This was, however, an unanticipated side-effect of the referenced
>> commit.  It changed behavior (that, before, did not conform to the
>> documentation) without that fact being called out in any of our files
>> that serve as release notes.
>>
>> It opens the question of whether anything else that previously was
>> "pre-fork" might have changed behavior at that point.
>
> Perhaps things of this sort?
>
>     % echo ${foo::=bar} | cat
>     % zmodload zsh/system; echo $sysparams[pid] | cat
>
> I cannot reproduce the 5.4.2 behavior reported by Jon, so I cannot
> verify whether the identified commit affects these test cases.
>
>     % docker run --rm zshusers/zsh:5.4.2 zsh -c 'repeat 2 echo $RANDOM |
> cat'
>     15988
>     9706

Yes, you can? The above command will output the same two numbers on
newer versions:
% zsh -c 'repeat 2; do echo $RANDOM | cat; done'
13741
13741
% zsh-5.4.2-dev-0-mika -c 'repeat 2; do echo $RANDOM | cat; done'
4521
27867

-- 
Mikael Magnusson

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Roman Perepelitsa]

On Thu, Jun 29, 2023 at 7:53 PM Mikael Magnusson <mikachu@gmail.com> wrote:
>
> On 6/29/23, Roman Perepelitsa <roman.perepelitsa@gmail.com> wrote:
> > On Thu, Jun 29, 2023 at 7:43 AM Bart Schaefer <schaefer@brasslantern.com>
> > wrote:
> >>
> >> On Wed, Jun 28, 2023 at 7:46 PM Eric Cook <llua@gmx.com> wrote:
> >> >
> >> > On 6/28/23 16:10, Jon Oster wrote:
> >> > > In 5.8.1, $RANDOM seems not to update its state when the command
> >> > > $RANDOM
> >> > > was used in is piped somewhere. For example:
> >> > >
> >> > > Is this considered a bug? Or is it intended behaviour?
> >> > >
> >> > > [1]
> >> > > https://sourceforge.net/p/zsh/code/ci/faf0035e532cde45528806e7a05ad28a0ab7c0fb/
> >> >
> >> > The LHS of the pipe should be a subshell, so the behavior is
> >> > intentional.
> >>
> >> This was, however, an unanticipated side-effect of the referenced
> >> commit.  It changed behavior (that, before, did not conform to the
> >> documentation) without that fact being called out in any of our files
> >> that serve as release notes.
> >>
> >> It opens the question of whether anything else that previously was
> >> "pre-fork" might have changed behavior at that point.
> >
> > Perhaps things of this sort?
> >
> >     % echo ${foo::=bar} | cat
> >     % zmodload zsh/system; echo $sysparams[pid] | cat
> >
> > I cannot reproduce the 5.4.2 behavior reported by Jon, so I cannot
> > verify whether the identified commit affects these test cases.
> >
> >     % docker run --rm zshusers/zsh:5.4.2 zsh -c 'repeat 2 echo $RANDOM |
> > cat'
> >     15988
> >     9706
>
> Yes, you can?

Of course!

I can also confirm that the two test cases I posted above behave
differently in zsh 5.4.2 and 5.9. In all these cases 5.9 behaves as I
would expect, and the behavior of 5.4.2 is surprising to me.

Roman.

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Bart Schaefer]

On Thu, Jun 29, 2023 at 11:16 AM Roman Perepelitsa
<roman.perepelitsa@gmail.com> wrote:
>
> > On 6/29/23, Roman Perepelitsa <roman.perepelitsa@gmail.com> wrote:
> > >
> > >     % echo ${foo::=bar} | cat
> > >     % zmodload zsh/system; echo $sysparams[pid] | cat
> > >
>
> I can also confirm that the two test cases I posted above behave
> differently in zsh 5.4.2 and 5.9. In all these cases 5.9 behaves as I
> would expect, and the behavior of 5.4.2 is surprising to me.

The change that had this effect dates from May 2018 and has been "in
production" since the 5.6 release in September 2018, so I think it's
reasonable to say that it hasn't had any significant impact on usage
-- MacOS Catalina has zsh 5.7, Ubuntu 20.04 has 5.8 -- but it's
probably worth mentioning somewhere.

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Mikael Magnusson]

On 6/28/23, Jon Oster <jon.i.oster@gmail.com> wrote:
> In 5.8.1, $RANDOM seems not to update its state when the command $RANDOM
> was used in is piped somewhere. For example:
>
> # echo $RANDOM
> 18163
> # echo $RANDOM
> 23472
> # echo $RANDOM
> 27685
> # echo $RANDOM|cat
> 10178
> # echo $RANDOM|cat
> 10178
> # echo $RANDOM
> 10178
> # echo $RANDOM
> 17862

A simple workaround for this, whether you consider it a bug or a feature,
% () { echo $1 | cat } $RANDOM
21589
% () { echo $1 | cat } $RANDOM
301

or
% echo $RANDOM > >(cat)
1324
% echo $RANDOM > >(cat)
9460

-- 
Mikael Magnusson

Re: Bug/regression: tt(RANDOM) seed not updating when $RANDOM is used in a pipe

[Roman Perepelitsa]

On Sun, Jul 2, 2023 at 9:19 AM Mikael Magnusson <mikachu@gmail.com> wrote:
>
> On 6/28/23, Jon Oster <jon.i.oster@gmail.com> wrote:
> > In 5.8.1, $RANDOM seems not to update its state when the command $RANDOM
> > was used in is piped somewhere. For example:
> >
> > # echo $RANDOM
> > 18163
> > # echo $RANDOM
> > 23472
> > # echo $RANDOM
> > 27685
> > # echo $RANDOM|cat
> > 10178
> > # echo $RANDOM|cat
> > 10178
> > # echo $RANDOM
> > 10178
> > # echo $RANDOM
> > 17862
>
> A simple workaround for this, whether you consider it a bug or a feature,
> % () { echo $1 | cat } $RANDOM
> 21589
> % () { echo $1 | cat } $RANDOM
> 301
>
> or
> % echo $RANDOM > >(cat)
> 1324
> % echo $RANDOM > >(cat)
> 9460

An alternative workaround is to use a random number generator whose
state is kept outside of the current process. Here's an example:
https://gist.github.com/romkatv/a6cede40714ec77d4da73605c5ddb36a. An
added advantage of this approach is that you get a 32-bit number
rather than the 15-bit provided by $RANDOM.

Roman.