zsh-users
 help / color / mirror / code / Atom feed
* shuffle array
@ 2019-11-29  3:20 Emanuel Berg
  2019-11-29  6:42 ` Marc Chantreux
  2019-11-29  8:44 ` Emanuel Berg
  0 siblings, 2 replies; 15+ messages in thread
From: Emanuel Berg @ 2019-11-29  3:20 UTC (permalink / raw)
  To: zsh-users

How do I shuffle an array?

I found [1] but it seems like a lot of code?
Not that that's a problem if that's the way
it is.


[1] http://www.codecodex.com/wiki/index.php?title=Shuffle_an_array&oldid=7474
-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-11-29  3:20 shuffle array Emanuel Berg
@ 2019-11-29  6:42 ` Marc Chantreux
  2019-11-29  8:19   ` Emanuel Berg
       [not found]   ` <8636e7w0w9.fsf__36104.0529723809$1575015640$gmane$org@zoho.eu>
  2019-11-29  8:44 ` Emanuel Berg
  1 sibling, 2 replies; 15+ messages in thread
From: Marc Chantreux @ 2019-11-29  6:42 UTC (permalink / raw)
  To: zsh-users

hello,

if you don't mind using a filter from GNU coreutils

print -l $array |shuff|read -A array

regards
marc

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-11-29  6:42 ` Marc Chantreux
@ 2019-11-29  8:19   ` Emanuel Berg
  2019-11-29  8:43     ` Marc Chantreux
       [not found]   ` <8636e7w0w9.fsf__36104.0529723809$1575015640$gmane$org@zoho.eu>
  1 sibling, 1 reply; 15+ messages in thread
From: Emanuel Berg @ 2019-11-29  8:19 UTC (permalink / raw)
  To: zsh-users

Marc Chantreux wrote:

> print -l $array |shuff|read -A array

*shuf

The first two steps work, but the result array
seems to have only one element?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-11-29  8:19   ` Emanuel Berg
@ 2019-11-29  8:43     ` Marc Chantreux
  2019-11-29  8:50       ` Emanuel Berg
  0 siblings, 1 reply; 15+ messages in thread
From: Marc Chantreux @ 2019-11-29  8:43 UTC (permalink / raw)
  To: zsh-users


> > print -l $array |shuff|read -A array

> The first two steps work, but the result array
> seems to have only one element?

those filters shuffle lines so you need to print 1 element per line
(-l).

can you show us examples?

regards
marc

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-11-29  3:20 shuffle array Emanuel Berg
  2019-11-29  6:42 ` Marc Chantreux
@ 2019-11-29  8:44 ` Emanuel Berg
  1 sibling, 0 replies; 15+ messages in thread
From: Emanuel Berg @ 2019-11-29  8:44 UTC (permalink / raw)
  To: zsh-users

This seems to work, I got help on
#zsh@irc.freenode.net, then rewrote it a bit as
I'm not familiar with the 'typeset' and
'integer' commands, also the parenthesis syntax
is unknown to me... but anyway, source at

    https://dataswamp.org/~incal/conf/.zsh/misc

Relevant part yanked:

# from: http://mika.l3ib.org/code/zsh-functions/shufflearray
swap-array () {
    zmodload zsh/mathfunc

    local -a reply
    reply=($@)

    local -a swap

    local n=$#reply
    local k

    for ((; n > 0; n-- )); do
        k=$((1 + int(rand48()*n)))
        swap=$reply[k]
        reply[k]=$reply[n]
        reply[n]=$swap
    done

    echo $reply
}

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-11-29  8:43     ` Marc Chantreux
@ 2019-11-29  8:50       ` Emanuel Berg
  2019-11-29 13:38         ` Clint Adams
  0 siblings, 1 reply; 15+ messages in thread
From: Emanuel Berg @ 2019-11-29  8:50 UTC (permalink / raw)
  To: zsh-users

Marc Chantreux wrote:

>>> print -l $array |shuff|read -A array
>>
>> The first two steps work, but the result
>> array seems to have only one element?
>
> those filters shuffle lines so you need to
> print 1 element per line (-l).
>
> can you show us examples?

$ local -a arg_array
$ arg_array=(1 2 3 a b c)
$ print -l $arg_array | shuf
3
b
2
c
a
1

works!

$ local -a arg_array
$ arg_array=(1 2 3 a b c)
$ print -l $arg_array | shuf | read -A new_array
$ echo $new_array
1

doesn't work, just one element...

$ echo $#new_array
1
$ echo $#arg_array
6

...

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-11-29  8:50       ` Emanuel Berg
@ 2019-11-29 13:38         ` Clint Adams
  0 siblings, 0 replies; 15+ messages in thread
From: Clint Adams @ 2019-11-29 13:38 UTC (permalink / raw)
  To: zsh-users

On Fri, Nov 29, 2019 at 09:50:12AM +0100, Emanuel Berg wrote:
> $ local -a arg_array
> $ arg_array=(1 2 3 a b c)
> $ print -l $arg_array | shuf | read -A new_array
> $ echo $new_array
> 1
> 
> doesn't work, just one element...

`read` only reads one line.

I don't remember where this came from, but I've been using

shuffle() {
  declare -A h
  local +h -Z 5 RANDOM=$EPOCHSECONDS
  integer i
  for ((i=1; i <= $#; ++i)) { h[$i.$RANDOM]=$argv[i] }
  reply=( $h )
}

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
       [not found]   ` <8636e7w0w9.fsf__36104.0529723809$1575015640$gmane$org@zoho.eu>
@ 2019-12-01 20:07     ` Stephane Chazelas
  2019-12-01 21:22       ` Emanuel Berg
                         ` (2 more replies)
  0 siblings, 3 replies; 15+ messages in thread
From: Stephane Chazelas @ 2019-12-01 20:07 UTC (permalink / raw)
  To: zsh-users

2019-11-29 09:19:02 +0100, Emanuel Berg:
> Marc Chantreux wrote:
> 
> > print -l $array |shuff|read -A array
> 
> *shuf
> 
> The first two steps work

The first two steps (with shuf instead of shuff)  only work if
array elements don't contain backslashes nor newline and are not
empty and the first one doesn't start with "-".

> but the result array
> seems to have only one element?
[...]

Yes, that's not what read -A does.

You could do things like:

array=("${(0@)$(printf '%s\0' "$array[@]" | shuf -z)[1,-2]}")

But that would still not work for array elements containing
NULs.

-- 
Stephane

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-12-01 20:07     ` Stephane Chazelas
@ 2019-12-01 21:22       ` Emanuel Berg
  2019-12-01 21:25         ` Emanuel Berg
  2019-12-01 23:46         ` Daniel Shahaf
  2019-12-01 23:38       ` Daniel Shahaf
       [not found]       ` <20191201233848.7selcpkvenlu65px__33142.5388505281$1575243619$gmane$org@tarpaulin.shahaf.local2>
  2 siblings, 2 replies; 15+ messages in thread
From: Emanuel Berg @ 2019-12-01 21:22 UTC (permalink / raw)
  To: zsh-users

Stephane Chazelas wrote:

> You could do things like:
>
> array=("${(0@)$(printf '%s\0' "$array[@]" | shuf -z)[1,-2]}")

It seems I always get the same shuffle
for that?

(1 2 3 a b c) -> (c 1 a 3 b 2)

and

(x y z 1 2 3) -> (3 x 1 z 2 y)

- the same.

Another thing, how do I use this to shuffle an
array from another function? How do I send the
array as a name/pointer argument, i.e. not just
the value? Or how can I return the new array
and set the old one's value to that?

# from SC on gmane.comp.shells.zsh.user
shuffle-array-2 () {
    local -a array
    array=($@)
    array=("${(0@)$(printf '%s\0' "$array[@]" | shuf -z)[1,-2]}")
    echo $array
}

https://dataswamp.org/~incal/conf/.zsh/shuffle

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-12-01 21:22       ` Emanuel Berg
@ 2019-12-01 21:25         ` Emanuel Berg
  2019-12-01 23:46         ` Daniel Shahaf
  1 sibling, 0 replies; 15+ messages in thread
From: Emanuel Berg @ 2019-12-01 21:25 UTC (permalink / raw)
  To: zsh-users

>> You could do things like:
>>
>> array=("${(0@)$(printf '%s\0' "$array[@]" | shuf -z)[1,-2]}")
>
> It seems I always get the same shuffle
> for that?
>
> (1 2 3 a b c) -> (c 1 a 3 b 2)
>
> and
>
> (x y z 1 2 3) -> (3 x 1 z 2 y)
>
> - the same.

Sorry, I had a typo, as for the shuffling, it
works fine.

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-12-01 20:07     ` Stephane Chazelas
  2019-12-01 21:22       ` Emanuel Berg
@ 2019-12-01 23:38       ` Daniel Shahaf
       [not found]       ` <20191201233848.7selcpkvenlu65px__33142.5388505281$1575243619$gmane$org@tarpaulin.shahaf.local2>
  2 siblings, 0 replies; 15+ messages in thread
From: Daniel Shahaf @ 2019-12-01 23:38 UTC (permalink / raw)
  To: zsh-users

Stephane Chazelas wrote on Sun, Dec 01, 2019 at 20:07:19 +0000:
> array=("${(0@)$(printf '%s\0' "$array[@]" | shuf -z)[1,-2]}")
> 
> But that would still not work for array elements containing
> NULs.

I think the following would work? —

    typeset -a l=(…)
    eval "l=(" $(print -rl -- "${(@qqqq)l}" | shuf) ")"

shuf(1) expects newline-separated output, and the output of ${(qqqq)} never
has literal newlines.

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-12-01 21:22       ` Emanuel Berg
  2019-12-01 21:25         ` Emanuel Berg
@ 2019-12-01 23:46         ` Daniel Shahaf
  2019-12-02  4:10           ` Emanuel Berg
  1 sibling, 1 reply; 15+ messages in thread
From: Daniel Shahaf @ 2019-12-01 23:46 UTC (permalink / raw)
  To: zsh-users

Emanuel Berg wrote on Sun, Dec 01, 2019 at 22:22:27 +0100:
> How do I send the array as a name/pointer argument, i.e. not just the value?

To read an array by name:

f() {
  local array_name=$1
  print -rl -- "${(@P)array_name}"
}
typeset -a foo=(bar baz)
f foo

To write an array by name:

f() { set -A "$1" bar baz }
typeset -a foo
f foo

> Or how can I return the new array and set the old one's value to that?

f() { reply=( bar baz ) }
typeset -a foo 
f
foo=( "${reply[@]}" )

> # from SC on gmane.comp.shells.zsh.user

gmane.comp.shells.zsh.user and zsh-users@zsh.org are the same thing. ☺

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-12-01 23:46         ` Daniel Shahaf
@ 2019-12-02  4:10           ` Emanuel Berg
  0 siblings, 0 replies; 15+ messages in thread
From: Emanuel Berg @ 2019-12-02  4:10 UTC (permalink / raw)
  To: zsh-users

Daniel Shahaf wrote:

>> How do I send the array as a name/pointer
>> argument, i.e. not just the value?
>
> To read an array by name:
>
> f() {
>  local array_name=$1
>  print -rl -- "${(@P)array_name}"
> }
> typeset -a foo=(bar baz)
> f foo [...]

Thanks a lot for all that wonderful info! :)

That should be the last piece of the puzzle Gw!

> gmane.comp.shells.zsh.user and
> zsh-users@zsh.org are the same thing

... OK?

-- 
underground experts united
http://user.it.uu.se/~embe8573
https://dataswamp.org/~incal


^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
       [not found]       ` <20191201233848.7selcpkvenlu65px__33142.5388505281$1575243619$gmane$org@tarpaulin.shahaf.local2>
@ 2019-12-02 14:14         ` Stephane Chazelas
  2019-12-02 14:29           ` Roman Perepelitsa
  0 siblings, 1 reply; 15+ messages in thread
From: Stephane Chazelas @ 2019-12-02 14:14 UTC (permalink / raw)
  To: Daniel Shahaf; +Cc: zsh-users

2019-12-01 23:38:48 +0000, Daniel Shahaf:
[...]
> I think the following would work? —
> 
>     typeset -a l=(…)
>     eval "l=(" $(print -rl -- "${(@qqqq)l}" | shuf) ")"
> 
> shuf(1) expects newline-separated output, and the output of ${(qqqq)} never
> has literal newlines.

$(...) should be quoted or there'll be IFS-splitting.

I wouldn't trust qqqq with eval in locales using charsets like
BIG5, BIG5-HKSCS, GB18030...

$ LC_ALL=zh_HK.big5hkscs luit
$ l=('α')
$ eval "print -r -- $(printf '%s\n' "${(@qqqq)l}")"
zsh: unmatched '

In pratice, I think only ${(q)...} (using single quotes) is
totally safe for reinput regardless of the locale (at least with
the locales typically available on GNU systems).

So you'd want to do something like:

(){ local LC_ALL=C
  eval "l=($(printf '%s\n' "${(@qqqq)l}" | shuf))"
}

That "eval" would still make me nervous.

The:

l=(/(Ne['reply=("$l[@]")']oe['REPLY=$RANDOM']))

approach I gave would not have this kind of issue and would
avoid the dependency on a GNU utility. (it's got its own issues
linked to zsh's $RANDOM taking only 32768 different values, or
being easy to guess...).

-- 
Stephane

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: shuffle array
  2019-12-02 14:14         ` Stephane Chazelas
@ 2019-12-02 14:29           ` Roman Perepelitsa
  0 siblings, 0 replies; 15+ messages in thread
From: Roman Perepelitsa @ 2019-12-02 14:29 UTC (permalink / raw)
  To: Daniel Shahaf, Zsh Users

Another alternative is to implement the standard inplace shuffle
algorithm from scratch:

    local -i i
    for ((i = 2; i <= $#l; ++i)); do
      local j=$((RANDOM % i + 1))
      # swap l[i] and l[j]
      local tmp=$l[i]
      l[i]=$l[j]
      l[j]=$tmp
    done

Due to RANDOM having a rather narrow range, this will introduce bias
on large arrays and won't work at all on arrays with more than 32k
elements. These issues can be mitigated by replacing RANDOM with
(RANDOM << 15 | RANDOM) or even with (RANDOM << 30 | RANDOM << 15 |
RANDOM).

Roman.

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2019-12-02 14:30 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-29  3:20 shuffle array Emanuel Berg
2019-11-29  6:42 ` Marc Chantreux
2019-11-29  8:19   ` Emanuel Berg
2019-11-29  8:43     ` Marc Chantreux
2019-11-29  8:50       ` Emanuel Berg
2019-11-29 13:38         ` Clint Adams
     [not found]   ` <8636e7w0w9.fsf__36104.0529723809$1575015640$gmane$org@zoho.eu>
2019-12-01 20:07     ` Stephane Chazelas
2019-12-01 21:22       ` Emanuel Berg
2019-12-01 21:25         ` Emanuel Berg
2019-12-01 23:46         ` Daniel Shahaf
2019-12-02  4:10           ` Emanuel Berg
2019-12-01 23:38       ` Daniel Shahaf
     [not found]       ` <20191201233848.7selcpkvenlu65px__33142.5388505281$1575243619$gmane$org@tarpaulin.shahaf.local2>
2019-12-02 14:14         ` Stephane Chazelas
2019-12-02 14:29           ` Roman Perepelitsa
2019-11-29  8:44 ` Emanuel Berg

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).