associative array questions

associative array questions

[Ray Andrews]

Experimenting with associative arrays:

typeset -Ag aa IN

test1 ()
{
     aa[first]=1
     aa[second]=two
     aa[third]='three blind mice'
}

test2 ()
{
     set -A IN ${(Pkv)${1}}
     IN[second]='tea for two'
     set -A "$1" ${(kv)IN}
}

test3 ()
{
     aarray="$1"
     print "aa[second] is: ${${(P)aarray}[second]}"
# BAD:
#    ${(P)aarray}[second]='tea for two'
# GOOD:
     eval "${aarray}[second]='tea for two'"

     print "\naa changed:\n"
     printf "%-10s %s\n" ${(kv)aa}
}

... test2 works fine and it's how my existing code does it. test2 and 
test3 might have to chew on one of several possible arrays so the use of 
'$1' is necessary.  However, C-brained as I am, I'm wanting to link to 
the input array directly via a pointer sort of operation so in test3 I'm 
trying to grab the name of the target array and modify it (the array) 
directly. The 'print' works fine.  So I think to myself: strip off the 
outer '${}' and now we have the name of the element, not it's expansion, 
so an assignment should be possible, but it doesn't work.  
'${(P)aarray'} should expand to 'aa' remembering that it's a variable -- 
so I think. However doing it via 'eval' works.   In practice test2 has 
the more streamlined code but I'm still curious that the BAD attempt at 
an assignment doesn't work.  I know I'm thinking too visually. BTW, it's 
also unexpected that the array prints upside down:

$ . test1; test1; test3 aa
aa[second] is: two

aa changed:

third        three blind mice
second    tea for two
first        1

... are associative arrays appended at the top?  I know that with 
associative arrays the element order isn't the primary thing but still.

Re: associative array questions

[Bart Schaefer]

On Sun, Dec 11, 2022 at 9:40 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
>[...] C-brained as I am, I'm wanting to link to
> the input array directly via a pointer sort of operation so in test3 I'm
> trying to grab the name of the target array and modify it (the array)
> directly.

There are no pointers to data structures in zsh.  Parameter reference
is always by value.  Even ${(P)other} returns a value, it's just the
value of the parameter whose name is in turn the value of $other.
Anything you do with the result of ${(P)other} acts on that value, not
on the indirectly-named parameter itself.

There is one sort-of exception:  The ${NAME=WORD} and related
assign-and-substitute operations.  E.g., if you use
${(P)other::=text}, then the parameter named by $other is assigned the
value "text".  I say this is only sort-of an exception because if
$other names an array, you can only assign to the entire array, not to
individual subscripted fields.

If you want to assign indirectly to a particular field (whether a
position in an ordinary array or a named element in an associative)
you have to store the entire subscript expression in $other, such as
  other="aa[second]"
  : ${(P)other::='tea for two'}

>      print "aa[second] is: ${${(P)aarray}[second]}"

Incidentally, this doesn't work in zsh earlier than 5.2.  Even in 5.9,
it doesn't carry over into the ${NAME=WORD} construct.

> # BAD:
> #    ${(P)aarray}[second]='tea for two'

Assignments are only parsed as such when there is an identifier
(string of nothing but alphanumerics or underscore, plus optional
subscript) to the left of the equal sign BEFORE any substitutions are
performed.  So this doesn't "look like" an assignment to the parser.

Lacking the outer ${...} enclosing the subscript, ${(P)aarray} expands
to all the values of the pointed-to associative array, and you get the
equivalent of

  "1" "two" "three blind mice"[second]

> # GOOD:
>      eval "${aarray}[second]='tea for two'"

In this case the "eval" turns the command into

  aa[second]='tea for two'

which satisfies the "identifier to the left" rule.

You could also do (another thing that doesn't work in older zsh)

  eval "${aarray}+=([second]='tea for two')"

> '${(P)aarray'} should expand to 'aa' remembering that it's a variable

Nope.  It expands to a value, not a variable reference.  Zsh doesn't
(yet) have what ksh calls "namerefs" ... block out everything you know
about pointers in C.

> also unexpected that the array prints upside down:

Associative arrays are unordered hash tables.  It might print "upside
down" or "inside out" depending on what keys are present.  They're
only even called "arrays" because that's how the value behaves if you
omit the subscript.

Re: associative array questions

[Ray Andrews]

On 2022-12-11 11:53, Bart Schaefer wrote:
> There are no pointers to data structures in zsh. 
I know, I'm trying to fake it.
> Parameter reference
> is always by value.  Even ${(P)other} returns a value, it's just the
> value of the parameter whose name is in turn the value of $other.
> Anything you do with the result of ${(P)other} acts on that value, not
> on the indirectly-named parameter itself.
So it seems.  I thought the (P) might force the issue.
> There is one sort-of exception:  The ${NAME=WORD} and related
> assign-and-substitute operations.  E.g., if you use
> ${(P)other::=text}, then the parameter named by $other is assigned the
> value "text".  I say this is only sort-of an exception because if
> $other names an array, you can only assign to the entire array, not to
> individual subscripted fields.
>
> If you want to assign indirectly to a particular field (whether a
> position in an ordinary array or a named element in an associative)
> you have to store the entire subscript expression in $other, such as
>    other="aa[second]"
>    : ${(P)other::='tea for two'}
That works.  I can intuit that parsing could be easier given the two 
stages, 'aa[second]' can only be interpreted one way.
> Assignments are only parsed as such when there is an identifier
> (string of nothing but alphanumerics or underscore, plus optional
> subscript) to the left of the equal sign BEFORE any substitutions are
> performed.
Before?  Nuts I thought the substitutions always happened first. 
Obviously not.  So leave expansion complexities on the right hand side.
>   So this doesn't "look like" an assignment to the parser.

set -x reports it as looking like nothing.

> Lacking the outer ${...} enclosing the subscript, ${(P)aarray} expands
> to all the values of the pointed-to associative array, and you get the
> equivalent of
>
>    "1" "two" "three blind mice"[second]
Exactly.  I was trying to force it to accept it as an identifier.

> You could also do (another thing that doesn't work in older zsh)
>    eval "${aarray}+=([second]='tea for two')"
Of course I'm trying to avoid the eval entirely.  You've always advised 
against it.

> Nope. It expands to a value, not a variable reference. Zsh doesn't
> (yet) have what ksh calls "namerefs" ... block out everything you know
> about pointers in C.
Well, at least I knew I was sinning.  Purely curiosity tho, I have no 
real issue to solve.  So we still have functionality to steal from ksh, 
that's interesting.
> Associative arrays are unordered hash tables.  It might print "upside
> down" or "inside out" depending on what keys are present.

Ok, so no order at all.  That's quite understandable, it would be extra 
overhead for the shell to even think about it and there's no need.

Thank you Sensei.

Re: associative array questions

[Bart Schaefer]

On Sun, Dec 11, 2022 at 12:52 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> Before?  Nuts I thought the substitutions always happened first.

Parsing has to happen first, otherwise it's not possible to find what
to substitute ...

> > You could also do (another thing that doesn't work in older zsh)
> >    eval "${aarray}+=([second]='tea for two')"
> Of course I'm trying to avoid the eval entirely.  You've always advised
> against it.

You can do

  typeset -g -- "${aarray}[second]"='tea for two'

to avoid eval.  Unfortunately typeset doesn't allow the += syntax.

Re: associative array questions

[Ray Andrews]

On 2022-12-11 13:41, Bart Schaefer wrote:
>
> Parsing has to happen first, otherwise it's not possible to find what
> to substitute ...
I should say that I thought substitution happened before assignment.  
But if I could watch how the wheels actually spin no doubt some of my 
naive assumptions would vanish.
>
>    typeset -g -- "${aarray}[second]"='tea for two'
>
I had no idea typeset could be used like that!  It  has the intuitive 
directness that I was hunting for.

Re: associative array questions

[Bart Schaefer]

On Sun, Dec 11, 2022 at 2:37 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> I should say that I thought substitution happened before assignment.

Well, of course it does happen before the assignment is actually
performed, but not before the parser decides that an assignment is
(not) what is syntactically indicated.

Re: associative array questions

[Ray Andrews]

On 2022-12-11 17:52, Bart Schaefer wrote:
> On Sun, Dec 11, 2022 at 2:37 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>> I should say that I thought substitution happened before assignment.
> Well, of course it does happen before the assignment is actually
> performed, but not before the parser decides that an assignment is
> (not) what is syntactically indicated.

If the learning curve wasn't about vertical I'd love to take a look 
under the hood.  One might cheerfully suppose that it's always obvious 
when an assignment is indicated but it's not hard to believe that life 
is not really so simple.  Given the multiple usage of so many characters 
I'm astonished that the parser could ever have been got working at all.  
Agonizing accretion I expect. BTW that would also explain why that 
'typedef' method you just showed me does what I want so painlessly -- 
the context of a typedef simplifies the parser's work.


>

Re: associative array questions

[Ray Andrews]

On 2022-12-11 13:41, Bart Schaefer wrote:

Bart:

I'm experimenting with your suggestions:

${(P)list::="tea for two"}

... and I get the 'command not found: 'tea for  two' ' message. Rather 
silly but:

echo ${(P)list::="tea for two"} > /dev/null

... and the real application works fine. The assignment happens and 
there's no looking for a command.

Re: associative array questions

[Bart Schaefer]

On Mon, Dec 12, 2022 at 7:31 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> ${(P)list::="tea for two"}
>
> ... and I get the 'command not found: 'tea for  two' ' message.

Well, yes, its a ${...} substitution so it does return a value, which
you then have to do something with.  The usual idiom is the ":"
builtin:

% : ${(P)list::="tea for two"}
%

Re: associative array questions

[Ray Andrews]

On 2022-12-12 20:39, Bart Schaefer wrote:
>
> % : ${(P)list::="tea for two"}
> %

Ha! I finally get to see that colon command do something. Anyway the 
real situation is running into trouble (I try to keep my questions here 
boiled down to the nub of the issue).  What's being modified is an array 
and the above method is interfering with the spit on lines; however it 
seems to be the perfect example of eval doing the work as a visual 
thinker would want it done:

local list=$1    # function takes name of array as argument.

eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.

... it has a sort of 'just-do-it' directness to it.   The function was 
previously working on only a specific array, and I'm trying to get it to 
work on any array.  Anyway the 'eval' method seems to work perfectly.  
Subject to better advice of course.

Re: associative array questions

[Roman Perepelitsa]

[-- Attachment #1: Type: text/plain, Size: 313 bytes --]

On Tue, 13 Dec 2022 at 17:12, Ray Andrews <rayandrews@eastlink.ca> wrote:

>
> local list=$1    # function takes name of array as argument.
>
> eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.


This eval will expand things that you don't want expanded. Use `set -A
$list` instead.

Roman.

[-- Attachment #2: Type: text/html, Size: 734 bytes --]

Re: associative array questions

[Ray Andrews]

[-- Attachment #1: Type: text/plain, Size: 633 bytes --]


On 2022-12-13 08:22, Roman Perepelitsa wrote:
> eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.
>
> This eval will expand things that you don't want expanded. Use `set -A 
> $list` instead.
>
> Roman.

I think I'm missing something Roman,  can't make it work.  Tried a few 
variations:

set -A $list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )
set -A $list=( "${(@M)${(P)list}#(#i)*$SEARCHBUF*}" )
set -A $list=( "${(@M)${(P)1}#(#i)*$SEARCHBUF*}" )

BTW is it possible to redirect the error messages and especially 'set 
-x' output?  All this is with zcurses active so the messages are hidden 
or chopped up.


[-- Attachment #2: Type: text/html, Size: 1379 bytes --]

Re: associative array questions

[Roman Perepelitsa]

On Tue, Dec 13, 2022 at 9:01 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2022-12-13 08:22, Roman Perepelitsa wrote:
>
> eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.
>
> This eval will expand things that you don't want expanded. Use `set -A $list` instead.
>
> Roman.
>
> I think I'm missing something Roman,  can't make it work.  Tried a few variations:
>
> set -A $list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )
> set -A $list=( "${(@M)${(P)list}#(#i)*$SEARCHBUF*}" )
> set -A $list=( "${(@M)${(P)1}#(#i)*$SEARCHBUF*}" )

`set` is a builtin (not a reserved word), so you use it like a normal
command. Your first email in this thread used it correctly. You can
also refer to `man zshbuiltins` for documentation.

> BTW is it possible to redirect the error messages and especially 'set -x' output?  All this is with zcurses active so the messages are hidden or chopped up.

Unlike bash, zsh cannot send xtrace output anywhere but to stderr. If
you redirect stderr, xtrace output will be redirected too.

Roman.

Re: associative array questions

[Ray Andrews]

On 2022-12-13 12:06, Roman Perepelitsa wrote:
> `set` is a builtin (not a reserved word), so you use it like a normal
> command. Your first email in this thread used it correctly. You can
> also refer to `man zshbuiltins` for documentation.

Nuts, I didn't even consider the problem being the equal sign. One gets 
used to using it in assignment, it looks strange without it.  But yes, 
referring back, I see the mistake.

set -A $list "${(@M)${(P)list}:#(#i)*$SEARCHBUF*}"

... that seems to be working.  BTW I know it's a builtin and it seems to 
me I used it like a normal command even if mistakenly, how not?

Bart:

> You missed where I said "if [$list in this case] names an array, you
can only assign to the entire array"?

It's nothing but the contents of a file broken down into lines. Yes, I'm assigning the whole thing.  This little piece of ... ah, that's where the confusion comes in -- at this juncture I'm not worried about the entire A-A, just the data array, namely the contents of a file.  So I suspect we're talking apples and oranges.  Subtle change in the topic, pardon! Anyway ... :

> eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.

> That second line isn't doing anything like what you want.  

But it does work perfectly. Remembering of course that there could be some lurking gotcha, but it *seems* fine.

> You don't
have the (P) flag in there anywhere to use the value of $list as a
parameter name, and you're just assigning that back to $list instead
of to the name from $1.  The eval accomplishes nothing because the
whole thing being eval'd is in single quotes to begin with; you might
as well drop both the single quotes and the word eval.

Can't say much, only to report that it does work. If I drop the eval and the outer quotes it does nothing, the original array is not changed. As I think I understand it, the second "list" simply becomes the name of the source array and, as I said, eval 'just does it' -- accesses the named array via it's name exactly as I'd intuitively want it to do.  So the '(P)' isn't needed.  Seems so, anyway. But I've been known to completely misunderstand things.

> set -A $list "${(P@M)list:#(#i)*$SEARCHBUF*}"

is what Roman is driving at.

Right, as above, I ... ah ... well maybe I didn't get it perfect, I 
don't have the '(P@M)' just '(P)' ... but so far so good. Once I dropped 
the equal sign I got it right the first time.

Re: associative array questions

[Bart Schaefer]

On Tue, Dec 13, 2022 at 2:47 PM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> > eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.
>
> > That second line isn't doing anything like what you want.
>
> But it does work perfectly.

It would work for the case where the array you want to rewrite is
actually named "list".  There's no way it can work for an array
indirectly named by the parameter $list.

Either you're not actually showing us what you're doing, or you're not
doing what you're telling us you are.

The closest thing that would do what you say you want is

eval $list'=( "${(@M)'$list':#(#i)*$SEARCHBUF*}" )'

Re: associative array questions

[Ray Andrews]

On 2022-12-13 15:06, Bart Schaefer wrote:
> It would work for the case where the array you want to rewrite is
> Either you're not actually showing us what you're doing, or you're not
> doing what you're telling us you are.

At the risk of making an ass of myself ... I'm quite at a loss, it 
damned well did work.  Tested 50 times, saved.  Restored the backup and 
... it doesn't work.  Dunno, one of those glitches in the matrix.  In 
the wrong directory?  Somehow created a real 'list'?  It damned well 
worked.  But somehow ... I've been known to make edits in the wrong file 
so what I'm editing and what I'm running ain't the same.  I'm quite at a 
loss.  Anyway the good news is that you are not mistaken -- which would 
be a serious difficulty.  I consume enough of you guy's time, I can't 
stand it when I'm totally wasting it barking up the wrong tree.  My 
apologies.  Anyway it is working again ... this time with 'set -A' which 
seems preferable besides it's expected to work.  God knows. Here's the 
whole function:

n_search ()
{
local list=$1 # Name of file buffer, no dollar sign idiot!!

     # Begin a search.
     if [[ ! "$SEARCHBUF" && ! "$prev_buffer" ]]; then
         msg="Begin case insensitive search ... "

     # There is no search, but there was in previous loop. i.e. we are 
searching, but we have backspaced the search to 'all' so re-enter search 
mode with virgin list:
     elif [[ ! "$SEARCHBUF" && "$prev_buffer" ]]; then
         prev_buffer=
         msg="No search, keep trying or UP to return to the list ... "

     # We are actively searching. Compute new list as each keystroke 
filters more narrowly. ENTER or up/dn arrows end search. Note, buffer 
can be preloaded when '$1' is given to nn_init(). But see 'c()' as an 
example of the argument to the function NOT being wanted as an initial 
search since that removes various heading lines.

     elif [[ "$SEARCHBUF" && "$SEARCHBUF" != "$prev_buffer" ]]; then
         prev_buffer="$SEARCHBUF"

         # Lines below:
         # 1) First remove lines starting with colon, they are messages.
         # 2) Next do the filtering:  Note that the search is not 
incremental even tho it looks like it is -- the search is always in the 
virgin list, else backspacing would require each previous search result 
to be saved separately:
         # 3) Now colorize the search string green. '#b': backreferences 
active, '#i':  case-insensitive.

###
# WORKS: (Tx Roman and Bart)
         set -A $list "${(@MP)list:#^:*}"
         set -A $list "${(@MP)list:#(#i)*$SEARCHBUF*}"
         set -A $list 
"${(@MP)list//(#bi)($SEARCHBUF)/$Color${match[1]}$nrm}"
         msg="Filtering with: $SEARCHBUF"
     fi
}

Re: associative array questions

[Bart Schaefer]

On Tue, Dec 13, 2022 at 8:12 AM Ray Andrews <rayandrews@eastlink.ca> wrote:
>
> On 2022-12-12 20:39, Bart Schaefer wrote:
> >
> > % : ${(P)list::="tea for two"}
> > %
>
> What's being modified is an array
> and the above method is interfering

You missed where I said "if [$list in this case] names an array, you
can only assign to the entire array"?  Or else you've actually got
  list="${1}[2]"
somewhere you're not showing us.

> local list=$1    # function takes name of array as argument.
>
> eval 'list=( "${(@M)list:#(#i)*$SEARCHBUF*}" )'     # filter the array.

That second line isn't doing anything like what you want.  You don't
have the (P) flag in there anywhere to use the value of $list as a
parameter name, and you're just assigning that back to $list instead
of to the name from $1.  The eval accomplishes nothing because the
whole thing being eval'd is in single quotes to begin with; you might
as well drop both the single quotes and the word eval.

set -A $list "${(P@M)list:#(#i)*$SEARCHBUF*}"

is what Roman is driving at.