saved from prince of eval

saved from prince of eval

[Ray Andrews]

I'm trying to get some mileage out of the '(e)' flag, but it frustrates me:

1:    $ foo="${(e)${array}[${top}, ${bottom}]}"

foo contains the name of the array, nothing more.  The doc says that 
these things can be nested. I tried a few things on one line with no 
luck. If I insert the literal name of the array in place of "${array}" 
everything is fine.

But this works:

2:    $ bar='\$${array}[${top}, ${bottom}]'
3:    $ foo="${(e)$(print -R "${(e)${bar}}")}"

I'm not sure how to interpret it tho. Is 'print' doing the work here, or 
is print a bystander as a nested use of '(e)' works?  The expansion of 
'bar' with it's single quoted string in '3' has a sort of linearity to 
it--you'd think that '1' would work, but it refuses to expand all three 
parameters whereas in '3' the expansion handles all three parameters in 
a 'dumb' but actually far more intuitive and helpful way--it just does 
it. I'd like to understand this better, it could be a case of some tiny 
syntactic error. Why does '1' not work? What's really going on in '3'? 
In any case, it replaces the evil eval which is very nice:

      $ foo='stranger'
      $ bar='echo howdy $foo'
      $ eval $bar
    howdy stranger                     << Sure ...

      $ eval baz=$bar; echo $baz
    zsh: command not found: howdy << Ferkrissakes
    echo howdy $foo

      $ baz=${(e)bar}; echo $baz << Tranquility
    echo howdy stranger

... so '(e)' is our friend. We should be warned from mother's milk not 
to use eval, and I have been warned, but, nuts, I didn't realize that 
there was a safe solution.

Re: saved from prince of eval

[Bart Schaefer]

On Nov 8,  9:07am, Ray Andrews wrote:
}
} I'm trying to get some mileage out of the '(e)' flag, but it frustrates me

If you haven't already, you should read through the "Rules" section
under "Parameter Expansion".  "man zshexpn" and search for "Rules".

In particular the last sentence of the #1 rule "Nested Substitution":

     ... Note  that,  unless  the  '(P)'
     flag is present, the flags and any subscripts apply directly to
     the value of the nested substitution; for example,  the  expan-
     sion ${${foo}} behaves exactly the same as ${foo}.

Then note that (e) isn't applied until nearly the end of the procedure,
at rule #10.

So in this expression:

} 1:    $ foo="${(e)${array}[${top}, ${bottom}]}"

First ${array} expands, and then [$[top},${bottom}] applies to that
value -- which isn't an array, it's a string, so the subscripts are
extracting a range of characters from that string.  Finally that
substring is re-evaluated (but probably is nothing interesting).

What you needed was

    foo="${(e)${(P)array}[${top}, ${bottom}]}"

though as previously explained that doesn't work as you want for an
associative $array.
 
} If I insert the literal name of the array in place of "${array}" 
} everything is fine.

In "${(e)the_literal_array[${top}, ${bottom}]}" the subscript applies
directly the array (because there is no nested expression).

} But this works:
} 
} 2:    $ bar='\$${array}[${top}, ${bottom}]'
} 3:    $ foo="${(e)$(print -R "${(e)${bar}}")}"

This describes that you've got at least three levels of unexpanded
references:

- whatever array is named by $array contains unexpanded references
  in the array elements
- bar contains unexpanded references to $array, $top, and $bottom
- expanding \$ in $bar becomes an unexpanded reference to the array
  named by $array with the subscript expanded from $top and $bottom

I have no idea why you want to put yourself in that situation, but if
you have three unexpanded levels then you need three re-evaluations:

- the inner (e) flag
- $(...), which you can think of as:  eval "..." | read
- the outer (e) flag
 
} I'm not sure how to interpret it tho. Is 'print' doing the work here, or 
} is print a bystander as a nested use of '(e)' works?

In this case the "print" is doing nothing except provide the standard
output to be captured from $(...).

You can replace the $(...) with another ${(e)...} to get your third
needed re-evaluation, and you don't need the inermost ${ } around bar:

    foo="${(e)${(e)${(e)bar}}}"

Again it's pretty ridiculous that you're doing anything like this in
the first place.

}       $ foo='stranger'
}       $ bar='echo howdy $foo'
}       $ eval baz=$bar; echo $baz
}     zsh: command not found: howdy << Ferkrissakes

Well, think about it a bit harder.  Or better yet, "setopt xtrace" and
watch what is happening.

torch% foo='stranger'
torch% bar='echo howdy $foo'
torch% setopt xtrace
torch% eval baz=$bar; echo $baz
+zsh:4> eval 'baz=echo howdy $foo'
+(eval):1> baz=echo howdy stranger
zsh: command not found: howdy

The assignment becomes "baz=echo" which is prefixed to the command
"howdy stranger".

Re: saved from prince of eval

[Ray Andrews]

On 11/08/2015 11:57 AM, Bart Schaefer wrote:
> If you haven't already, you should read through the "Rules" section
> under "Parameter Expansion".  "man zshexpn" and search for "Rules".

Indeed.  I've bounced off it several times, it's the thickest of the 
thick.  Yes, I'll bone up and master it.
> In particular the last sentence of the #1 rule "Nested Substitution":
>
>       ... Note  that,  unless  the  '(P)'
>       flag is present, the flags and any subscripts apply directly to
>       the value of the nested substitution; for example,  the  expan-
>       sion ${${foo}} behaves exactly the same as ${foo}.
>
> Then note that (e) isn't applied until nearly the end of the procedure,
> at rule #10.
>
> So in this expression:
>
> } 1:    $ foo="${(e)${array}[${top}, ${bottom}]}"
>
> First ${array} expands, and then [$[top},${bottom}] applies to that
> value -- which isn't an array, it's a string, so the subscripts are
> extracting a range of characters from that string.  Finally that
> substring is re-evaluated (but probably is nothing interesting).

Yes, by gum.  Running diagnostics, there were several times when what 
was printed were just a few chars.  I didn't even try to make sense of it.
>
> I have no idea why you want to put yourself in that situation,

It seems unavoidable if I'm to use arrays to  pass values to functions.


     foo='\$${IN[list]}[${IN[topE]}, ${IN[bottomE]}]'
     IN[PAGE]="${(e)$(print -R "${(e)${foo}}")}"

... 'IN' is the array copied by value from whatever array holds the data 
for the particular window in play. So in the above, I'm setting the 
contents of the current displayed page to a range from within the entire 
available data which is stored in the array who's name is  held in ' 
IN[list]' (and which varies depending on the window). It's called like this:

     set -A IN ${(kv)main}  # The window in play has all it's data in 
the array 'main'.
     n_list_draw 0 1

In the original, it's done like this:

         n-list-draw "$(( (NLIST_CURRENT_IDX-1) % page_height + 1 ))" \
             "$page_height" "$page_width" 0 1 "$NLIST_TEXT_OFFSET" inner 
ansi \
             "${(@)col_list[NLIST_FROM_WHAT_IDX_LIST_IS_SHOWN, end_idx]}"

... and of course there are all the local values in the function to 
receive the arguments, and at the end of the function all the changes 
have to recaptured to specific variables depending on the window.

Mine seems simpler, however I'm open to any suggestion.  I quite 
understand that I'm pushing zsh past it's natural comfort zone, but it 
does seem to be working, it's readable and extendable, but there might 
be an entirely different and better way of going about it.
> You can replace the $(...) with another ${(e)...} to get your third
> needed re-evaluation, and you don't need the inermost ${ } around bar:
>
>      foo="${(e)${(e)${(e)bar}}}"

Marvelous. It seems a bit of a labor, but it's understandable. Puts me 
in mind to ask why we can't have the 'expand everything in one go' flag:

      foo="$(E)bar}
>
> Again it's pretty ridiculous that you're doing anything like this in
> the first place.

As above, is there a better way?  I confess again that I'm trying to 
make zsh into C, but I'm getting away with it.  Why shouldn't structured 
programming be available in zsh?
>
> }       $ foo='stranger'
> }       $ bar='echo howdy $foo'
> }       $ eval baz=$bar; echo $baz
> }     zsh: command not found: howdy << Ferkrissakes
>
> Well, think about it a bit harder.  Or better yet, "setopt xtrace" and
> watch what is happening.

I understand what eval is doing, my point is just that it's hardly the 
safe thing when all I want to do is set a parameter from another 
parameter than contains parameters itself.  Of course there are times 
when we want eval, but setting a parameter ain't it. (e) is it.

Thanks Bart, a superb explanation as always.

Re: saved from prince of eval

[Ray Andrews]

On 11/08/2015 02:04 PM, Ray Andrews wrote:

Letting double quotes do some of the work:


     foo="\$${IN[list]}[${IN[topE]}, ${IN[bottomE]}]"
     IN[PAGE]="${(e)foo}"

... very near transparent and syntactically pleasing.  Still, I can't 
get it on one line.  It seems, so far, as if the double quotes around 
the first line are somehow conflicting with the '(e)' as if the (e) 
wants to do all the work if the contents of foo are merge in on one 
line. Not that the above is unsatisfactory, but I wonder if the 
intermediary 'foo' can be dispensed with.
>
>

Re: saved from prince of eval

[Bart Schaefer]

On Nov 9, 12:50am, Ray Andrews wrote:
}
}      foo="\$${IN[list]}[${IN[topE]}, ${IN[bottomE]}]"
}      IN[PAGE]="${(e)foo}"

${IN[list]} must name a normal array (because you can't do ranges of an
associative array), so I don't think you need (e), you just need (P):

   IN[PAGE]=${${(P)IN[list]}[IN[topE],IN[bottomE]]}

Because of math context in the enclosing [subscript]ing you don't need
the ${ } on IN[topE] etc.

The only time you'd want (e) is if you had something like

   IN[list]=array
   array=( '$x' '$y' )

Re: saved from prince of eval

[Ray Andrews]

On 11/09/2015 10:41 AM, Bart Schaefer wrote:
> On Nov 9, 12:50am, Ray Andrews wrote:
> }
> }      foo="\$${IN[list]}[${IN[topE]}, ${IN[bottomE]}]"
> }      IN[PAGE]="${(e)foo}"
>
> ${IN[list]} must name a normal array (because you can't do ranges of an
> associative array), so I don't think you need (e), you just need (P):

Yeah, IN[list] is the raw data coming in to the thing, just broken by 
newlines so that's a normal arry.
>
>     IN[PAGE]=${${(P)IN[list]}[IN[topE],IN[bottomE]]}
>
> Because of math context in the enclosing [subscript]ing you don't need
> the ${ } on IN[topE] etc.
>
> The only time you'd want (e) is if you had something like
>
>     IN[list]=array
>     array=( '$x' '$y' )
>
Ok, I'll try it.

Re: saved from prince of eval

[Ray Andrews]

On 11/09/2015 10:41 AM, Bart Schaefer wrote:
>     IN[PAGE]=${${(P)IN[list]}[IN[topE],IN[bottomE]]}
>
Thanks.

I've realized that my friend '(e)' was with the Enemy all along.  I have 
a function that prints out a half sane modification of 'set', and sure 
enough:
> }      foo="\$${IN[list]}[${IN[topE]}, ${IN[bottomE]}]"
> }      IN[PAGE]="${(e)foo}"
Ended up 'evaling'  the content and doubling the size of my environment 
every time it was run.  It's seems strange tho, how can a parameter be 
set twice (or 16 times) to exactly the same value? But your line above 
seems, finally,  to do what it's asked to do without evaling anything.

Speaking of which, we have 'env' which only gives us the--don't know the 
proper term--lowest level system stuff. And we have 'set' which gives us 
everything including the kitchen sink.  So I have this:

    #1) strip out the color.
    #2) cut lines down to 80 chars (some are a thousand).
    #3) select only things that are probably variables.
    eenv ()
    {
         set |
         sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g" |
         sed 's|^\(.\{1,80\}\).*|\1|' |
         grep --binary-files=text '='
    }


... which is an effort to display just the current parameters. However 
the output refuses to open in any of my editors if piped to a file, they 
all think it's binary.  Looking at it in hex, I see that it's a soup of 
backslashes and single quotes. Why is that?  Is there a civilized way of 
listing all current parameters that's just plain text, no backslashes?  
Not knowing the hierarchy, perhaps even avoiding those deep things like 
'color=' which isn't something one wants to look at very often--those 
deep internal things that have nothing to do with the user.  It would be 
a useful thing.

Re: saved from prince of eval

[Bart Schaefer]

On Nov 10,  5:07pm, Ray Andrews wrote:
}
} It's seems strange tho, how can a parameter be 
} set twice (or 16 times) to exactly the same value?

Obviously it can't.  You're either accidentally appending to something,
or setting the value of a new key in an associative array.

} Speaking of which, we have 'env' which only gives us the--don't know the 
} proper term--lowest level system stuff.

It's called "env" for "environment" -- it shows you the stuff that has
been "export"ed (or that was already in the environment before the
shell started).

} And we have 'set' which gives us 
} everything including the kitchen sink.

You probably want "typeset -p" followed by a list of the parameter
names you actually care about seeing.

} the output refuses to open in any of my editors if piped to a file, they 
} all think it's binary.

Probably because $IFS includes a NUL (zero) byte and bracketed_paste
contains a couple of ESC characters.

} Looking at it in hex, I see that it's a soup of 
} backslashes and single quotes. Why is that?

Because it's being output in the format that the shell would have to
parse in order to read it back in again.  However, the backslashes
are probably of your own making, from your attempts to use (e).  I've
got exactly two parameters that output with a backslash in the value:
SPROMPT and WORDCHARS.

Re: saved from prince of eval

[Ray Andrews]

On 11/10/2015 08:18 PM, Bart Schaefer wrote:
> On Nov 10,  5:07pm, Ray Andrews wrote:
> }
> } It's seems strange tho, how can a parameter be
> } set twice (or 16 times) to exactly the same value?
>
> Obviously it can't.  You're either accidentally appending to something,
> or setting the value of a new key in an associative array.
I'd not have thought so.  Whatever strange thing was happening it 
stopped with your last code suggestion.  I suppose I should try to 
figure out what was going on.  But it only happened with the environment 
viewer. Must have been evaling the output, but somehow to new values tho 
it sure didn't look like it.  In the program after four runs:

    ...
    │ -=5679ENVXgiks │
    │ -=5679ENVXgiks │
    │ -=5679ENVXgiks │
    │ A=/aArchive │
    │ A=/aArchive │
    │ A=/aArchive │
    │ A=/aArchive │
    │ aa=/tmp/tmp_to_env.t9 │
    │ aa=/tmp/tmp_to_env.t9 │
    │ aa=/tmp/tmp_to_env.t9 │
    │ aa=/tmp/tmp_to_env.t9
    ...

Then seeing what's going on:

    $ set | grep "aArchive"

    A=/aArchive
    A=/aArchive
    A=/aArchive
    A=/aArchive
    A=/aArchive

... so God knows ...
Some invisible difference? Some non-printing char that's significant, 
but invisible? But it's not a problem any more.


> Because it's being output in the format that the shell would have to
> parse in order to read it back in again.  However, the backslashes
> are probably of your own making, from your attempts to use (e).  I've
> got exactly two parameters that output with a backslash in the value:
> SPROMPT and WORDCHARS.
Here's what the output of " typeset -p " piped to a file looks like in 
geany:


℧㴧ਰ⌧㴧ਰ␧㴧ㄳ㌶ਲ਼⨧㴧
⤠ⴊ㌽㔴㜶䌹䕄乊噔婘楧浫ੳ㴰獺੨㼧㴧ਰ㵀
⤠䄊⼽䅡捲楨敶䄊 
䝒㵃ਰ㵂愯潗歲湩⽧獚⽨潂瑯䌊偄呁㵈✧䌊䱏剏䕔䵒砽捦 㑥琭牥業慮੬佃啌乍㵓〸䌊 
啐奔䕐椽㠶ਸ਼䉄单卟卅䥓乏䉟单䅟䑄䕒卓✽湵硩愺獢牴捡㵴琯灭搯畢⵳䵰牷㕂䍸湈本 
極㵤㔳ㄸ攳搴晢㑦戶㌸ㅡ㤱㜲挲㘵攳ㄳ㥢ਧ 䕄䭓佔彐䕓卓佉㵎晸散䐊剉呓䍁㵋愯潗 
歲湩⽧獚⽨潂瑯搯物瑳捡੫䥄偓䅌㵙〺〮䔊䥄佔㵒捭摥瑩䔊䥇㵄ਰ啅䑉〽䘊䝉低䕒✽ਧ偆 
呁㵈甯牳氯捯污猯慨敲稯桳 猯瑩ⵥ畦据楴湯㩳稯桳猯慨敲稯桳猯瑩ⵥ畦据楴湯㩳稯 
桳猯慨敲稯桳㔯ㄮㄮ搭癥〭是湵瑣潩獮䜊䑉〽䜊䅌䕄䍟呁䱁䝏偟呁㵈਺䱇䑁彅位啄䕌 
偟呁㵈਺䱇䑁彅 䥐䵘偁偟呁㵈਺假彇䝁久彔义但⼽浴⽰灧ⵧふ䉐兗匯朮杰愭敧瑮ㄺㄴ 
㨶਱䥈呓䡃剁㵓℧⍞ਧ䥈呓䵃㵄㔱㔰਷䥈呓䥆䕌⼽坡牯楫杮娯桳䈯潯⽴楨瑳楦敬䠊卉協 
婉㵅〲〰ਰ佈䕍⼽潲瑯䠊协㵔偈礭ⴸ㘭䐭扥慩㉮䠊单䱈䝏义䘽䱁䕓䤊卆✽ठ

However, I tried mcedit on the same file ... and it looks absolutely 
fine, so go figger.  Some charset or UTF or encoding or locale or who 
knows what.  Nevermind.