The following two functions are normally equivalent: () { typeset -g tst } () { tst= } Both result in a global variable 'tst' being created. However when the second function is called from a function that happens to declare a private variable with the same name, it no longer creates a global variable but instead aborts with an error: % () { private tst; () { tst= } } (anon):1: tst: attempt to assign private in nested scope % This is not how one would expect private variables to behave. Inside the inner function, the private variable should be completely out of scope and the `tst=` statement should result in the creation of a global variable. Note that the error above does not happens when there already exists a global variable with the same name: % typeset -g tst % () { private tst; () { tst= } } %
Marlon Richert wrote on Wed, Sep 01, 2021 at 14:49:58 +0300:
> This is not how one would expect private variables to behave. Inside
> the inner function, the private variable should be completely out of
> scope and the `tst=` statement should result in the creation of a
> global variable.
Makes sense to me. A regression test (one with the 'F' flag) would be
welcome, if you've got spare tuits.
On Wed, Sep 1, 2021 at 4:51 AM Marlon Richert <marlon.richert@gmail.com> wrote:
>
> % () { private tst; () { tst= } }
> (anon):1: tst: attempt to assign private in nested scope
> %
>
> This is not how one would expect private variables to behave.
But it's how variable scoping behaves in the shell language. If I do
% () { local tst; () { tst=foo } }
Then it is the tst in the surrounding scope that gets altered, not the
global one. If the inner function is INTENDED to create a global, it
should be using "typeset -g" explicitly.
On 9/1/21, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Wed, Sep 1, 2021 at 4:51 AM Marlon Richert <marlon.richert@gmail.com>
> wrote:
>>
>> % () { private tst; () { tst= } }
>> (anon):1: tst: attempt to assign private in nested scope
>> %
>>
>> This is not how one would expect private variables to behave.
>
> But it's how variable scoping behaves in the shell language. If I do
>
> % () { local tst; () { tst=foo } }
>
> Then it is the tst in the surrounding scope that gets altered, not the
> global one. If the inner function is INTENDED to create a global, it
> should be using "typeset -g" explicitly.
This seems weird to me, since private is stated to be "used to create
parameters whose scope is limited to the current function body, and
not to other functions called by the current function." If the
private parameter is not in scope, then any called functions should
behave as if it is not in scope, ie automatically create a parameter
scoped to whatever level is above the one that has the private.
(technically speaking, this is probably hard). If private parameters
are only compatible with code that runs without warnings under setopt
WARN_CREATE_GLOBAL, then we should document this.
--
Mikael Magnusson
On Wed, Sep 1, 2021 at 7:03 PM Mikael Magnusson <mikachu@gmail.com> wrote: > > On 9/1/21, Bart Schaefer <schaefer@brasslantern.com> wrote: > > On Wed, Sep 1, 2021 at 4:51 AM Marlon Richert <marlon.richert@gmail.com> > > wrote: > >> > >> % () { private tst; () { tst= } } > >> (anon):1: tst: attempt to assign private in nested scope > >> % > >> > >> This is not how one would expect private variables to behave. > > > > But it's how variable scoping behaves in the shell language. If I do > > > > % () { local tst; () { tst=foo } } > > > > Then it is the tst in the surrounding scope that gets altered, not the > > global one. If the inner function is INTENDED to create a global, it > > should be using "typeset -g" explicitly. > > This seems weird to me, since private is stated to be "used to create > parameters whose scope is limited to the current function body, and > not to other functions called by the current function." If the > private parameter is not in scope, then any called functions should > behave as if it is not in scope, ie automatically create a parameter > scoped to whatever level is above the one that has the private. > (technically speaking, this is probably hard). Also note that when _reading_ variables, the `private` keyword already behaves like this, that is, the private variable is out of scope for the inner function: % () { ; private tst=bar; () { print tst=$tst } } tst= % % tst=foo; () { ; private tst=bar; () { print tst=$tst } } tst=foo % > If private parameters > are only compatible with code that runs without warnings under setopt > WARN_CREATE_GLOBAL, then we should document this. That would largely defeat the purpose of private variables, wouldn't it? :)
On Wed, Sep 1, 2021 at 5:10 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote: > > Marlon Richert wrote on Wed, Sep 01, 2021 at 14:49:58 +0300: > > This is not how one would expect private variables to behave. Inside > > the inner function, the private variable should be completely out of > > scope and the `tst=` statement should result in the creation of a > > global variable. > > Makes sense to me. A regression test (one with the 'F' flag) would be > welcome, if you've got spare tuits. I was about to create one in V10private.ztst, when I noticed it appears to exist already on lines 230 - 250. :) typeset -A hash_test=(top level) () { local -Pa array_test=(in function) local -PA hash_test=($array_test) () { print X ${(kv)hash_test} hash_test=(even deeper) { array_test+=(${(kv)hash_test}) } always { print ${array_test-array_test not set} ${(t)array_test} } } print Y ${(kv)hash_test} Z $array_test } print ${(kv)hash_test} ${(t)array_test} 1:privates are not visible in anonymous functions, part 3 >X top level >array_test not set ?(anon):4: array_test: attempt to assign private in nested scope F:future revision will create a global with this assignment
On 9/1/21, Marlon Richert <marlon.richert@gmail.com> wrote:
> On Wed, Sep 1, 2021 at 7:03 PM Mikael Magnusson <mikachu@gmail.com> wrote:
>> If private parameters
>> are only compatible with code that runs without warnings under setopt
>> WARN_CREATE_GLOBAL, then we should document this.
>
> That would largely defeat the purpose of private variables, wouldn't it? :)
Ah, I misinterpreted one of the examples, I thought that this would
work as you expect:
() { private foo; () { typeset -g foo=bar } }
(while this wouldn't:
() { private foo; () { foo=bar } }
)
But in both cases it is an error (although only the latter prints a
message and aborts execution), and foo is not set globally.
In fact a typeset -g is needed on the global level before the function
using private is called, which seems impossible for the innermost
function to enforce.
--
Mikael Magnusson
On Wed, Sep 1, 2021 at 10:00 AM Mikael Magnusson <mikachu@gmail.com> wrote: > > Ah, I misinterpreted one of the examples, I thought that this would > work as you expect: > () { private foo; () { typeset -g foo=bar } } > (while this wouldn't: > () { private foo; () { foo=bar } } > ) > > But in both cases it is an error (although only the latter prints a > message and aborts execution), and foo is not set globally. Something else has changed since the implementation of private, I think, because this is not what I expected: top () { private foo=top; mid } mid () { typeset -g foo=middle; bot } bot () { print $foo } functions -t top % top +top:0> private foo=top +top:0> mid +mid:0> typeset -g foo=middle +mid:0> bot bot: foo: attempt to assign private in nested scope What makes the expansion of $foo in bot into an assignment? > In fact a typeset -g is needed on the global level before the function > using private is called, which seems impossible for the innermost > function to enforce. As you can see from the "F:" test Marlon quoted, this is something I eventually intended to change. It's rather complicated because of the way locals are implemented as a LIFO at each individual named entry in the parameter table, rather than a LIFO of parameter tables. Even without private, "typeset -g" only reaches as far upwards as the most recent local declaration of the name.
[-- Attachment #1: Type: text/plain, Size: 904 bytes --] On Wed, Sep 1, 2021 at 10:35 AM Bart Schaefer <schaefer@brasslantern.com> wrote: > > Something else has changed since the implementation of private, I > think, because this is not what I expected: > > top () { private foo=top; mid } > mid () { typeset -g foo=middle; bot } > bot () { print $foo } > functions -t top The attached patch addresses this, producing the output I expected: +top:0> private foo=top +top:0> mid +mid:0> typeset -g foo=middle +mid:0> bot +bot:0> print middle middle It also improves the error message in the event that a typeset attempts to change the scope or properties of a private parameter. > Even > without private, "typeset -g" only reaches as far upwards as the most > recent local declaration of the name. A documentation patch to better explain this is included. As a bonus, there's also an explanation of why some parameters do not appear in "typeset -p" output. [-- Attachment #2: private-scoping.txt --] [-- Type: text/plain, Size: 6158 bytes --] diff --git a/Doc/Zsh/mod_private.yo b/Doc/Zsh/mod_private.yo index 78aee0acf..6da18f814 100644 --- a/Doc/Zsh/mod_private.yo +++ b/Doc/Zsh/mod_private.yo @@ -10,12 +10,16 @@ ifnzman() startitem() findex(private) cindex(private parameter, creating) -item(tt(private) [ {tt(PLUS())|tt(-)}tt(AHUahlprtux) ] \ +item(tt(private) [ {tt(PLUS())|tt(-)}tt(AHUahlmrtux) ] \ [ {tt(PLUS())|tt(-)}tt(EFLRZi) [ var(n) ] ] [ var(name)[tt(=)var(value)] ... ])( The tt(private) builtin accepts all the same options and arguments as tt(local) (ifzman(zmanref(zshbuiltins))ifnzman(noderef(Shell Builtin Commands))) except for the `tt(-)tt(T)' option. Tied parameters may not be made private. +The `tt(-)tt(p)' option is presently a no-op because the state of +private parameters cannot be reliably be reloaded. This also applies +to printing private parameters with `tt(typeset -p)'. + If used at the top level (outside a function scope), tt(private) creates a normal parameter in the same manner as tt(declare) or tt(typeset). A warning about this is printed if tt(WARN_CREATE_GLOBAL) is set @@ -75,7 +79,8 @@ itemiz(Within any other function called by the declaring function, the private parameter does em(NOT) hide other parameters of the same name, so for example a global parameter of the same name is visible and may be assigned or unset. This includes calls to anonymous functions, although -that may also change in the future.) +that may also change in the future. However, the private name may not be +created outside the local scope when it was not previously declared.) itemiz(An exported private remains in the environment of inner scopes but appears unset for the current shell in those scopes. Generally, exporting private parameters should be avoided.) diff --git a/Doc/Zsh/params.yo b/Doc/Zsh/params.yo index b514eb072..a88e44d4f 100644 --- a/Doc/Zsh/params.yo +++ b/Doc/Zsh/params.yo @@ -632,6 +632,14 @@ In the parameter lists that follow, the mark `<S>' indicates that the parameter is special. `<Z>' indicates that the parameter does not exist when the shell initializes in tt(sh) or tt(ksh) emulation mode. +The parameters `tt(!)', `tt(#)', `tt(*)', `tt(-)', `tt(?)', `tt(@)', +`tt($)', `tt(ARGC)', `tt(HISTCMD)', `tt(LINENO)', `tt(PPID)', +`tt(status)', `tt(TTYIDLE)', `tt(zsh_eval_context)', +`tt(ZSH_EVAL_CONTEXT)', and `tt(ZSH_SUBSHELL)' are read-only and thus +cannot be restored by the user, so they are not output by +`tt(typeset -p)'. This also applies to many read-only parameters loaded +from modules. + The following parameters are automatically set by the shell: startitem() diff --git a/Src/Modules/param_private.c b/Src/Modules/param_private.c index 24545819d..c53839152 100644 --- a/Src/Modules/param_private.c +++ b/Src/Modules/param_private.c @@ -125,7 +125,7 @@ makeprivate(HashNode hn, UNUSED(int flags)) break; } /* PM_HIDE so new parameters in deeper scopes do not shadow */ - pm->node.flags |= (PM_HIDE|PM_SPECIAL|PM_REMOVABLE); + pm->node.flags |= (PM_HIDE|PM_SPECIAL|PM_REMOVABLE|PM_RO_BY_DESIGN); pm->level -= 1; } } @@ -171,7 +171,7 @@ bin_private(char *nam, char **args, LinkList assigns, Options ops, int func) { int from_typeset = 1; int ofake = fakelevel; /* paranoia in case of recursive call */ - int hasargs = *args != NULL || (assigns && firstnode(assigns)); + int hasargs = /* *args != NULL || */ (assigns && firstnode(assigns)); makeprivate_error = 0; if (!OPT_ISSET(ops, 'P')) { @@ -190,8 +190,10 @@ bin_private(char *nam, char **args, LinkList assigns, Options ops, int func) return bin_typeset("private", args, assigns, ops, func); } - ops->ind['g'] = 2; /* force bin_typeset() to behave as "local" */ - if (OPT_ISSET(ops, 'p') || (!hasargs && OPT_ISSET(ops, '+'))) { + if (!(OPT_ISSET(ops, 'm') || OPT_ISSET(ops, '+'))) + ops->ind['g'] = 2; /* force bin_typeset() to behave as "local" */ + if (OPT_ISSET(ops, 'p') || OPT_ISSET(ops, 'm') || + (!hasargs && OPT_ISSET(ops, '+'))) { return bin_typeset("private", args, assigns, ops, func); } @@ -559,7 +561,7 @@ printprivatenode(HashNode hn, int printflags) static struct builtin bintab[] = { /* Copied from BUILTIN("local"), "P" added */ - BUILTIN("private", BINF_PLUSOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL | BINF_ASSIGN, (HandlerFunc)bin_private, 0, -1, 0, "AE:%F:%HL:%PR:%TUZ:%ahi:%lprtux", "P") + BUILTIN("private", BINF_PLUSOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL | BINF_ASSIGN, (HandlerFunc)bin_private, 0, -1, 0, "AE:%F:%HL:%PR:%TUZ:%ahi:%lmprtux", "P") }; static struct features module_features = { diff --git a/Src/params.c b/Src/params.c index 704aad588..b703a97ce 100644 --- a/Src/params.c +++ b/Src/params.c @@ -1013,6 +1013,11 @@ createparam(char *name, int flags) (oldpm->node.flags & PM_SPECIAL) || /* POSIXBUILTINS horror: we need to retain 'export' flags */ (isset(POSIXBUILTINS) && (oldpm->node.flags & PM_EXPORTED))) { + if (oldpm->node.flags & PM_RO_BY_DESIGN) { + zerr("%s: can't change parameter attribute", + name); + return NULL; + } oldpm->node.flags &= ~PM_UNSET; if ((oldpm->node.flags & PM_SPECIAL) && oldpm->ename) { Param altpm = diff --git a/Test/V10private.ztst b/Test/V10private.ztst index 03e8259d5..56ffbc5b4 100644 --- a/Test/V10private.ztst +++ b/Test/V10private.ztst @@ -116,14 +116,14 @@ } outer () { local -PA hash_test=(in function) - typeset -p hash_test + private + hash_test inner } outer print ${(kv)hash_test} 0:private hides value from surrounding scope in nested scope >typeset -a hash_test=( top level ) ->typeset -A hash_test=( [in]=function ) +>hash_test=( [in]=function ) >typeset -g -a hash_test=( top level ) >array-local top level >top level @@ -246,7 +246,7 @@ F:note "typeset" rather than "private" in output from outer 1:privates are not visible in anonymous functions, part 3 >X top level >array_test not set -?(anon):4: array_test: attempt to assign private in nested scope +?(anon):4: array_test: can't change parameter attribute F:future revision will create a global with this assignment typeset -a array_test
On 10/3/21, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Wed, Sep 1, 2021 at 10:35 AM Bart Schaefer <schaefer@brasslantern.com>
> wrote:
>>
>> Something else has changed since the implementation of private, I
>> think, because this is not what I expected:
>>
>> top () { private foo=top; mid }
>> mid () { typeset -g foo=middle; bot }
>> bot () { print $foo }
>> functions -t top
>
> The attached patch addresses this, producing the output I expected:
>
> +top:0> private foo=top
> +top:0> mid
> +mid:0> typeset -g foo=middle
> +mid:0> bot
> +bot:0> print middle
> middle
>
> It also improves the error message in the event that a typeset
> attempts to change the scope or properties of a private parameter.
>
>> Even
>> without private, "typeset -g" only reaches as far upwards as the most
>> recent local declaration of the name.
>
> A documentation patch to better explain this is included. As a bonus,
> there's also an explanation of why some parameters do not appear in
> "typeset -p" output.
+private parameters cannot be reliably be reloaded. This also applies
This sentence has an extra "be".
--
Mikael Magnusson
On Sun, Oct 3, 2021 at 9:54 PM Mikael Magnusson <mikachu@gmail.com> wrote:
>
> This sentence has an extra "be".
So everyone seems to be telling me. :-) I'll fix it when committing.