Proof of concept: "static" parameter scope

Proof of concept: "static" parameter scope

[Bart Schaefer]

Worked on this quite a while ago but was waiting for 5.1+ to settle.

The patch below creates a module "zsh/param/static" which supports a
single builtin command "static".  The included doc section describes
"static" in detail, but briefly, it works like the "local" builtin
except that variables so declared are not dynamically visible to any
called functions.

"Called functions" includes recursive calls to the function itself, so
this doesn't work like C "static".  Therefore I'm in the market for a
better name.

Modules can't declare keywords, so this isn't able to support the new
array assignment syntax used by "local" et al.  However, the entry point
is coded to match that call signature.

There are some tricks played to avoid creating new PM_* flag bits.  If
this ceased to be a module and became a "real" builtin (something about
which I'm ambivalent) there are parts that could be simplified by using
another flag bit.

The patch includes a couple of tweaks to builtin.c and params.c to
cause them to use the "paramtab" abstraction rather than taking the
shortcut of directly calling default parameter table functions.  This
wasn't a widespread problem.

One remaining issue is that the "typeset" keyword is hardwired into
printparamnode().  That has other ramifications that we might want to
discuss at some point.

Please note that if you apply this patch and then re-"configure" your
source tree, it'll be exceptionally difficult to return that tree to
a state where it does not attempt to compile either param_static.o or
mod_static.yo even though those files have been removed.  I think I
complained about this a while ago.  I recommend playing with this in
a completely separate source tree (not just a parallel build tree).


diff --git a/Doc/Makefile.in b/Doc/Makefile.in
index 7645f42..16ee9d4 100644
--- a/Doc/Makefile.in
+++ b/Doc/Makefile.in
@@ -63,7 +63,7 @@ Zsh/mod_datetime.yo Zsh/mod_db_gdbm.yo Zsh/mod_deltochar.yo \
 Zsh/mod_example.yo Zsh/mod_files.yo Zsh/mod_langinfo.yo \
 Zsh/mod_mapfile.yo Zsh/mod_mathfunc.yo Zsh/mod_newuser.yo \
 Zsh/mod_parameter.yo Zsh/mod_pcre.yo Zsh/mod_regex.yo \
-Zsh/mod_sched.yo Zsh/mod_socket.yo \
+Zsh/mod_sched.yo Zsh/mod_socket.yo Zsh/mod_static.yo \
 Zsh/mod_stat.yo  Zsh/mod_system.yo Zsh/mod_tcp.yo \
 Zsh/mod_termcap.yo Zsh/mod_terminfo.yo \
 Zsh/mod_zftp.yo Zsh/mod_zle.yo Zsh/mod_zleparameter.yo \
diff --git a/Doc/Zsh/mod_static.yo b/Doc/Zsh/mod_static.yo
new file mode 100644
index 0000000..4342a92
--- /dev/null
+++ b/Doc/Zsh/mod_static.yo
@@ -0,0 +1,61 @@
+COMMENT(!MOD!zsh/param/static
+Builtins for managing static-scoped parameters in function context.
+!MOD!)
+The tt(zsh/param/static) module is used to create parameters whose scope
+is limited to the current function body, and em(not) to other functions
+called by the current function.
+
+This module provides a single autoloaded builtin:
+ifnzman()
+startitem()
+findex(static)
+cindex(static parameter, creating)
+item(tt(static) [ {tt(PLUS())|tt(-)}tt(AHUahlprtux) ] \
+[ {tt(PLUS())|tt(-)}tt(EFLRZi) [ var(n) ] ] [ var(name)[tt(=)var(value)] ... ])(
+The tt(static) builtin accepts all the same options and arguments as tt(local)
+(ifzman(zmanref(zshbuiltins))ifnzman(noderef(Shell Builtin Commands))) except
+for the `tt(-)tt(T)' option.  Tied parameters may not be made static.
+
+Although var(name)tt(=)var(value) sytax may be used in the argument list
+to initialize scalar parameters, there is no corresponding reserved word
+for tt(static), so the more general assignment syntax may not be used to
+initialize arrays.  This may change in a future release.
+
+If used at the top level (outside a function scope), tt(static) creates a
+normal parameter in the same manner as tt(declare) or tt(typeset).  A
+warning about this is printed if tt(WARN_CREATE_GLOBAL) is set
+(ifzman(zmanref(zshoptions))ifnzman(noderef(Options))).  Used inside a
+function scope, tt(static) creates a local parameter similar to one
+declared with tt(local), except having special properties noted below.
+
+Special parameters which expose or manipulate internal shell state, such
+as tt(ARGC), tt(argv), tt(COLUMNS), tt(LINES), tt(UID), tt(EUID), tt(IFS),
+tt(PROMPT), tt(RANDOM), tt(SECONDS), etc., cannot be made static unless
+the `tt(-)tt(h)' option is used to hide the special meaning of the
+parameter.  This may change in the future.
+)
+enditem()
+
+Parameters declared with tt(static) have the following properties:
+ifnzman()
+startitemize()
+itemiz(Within the function body where it is declared, the parameter
+behaves as a local, except as noted above for tied or special parameters.)
+itemiz(The type of a parameter declared static cannot be changed in the
+scope where it was declared, even if the parameter is unset.  Thus an
+array cannot be assigned to a static scalar, etc.)
+itemiz(Within any other function called by the declaring function, the
+static parameter does em(NOT) hide other parameters of the same name, so
+for example a global parameter of the same name is visible and may be
+assigned or unset.  This includes calls to anonymous functions, although
+that may also change in the future.)
+itemiz(An exported static remains in the environment of inner scopes but
+appears unset for the current shell in those scopes.)
+enditemize()
+
+Note that this differs from the static scope defined by compiled languages
+derived from C, in that the a new call to the same function creates a new
+scope, i.e., the parameter is still associated with the call stack rather
+than with the function definition.  It differs from ksh `tt(typeset -S)'
+because the syntax used to define the function has no bearing on whether
+the parameter scope is respected.
diff --git a/Src/Modules/param_static.c b/Src/Modules/param_static.c
new file mode 100644
index 0000000..b6c2403
--- /dev/null
+++ b/Src/Modules/param_static.c
@@ -0,0 +1,563 @@
+/*
+ * param_static.c - bindings for static parameter scopes
+ *
+ * This file is part of zsh, the Z shell.
+ *
+ * Copyright (c) 2015 Barton E. Schaefer
+ * All rights reserved.
+ *
+ * Permission is hereby granted, without written agreement and without
+ * license or royalty fees, to use, copy, modify, and distribute this
+ * software and to distribute modified versions of this software for any
+ * purpose, provided that the above copyright notice and the following
+ * two paragraphs appear in all copies of this software.
+ *
+ * In no event shall Barton E. Schaefer or the Zsh Development
+ * Group be liable to any party for direct, indirect, special, incidental, or
+ * consequential damages arising out of the use of this software and its
+ * documentation, even if Barton E. Schaefer and the Zsh
+ * Development Group have been advised of the possibility of such damage.
+ *
+ * Barton E. Schaefer and the Zsh Development Group
+ * specifically disclaim any warranties, including, but not limited to, the
+ * implied warranties of merchantability and fitness for a particular purpose.
+ * The software provided hereunder is on an "as is" basis, and
+ * Barton E. Schaefer and the Zsh Development Group have no
+ * obligation to provide maintenance, support, updates, enhancements, or
+ * modifications.
+ *
+ */
+
+#include "param_static.mdh"
+#include "param_static.pro"
+
+struct gsu_closure {
+    union {
+	struct gsu_scalar s;
+	struct gsu_integer i;
+	struct gsu_float f;
+	struct gsu_array a;
+	struct gsu_hash h;
+    } u;
+    void *g;
+};
+
+const struct gsu_scalar scalar_static_gsu =
+{ ss_getfn, ss_setfn, ss_unsetfn };
+
+const struct gsu_integer integer_static_gsu =
+{ si_getfn, si_setfn, si_unsetfn };
+
+const struct gsu_float float_static_gsu =
+{ sf_getfn, sf_setfn, sf_unsetfn };
+
+const struct gsu_array array_static_gsu =
+{ sa_getfn, sa_setfn, sa_unsetfn };
+
+const struct gsu_hash hash_static_gsu =
+{ sh_getfn, sh_setfn, sh_unsetfn };
+
+/*
+ * The trick here is:
+ *
+ * bin_static() opens a new parameter scope, then calls bin_typeset().
+ *
+ * bin_typeset() handles the usual parameter creation and error checks.
+ *
+ * makestatic() then finds all parameters created in the new scope and
+ * rejects them if they can't be "promoted" to the surrounding scope.
+ * Otherwise it swaps out their GSU structure and promotes them so they
+ * will be removed when the surrounding scope ends.
+ *
+ * bin_static() then ends the current scope, which discards any of the
+ * parameters rejected by makestatic().
+ *
+ */
+
+static int makestatic_error = 0;
+
+static void
+makestatic(HashNode hn, UNUSED(int flags))
+{
+    Param pm = (Param)hn;
+    if (pm->level == locallevel) {
+	if (pm->ename || (pm->node.flags & PM_NORESTORE) ||
+	    (pm->old &&
+	     (pm->old->level == locallevel - 1 ||
+	      ((pm->node.flags & (PM_SPECIAL|PM_REMOVABLE)) == PM_SPECIAL &&
+	       /* typeset_single() line 2300 discards PM_REMOVABLE -- why? */
+	       !is_static(pm->old))))) {
+	    zwarnnam("static", "can't change scope of existing param: %s",
+		     pm->node.nam);
+	    makestatic_error = 1;
+	    return;
+	}
+	struct gsu_closure *gsu = zhalloc(sizeof(struct gsu_closure));
+	switch (PM_TYPE(pm->node.flags)) {
+	case PM_SCALAR:
+	    gsu->g = (void *)(pm->gsu.s);
+	    gsu->u.s = scalar_static_gsu;
+	    pm->gsu.s = (GsuScalar)gsu;
+	    break;
+	case PM_INTEGER:
+	    gsu->g = (void *)(pm->gsu.i);
+	    gsu->u.i = integer_static_gsu;
+	    pm->gsu.i = (GsuInteger)gsu;
+	    break;
+	case PM_EFLOAT:
+	case PM_FFLOAT:
+	    gsu->g = (void *)(pm->gsu.f);
+	    gsu->u.f = float_static_gsu;
+	    pm->gsu.f = (GsuFloat)gsu;
+	    break;
+	case PM_ARRAY:
+	    gsu->g = (void *)(pm->gsu.a);
+	    gsu->u.a = array_static_gsu;
+	    pm->gsu.a = (GsuArray)gsu;
+	    break;
+	case PM_HASHED:
+	    gsu->g = (void *)(pm->gsu.h);
+	    gsu->u.h = hash_static_gsu;
+	    pm->gsu.h = (GsuHash)gsu;
+	    break;
+	default:
+	    makestatic_error = 1;
+	    break;
+	}
+	/* PM_HIDE so new parameters in deeper scopes do not shadow */
+	pm->node.flags |= (PM_HIDE|PM_SPECIAL|PM_REMOVABLE);
+	pm->level -= 1;
+    }
+}
+
+/**/
+static int
+is_static(Param pm)
+{
+    switch (PM_TYPE(pm->node.flags)) {
+    case PM_SCALAR:
+	if (!pm->gsu.s || pm->gsu.s->unsetfn != ss_unsetfn)
+	    return 0;
+	break;
+    case PM_INTEGER:
+	if (!pm->gsu.i || pm->gsu.i->unsetfn != si_unsetfn)
+	    return 0;
+	break;
+    case PM_EFLOAT:
+    case PM_FFLOAT:
+	if (!pm->gsu.f || pm->gsu.f->unsetfn != sf_unsetfn)
+	    return 0;
+	break;
+    case PM_ARRAY:
+	if (!pm->gsu.a || pm->gsu.a->unsetfn != sa_unsetfn)
+	    return 0;
+	break;
+    case PM_HASHED:
+	if (!pm->gsu.h || pm->gsu.h->unsetfn != sh_unsetfn)
+	    return 0;
+	break;
+    default:
+	/* error */
+	return 0;
+    }
+    return 1;
+}
+
+static int fakelevel;
+
+/**/
+static int
+bin_static(char *nam, char **args, LinkList assigns, Options ops, int func)
+{
+    int from_typeset = 1;
+    makestatic_error = 0;
+
+    if (locallevel == 0) {
+	if (isset(WARNCREATEGLOBAL))
+	    zwarnnam(nam, "invalid local scope, using globals");
+	return bin_typeset("static", args, assigns, ops, func);
+    }
+
+    ops->ind['g'] = 2;	/* force bin_typeset() to behave as "local" */
+
+    queue_signals();
+    fakelevel = locallevel;
+    startparamscope();
+    from_typeset = bin_typeset("static", args, assigns, ops, func);
+    scanhashtable(paramtab, 0, 0, 0, makestatic, 0);
+    endparamscope();
+    fakelevel = 0;
+    unqueue_signals();
+
+    return makestatic_error | from_typeset;
+}
+
+static void
+setfn_error(Param pm)
+{
+    pm->node.flags |= PM_UNSET;
+    zerr("%s: attempt to assign static in nested scope", pm->node.nam);
+}
+
+/*
+ * How the GSU functions work:
+ *
+ * The getfn and setfn family compare to locallevel and then call through
+ * to the original getfn or setfn.  This means you can't assign at a
+ * deeper scope to any parameter declared static unless you first declare
+ * it local again at the new scope.  Testing locallevel in getfn is most
+ * likely unnecessary given the scopestatic() wrapper installed below.
+ *
+ * The unsetfn family compare locallevel and restore the old GSU before
+ * calling the original unsetfn.  This assures that if the old unsetfn
+ * wants to use its getfn or setfn, they're unconditionally present.
+ *
+ */ 
+
+/**/
+static char *
+ss_getfn(Param pm)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.s);
+    GsuScalar gsu = (GsuScalar)(c->g);
+
+    if (locallevel >= pm->level)
+	return gsu->getfn(pm);
+    else
+	return (char *) hcalloc(1);
+}
+
+/**/
+static void
+ss_setfn(Param pm, char *x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.s);
+    GsuScalar gsu = (GsuScalar)(c->g);
+    if (locallevel == pm->level)
+	gsu->setfn(pm, x);
+    else
+	setfn_error(pm);
+}
+
+/**/
+static void
+ss_unsetfn(Param pm, int x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.s);
+    GsuScalar gsu = (GsuScalar)(c->g);
+    pm->gsu.s = gsu;
+    if (locallevel <= pm->level)
+	gsu->unsetfn(pm, x);
+}
+
+/**/
+static zlong
+si_getfn(Param pm)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.i);
+    GsuInteger gsu = (GsuInteger)(c->g);
+    if (locallevel >= pm->level)
+	return gsu->getfn(pm);
+    else
+	return 0;
+}
+
+/**/
+static void
+si_setfn(Param pm, zlong x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.i);
+    GsuInteger gsu = (GsuInteger)(c->g);
+    if (locallevel == pm->level)
+	gsu->setfn(pm, x);
+    else
+	setfn_error(pm);
+}
+
+/**/
+static void
+si_unsetfn(Param pm, int x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.i);
+    GsuInteger gsu = (GsuInteger)(c->g);
+    pm->gsu.i = gsu;
+    if (locallevel <= pm->level)
+	gsu->unsetfn(pm, x);
+}
+
+/**/
+static double
+sf_getfn(Param pm)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.f);
+    GsuFloat gsu = (GsuFloat)(c->g);
+    if (locallevel >= pm->level)
+	return gsu->getfn(pm);
+    else
+	return 0;
+}
+
+/**/
+static void
+sf_setfn(Param pm, double x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.f);
+    GsuFloat gsu = (GsuFloat)(c->g);
+    if (locallevel == pm->level)
+	gsu->setfn(pm, x);
+    else
+	setfn_error(pm);
+}
+
+/**/
+static void
+sf_unsetfn(Param pm, int x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.f);
+    GsuFloat gsu = (GsuFloat)(c->g);
+    pm->gsu.f = gsu;
+    if (locallevel <= pm->level)
+	gsu->unsetfn(pm, x);
+}
+
+/**/
+static char **
+sa_getfn(Param pm)
+{
+    static char *nullarray = NULL;
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.a);
+    GsuArray gsu = (GsuArray)(c->g);
+    if (locallevel >= pm->level)
+	return gsu->getfn(pm);
+    else
+	return &nullarray;
+}
+
+/**/
+static void
+sa_setfn(Param pm, char **x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.a);
+    GsuArray gsu = (GsuArray)(c->g);
+    if (locallevel == pm->level)
+	gsu->setfn(pm, x);
+    else
+	setfn_error(pm);
+}
+
+/**/
+static void
+sa_unsetfn(Param pm, int x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.a);
+    GsuArray gsu = (GsuArray)(c->g);
+    pm->gsu.a = gsu;
+    if (locallevel <= pm->level)
+	gsu->unsetfn(pm, x);
+}
+
+static HashTable emptytable;
+
+/**/
+static HashTable
+sh_getfn(Param pm)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.h);
+    GsuHash gsu = (GsuHash)(c->g);
+    if (locallevel >= pm->level)
+	return gsu->getfn(pm);
+    else
+	return emptytable;
+}
+
+/**/
+static void
+sh_setfn(Param pm, HashTable x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.h);
+    GsuHash gsu = (GsuHash)(c->g);
+    if (locallevel == pm->level)
+	gsu->setfn(pm, x);
+    else
+	setfn_error(pm);
+}
+
+/**/
+static void
+sh_unsetfn(Param pm, int x)
+{
+    struct gsu_closure *c = (struct gsu_closure *)(pm->gsu.h);
+    GsuHash gsu = (GsuHash)(c->g);
+    pm->gsu.h = gsu;
+    if (locallevel <= pm->level)
+	gsu->unsetfn(pm, x);
+}
+
+/*
+ * How visibility works:
+ *
+ * Upon entering a new function scope, we find all the static parameters
+ * at this locallevel.  Any that we find are marked PM_UNSET.  If they are
+ * already unset, they are also marked as PM_NORESTORE.
+ *
+ * On exit from the scope, we find the same parameters again and remove
+ * the PM_UNSET and PM_NORESTORE flags as appropriate.  We're guaraneed
+ * by makestatic() that PM_NORESTORE won't conflict with anything here.
+ *
+ */
+
+static void
+scopestatic(HashNode hn, int onoff)
+{
+    Param pm = (Param)hn;
+    if (pm->level == locallevel) {
+	if (!is_static(pm))
+	    return;
+	if (onoff == PM_UNSET)
+	    if (pm->node.flags & PM_UNSET)
+		pm->node.flags |= PM_NORESTORE;
+	    else
+		pm->node.flags |= PM_UNSET;
+	else if (!(pm->node.flags & PM_NORESTORE))
+	    pm->node.flags &= ~PM_UNSET;
+	pm->node.flags &= ~PM_NORESTORE;
+    }
+}
+
+static struct funcwrap wrapper[] = {
+    WRAPDEF(wrap_static)
+};
+
+/**/
+static int
+wrap_static(Eprog prog, FuncWrap w, char *name)
+{
+    static int wraplevel = 0;
+
+    if (wraplevel < locallevel /* && strcmp(name, "(anon)") != 0 */) {
+	int owl = wraplevel;
+	wraplevel = locallevel;
+	scanhashtable(paramtab, 0, 0, 0, scopestatic, PM_UNSET);
+	runshfunc(prog, w, name);
+	scanhashtable(paramtab, 0, 0, 0, scopestatic, 0);
+	wraplevel = owl;
+	return 0;
+    }
+    return 1;
+}
+
+static HashNode (*getparamnode) _((HashTable, const char *));
+
+/**/
+static HashNode
+getstaticnode(HashTable ht, const char *nam)
+{
+    HashNode hn = getparamnode(ht, nam);
+    Param pm = (Param) hn;
+
+    while (!fakelevel && pm && locallevel > pm->level && is_static(pm))
+	pm = pm->old;
+    return (HashNode)pm;
+}
+
+/**/
+static HashNode
+getstaticnode2(HashTable ht, const char *nam)
+{
+    /* getparamnode() would follow autoloads, we must not do that here */
+    HashNode hn = gethashnode2(ht, nam);
+    Param pm = (Param) hn;
+
+    while (!fakelevel && pm && locallevel > pm->level && is_static(pm))
+	pm = pm->old;
+    return (HashNode)pm;
+}
+
+/**/
+static void
+printstaticnode(HashNode hn, int printflags)
+{
+    Param pm = (Param) hn;
+    while (pm && (!fakelevel ||
+		  (fakelevel > pm->level && (pm->node.flags & PM_UNSET))) &&
+	   locallevel > pm->level && is_static(pm))
+	pm = pm->old;
+    /* Ideally, we'd print the word "static" here instead of "typeset"
+     * when the parameter is in fact a static, but that would require
+     * re-implementing the entirety of printparamnode(). */
+    if (pm)
+	printparamnode((HashNode)pm, printflags);
+}
+
+/*
+ * Standard module configuration/linkage
+ */
+
+static struct builtin bintab[] = {
+    /* Copied almost verbatim from BUILTIN("local") */
+    BUILTIN("static", BINF_PLUSOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL | BINF_ASSIGN, (HandlerFunc)bin_static, 0, -1, 0, "AE:%F:%HL:%R:%UZ:%ahi:%lprtux", NULL)
+};
+
+static struct features module_features = {
+    bintab, sizeof(bintab)/sizeof(*bintab),
+    NULL, 0,
+    NULL, 0,
+    NULL, 0,
+    0
+};
+
+/**/
+int
+setup_(UNUSED(Module m))
+{
+    /* Horrible, horrible hack */
+    getparamnode = realparamtab->getnode;
+    realparamtab->getnode = getstaticnode;
+    realparamtab->getnode2 = getstaticnode2;
+    realparamtab->printnode = printstaticnode;
+
+    return 0;
+}
+
+/**/
+int
+features_(Module m, char ***features)
+{
+    *features = featuresarray(m, &module_features);
+    return 0;
+}
+
+/**/
+int
+enables_(Module m, int **enables)
+{
+    return handlefeatures(m, &module_features, enables);
+}
+
+/**/
+int
+boot_(Module m)
+{
+    emptytable = newparamtable(1, "static");
+    return addwrapper(m, wrapper);
+}
+
+/**/
+int
+cleanup_(Module m)
+{
+    realparamtab->getnode = getparamnode;
+    realparamtab->getnode2 = gethashnode2;
+    realparamtab->printnode = printparamnode;
+
+    deletewrapper(m, wrapper);
+    return setfeatureenables(m, &module_features, NULL);
+}
+
+/**/
+int
+finish_(UNUSED(Module m))
+{
+    deletehashtable(emptytable);
+    return 0;
+}
diff --git a/Src/Modules/param_static.mdd b/Src/Modules/param_static.mdd
new file mode 100644
index 0000000..7db4e4d
--- /dev/null
+++ b/Src/Modules/param_static.mdd
@@ -0,0 +1,7 @@
+name=zsh/param/static
+link=dynamic
+load=no
+
+autofeatures="b:static"
+
+objects="param_static.o"
diff --git a/Src/builtin.c b/Src/builtin.c
index 97022ad..4cbc4f0 100644
--- a/Src/builtin.c
+++ b/Src/builtin.c
@@ -2331,7 +2331,8 @@ typeset_single(char *cname, char *pname, Param pm, UNUSED(int func),
 	} else if ((on & PM_LOCAL) && locallevel) {
 	    *subscript = 0;
 	    pm = (Param) (paramtab == realparamtab ?
-			  gethashnode2(paramtab, pname) :
+			  /* getnode2() to avoid autoloading */
+			  paramtab->getnode2(paramtab, pname) :
 			  paramtab->getnode(paramtab, pname));
 	    *subscript = '[';
 	    if (!pm || pm->level != locallevel) {
@@ -2819,11 +2820,12 @@ bin_typeset(char *name, char **argv, LinkList assigns, Options ops, int func)
     /* Take arguments literally.  Don't glob */
     while ((asg = getasg(&argv, assigns))) {
 	HashNode hn = (paramtab == realparamtab ?
-		       gethashnode2(paramtab, asg->name) :
+		       /* getnode2() to avoid autoloading */
+		       paramtab->getnode2(paramtab, asg->name) :
 		       paramtab->getnode(paramtab, asg->name));
 	if (OPT_ISSET(ops,'p')) {
 	    if (hn)
-		printparamnode(hn, printflags);
+		paramtab->printnode(hn, printflags);
 	    else {
 		zwarnnam(name, "no such variable: %s", asg->name);
 		returnval = 1;
@@ -3313,7 +3315,8 @@ bin_unset(char *name, char **argv, Options ops, int func)
 	    *ss = 0;
 	}
 	pm = (Param) (paramtab == realparamtab ?
-		      gethashnode2(paramtab, s) :
+		      /* getnode2() to avoid autoloading */
+		      paramtab->getnode2(paramtab, s) :
 		      paramtab->getnode(paramtab, s));
 	/*
 	 * Unsetting an unset variable is not an error.
diff --git a/Src/params.c b/Src/params.c
index de151a4..8958eea 100644
--- a/Src/params.c
+++ b/Src/params.c
@@ -447,7 +447,7 @@ newparamtable(int size, char const *name)
     ht->cmpnodes    = strcmp;
     ht->addnode     = addhashnode;
     ht->getnode     = getparamnode;
-    ht->getnode2    = getparamnode;
+    ht->getnode2    = gethashnode2;
     ht->removenode  = removehashnode;
     ht->disablenode = NULL;
     ht->enablenode  = NULL;
@@ -868,6 +868,7 @@ createparam(char *name, int flags)
 
     if (name != nulstring) {
 	oldpm = (Param) (paramtab == realparamtab ?
+			 /* gethashnode2() for direct table read */
 			 gethashnode2(paramtab, name) :
 			 paramtab->getnode(paramtab, name));
 
@@ -3035,7 +3036,8 @@ unsetparam(char *s)
 
     queue_signals();
     if ((pm = (Param) (paramtab == realparamtab ?
-		       gethashnode2(paramtab, s) :
+		       /* getnode2() to avoid autoloading */
+		       paramtab->getnode2(paramtab, s) :
 		       paramtab->getnode(paramtab, s))))
 	unsetparam_pm(pm, 0, 1);
     unqueue_signals();
diff --git a/Test/V10static.ztst b/Test/V10static.ztst
new file mode 100644
index 0000000..11f54e0
--- /dev/null
+++ b/Test/V10static.ztst
@@ -0,0 +1,263 @@
+# Tests for the zsh/param/static module
+
+%prep
+
+ zmodload zsh/param/static
+
+%test
+
+ typeset scalar_test=toplevel
+ () {
+  print $scalar_test
+  static scalar_test
+  print $+scalar_test
+  unset scalar_test
+  print $+scalar_test
+ }
+ print $scalar_test
+0:basic scope hiding
+>toplevel
+>1
+>0
+>toplevel
+
+ typeset scalar_test=toplevel
+ print $scalar_test
+ () {
+  static scalar_test=function
+  print $scalar_test
+ }
+ print $scalar_test
+0:enter and exit a scope
+>toplevel
+>function
+>toplevel
+
+ print $+unset_test
+ () {
+  static unset_test
+  print $+unset_test
+  unset_test=setme
+  print $unset_test
+ }
+ print $+unset_test
+0:variable defined only in scope
+>0
+>1
+>setme
+>0
+
+ # Depends on zsh-5.0.9 typeset keyword
+ typeset -a array_test=(top level)
+ () {
+  static -a array_test
+  array_test=(in function)
+  () {
+   static array_test
+   print $+array_test
+  }
+  print $array_test
+ }
+ print $array_test
+0:nested scope with different type, correctly restored
+>1
+>in function
+>top level
+
+ typeset -a array_test=(top level)
+ () {
+  static array_test
+  array_test=(in function)
+ }
+1:type of static may not be changed by assignment
+?(anon):2: array_test: attempt to assign array value to non-array
+
+ typeset -A hash_test=(top level)
+ () {
+  setopt localoptions noglob
+  static hash_test[top]
+ }
+1:associative array fields may not be static
+?(anon):static:2: hash_test[top]: can't create local array elements
+
+ () {
+  static path
+ }
+1:tied params may not be static, part 1
+?(anon):static:1: can't change scope of existing param: path
+
+ () {
+  static PATH
+ }
+1:tied params may not be static, part 2
+?(anon):static:1: can't change scope of existing param: PATH
+
+ () {
+  static -h path
+  print X$path
+ }
+0:statics may hide tied paramters
+>X
+
+ # Deliberate type mismatch here
+ typeset -a hash_test=(top level)
+ typeset -p hash_test
+ inner () {
+  static -p hash_test
+  print ${(t)hash_test} ${(kv)hash_test}
+ }
+ outer () {
+  static -A hash_test; hash_test=(in function)
+  typeset -p hash_test
+  inner
+ }
+ outer
+ print ${(kv)hash_test}
+0:static hides value from surrounding scope in nested scope
+>typeset -a hash_test
+>hash_test=( top level )
+>typeset -A hash_test
+>hash_test=( in function )
+>typeset -a hash_test
+>hash_test=( top level )
+>array-local top level
+>top level
+F:note "typeset" rather than "static" in output from outer
+
+ () {
+  static -a array_test
+  local array_test=scalar
+ }
+1:static cannot be re-declared as local
+?(anon):local:2: array_test: inconsistent type for assignment
+
+ () {
+  local hash_test=scalar
+  static -A hash_test
+ }
+1:local cannot be re-declared as static
+?(anon):static:2: can't change scope of existing param: hash_test
+
+ inner () {
+  print $+scalar_test
+  $ZTST_testdir/../Src/zsh -fc 'print X $scalar_test'
+ }
+ () {
+  static -x scalar_test=whaat
+  $ZTST_testdir/../Src/zsh -fc 'print X $scalar_test'
+  inner
+  print Y $scalar_test
+ }
+0:exported static behaves like a local, part 1
+>X whaat
+>0
+>X whaat
+>Y whaat
+
+ inner () {
+  typeset -p array_test
+  $ZTST_testdir/../Src/zsh -fc 'print X $array_test'
+ }
+ () {
+  static -ax array_test; array_test=(whaat)
+  print Y $array_test
+  $ZTST_testdir/../Src/zsh -fc 'print X $array_test'
+  inner
+ }
+0:exported static behaves like a local, part 2 (arrays do not export)
+?inner:typeset:1: no such variable: array_test
+>Y whaat
+>X
+>X
+
+ inner () {
+  print $+scalar_test
+  $ZTST_testdir/../Src/zsh -fc 'print X $scalar_test'
+ }
+ () {
+  static scalar_test=whaat
+  export scalar_test
+  $ZTST_testdir/../Src/zsh -fc 'print X $scalar_test'
+  inner
+  () {
+   print $+scalar_test
+   $ZTST_testdir/../Src/zsh -fc 'print X $scalar_test'
+  }
+  print Y $scalar_test
+ }
+0:exported static behaves like a local, part 3 (export does not change scope)
+>X whaat
+>0
+>X whaat
+>0
+>X whaat
+>Y whaat
+
+ typeset -A hash_test=(top level)
+ () {
+  static -A hash_test; hash_test=(in function)
+  () {
+   print X ${(kv)hash_test}
+  }
+  print Y ${(kv)hash_test}
+ }
+ print ${(kv)hash_test}
+0:statics are not visible in anonymous functions, part 1
+>X top level
+>Y in function
+>top level
+
+ typeset -A hash_test=(top level)
+ () {
+  static -A hash_test; hash_test=(in function)
+  () {
+   print X ${(kv)hash_test}
+   hash_test[in]=deeper
+  }
+  print Y ${(kv)hash_test}
+ }
+ print ${(okv)hash_test}
+0:statics are not visible in anonymous functions, part 2
+>X top level
+>Y in function
+>deeper in level top
+
+ typeset -A hash_test=(top level)
+ () {
+  static -a array_test; array_test=(in function)
+  static -A hash_test; hash_test=($array_test)
+  () {
+   print X ${(kv)hash_test}
+   hash_test=(even deeper)
+   array_test+=(${(kv)hash_test})
+  }
+  print Y ${(kv)hash_test} Z $array_test
+ }
+ print ${(kv)hash_test}
+0:statics are not visible in anonymous functions, part 3
+>X top level
+>Y in function Z in function
+>even deeper
+
+ typeset -A hash_test=(top level)
+ () {
+  static -A hash_test; hash_test=(in function)
+  () {
+   print X ${(kv)hash_test}
+   unset hash_test
+  }
+  print Y ${(kv)hash_test}
+ }
+ print ${(t)hash_test} ${(kv)hash_test}
+0:statics are not visible in anonymous functions, part 4
+>X top level
+>Y in function
+>
+
+ # Subshell because otherwise this silently dumps core when broken
+ ( () { static SECONDS } )
+1:special parameters cannot be made static
+?(anon):static: can't change scope of existing param: SECONDS
+
+ () { static -h SECONDS }
+0:static parameter may hide a special parameter

Re: Proof of concept: "static" parameter scope

[Peter Stephenson]

On Thu, 24 Sep 2015 19:23:05 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> Modules can't declare keywords, so this isn't able to support the new
> array assignment syntax used by "local" et al.  However, the entry point
> is coded to match that call signature.

I think the biggest issue with fixing this is it's going to change the
"features" interface to modules as we'll have to add a feature type that
defines a typeset-style interface (and a corresponding reserved word,
but I think we can tie the two together).  But fiddling with the system
to get a module added is already tough enough that that may not cause
much real practical trouble.

Typically, I think modules would then do what "typeset" effectively does
(hard-wired rather than via features) and declare both a builtin and an
enhanced version as separate features, and we should probably mention
this in the development guidelines.

Not sure what to do about printparamnode().

pws

Re: Proof of concept: "static" parameter scope

[Bart Schaefer]

On Sep 25, 10:15am, Peter Stephenson wrote:
} Subject: Re: Proof of concept: "static" parameter scope
}
} On Thu, 24 Sep 2015 19:23:05 -0700
} Bart Schaefer <schaefer@brasslantern.com> wrote:
} > Modules can't declare keywords, so this isn't able to support the new
} > array assignment syntax used by "local" et al.  However, the entry point
} > is coded to match that call signature.
} 
} I think the biggest issue with fixing this is it's going to change the
} "features" interface to modules as we'll have to add a feature type that
} defines a typeset-style interface (and a corresponding reserved word,
} but I think we can tie the two together).

Or ... 


diff --git a/Doc/Zsh/mod_static.yo b/Doc/Zsh/mod_static.yo
index 4342a92..6b461e7 100644
--- a/Doc/Zsh/mod_static.yo
+++ b/Doc/Zsh/mod_static.yo
@@ -36,6 +36,11 @@ parameter.  This may change in the future.
 )
 enditem()
 
+The module also adds the option `tt(-S)' to the `tt(local)' builtin.
+Using `tt(local -S)' is equivalent to calling `tt(static)', but allows
+use of array assignment (parenthesized word list) syntax in the
+declaration.
+
 Parameters declared with tt(static) have the following properties:
 ifnzman()
 startitemize()
diff --git a/Src/Modules/param_static.c b/Src/Modules/param_static.c
index b6c2403..c5354b2 100644
--- a/Src/Modules/param_static.c
+++ b/Src/Modules/param_static.c
@@ -172,6 +172,13 @@ bin_static(char *nam, char **args, LinkList assigns, Options ops, int func)
     int from_typeset = 1;
     makestatic_error = 0;
 
+    if (!OPT_ISSET(ops, 'S'))
+	return bin_typeset(nam, args, assigns, ops, func);
+    else if (OPT_ISSET(ops, 'T')) {
+	zwarn("bad option: -T");
+	return 1;
+    }
+
     if (locallevel == 0) {
 	if (isset(WARNCREATEGLOBAL))
 	    zwarnnam(nam, "invalid local scope, using globals");
@@ -495,7 +502,7 @@ printstaticnode(HashNode hn, int printflags)
 
 static struct builtin bintab[] = {
     /* Copied almost verbatim from BUILTIN("local") */
-    BUILTIN("static", BINF_PLUSOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL | BINF_ASSIGN, (HandlerFunc)bin_static, 0, -1, 0, "AE:%F:%HL:%R:%UZ:%ahi:%lprtux", NULL)
+    BUILTIN("static", BINF_PLUSOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL | BINF_ASSIGN, (HandlerFunc)bin_static, 0, -1, 0, "AE:%F:%HL:%R:%SUZ:%ahi:%lprtux", "S")
 };
 
 static struct features module_features = {
@@ -506,16 +513,27 @@ static struct features module_features = {
     0
 };
 
+/* Same optstr as BUILTIN("local") but with 'S' added */
+static char *optstr_local = "AE:%F:%HL:%R:%STUZ:%ahi:%lprtux";
+static struct builtin save_local;
+
 /**/
 int
 setup_(UNUSED(Module m))
 {
+    HashNode hn = builtintab->getnode(builtintab, "local");
+
     /* Horrible, horrible hack */
     getparamnode = realparamtab->getnode;
     realparamtab->getnode = getstaticnode;
     realparamtab->getnode2 = getstaticnode2;
     realparamtab->printnode = printstaticnode;
 
+    /* Even more horrible hack */
+    save_local = *(Builtin)hn;
+    ((Builtin)hn)->handlerfunc = bintab[0].handlerfunc;
+    ((Builtin)hn)->optstr = optstr_local;
+
     return 0;
 }
 
@@ -546,6 +564,9 @@ boot_(Module m)
 int
 cleanup_(Module m)
 {
+    HashNode hn = builtintab->getnode(builtintab, "local");
+    *(Builtin)hn = save_local;
+    
     realparamtab->getnode = getparamnode;
     realparamtab->getnode2 = gethashnode2;
     realparamtab->printnode = printparamnode;
diff --git a/Test/V10static.ztst b/Test/V10static.ztst
index 11f54e0..cc69db7 100644
--- a/Test/V10static.ztst
+++ b/Test/V10static.ztst
@@ -50,8 +50,7 @@
  # Depends on zsh-5.0.9 typeset keyword
  typeset -a array_test=(top level)
  () {
-  static -a array_test
-  array_test=(in function)
+  local -Sa array_test=(in function)
   () {
    static array_test
    print $+array_test
@@ -107,7 +106,7 @@
   print ${(t)hash_test} ${(kv)hash_test}
  }
  outer () {
-  static -A hash_test; hash_test=(in function)
+  local -SA hash_test=(in function)
   typeset -p hash_test
   inner
  }
@@ -159,7 +158,7 @@ F:note "typeset" rather than "static" in output from outer
   $ZTST_testdir/../Src/zsh -fc 'print X $array_test'
  }
  () {
-  static -ax array_test; array_test=(whaat)
+  local -Sax array_test=(whaat)
   print Y $array_test
   $ZTST_testdir/../Src/zsh -fc 'print X $array_test'
   inner
@@ -195,7 +194,7 @@ F:note "typeset" rather than "static" in output from outer
 
  typeset -A hash_test=(top level)
  () {
-  static -A hash_test; hash_test=(in function)
+  local -SA hash_test=(in function)
   () {
    print X ${(kv)hash_test}
   }
@@ -209,7 +208,7 @@ F:note "typeset" rather than "static" in output from outer
 
  typeset -A hash_test=(top level)
  () {
-  static -A hash_test; hash_test=(in function)
+  local -SA hash_test=(in function)
   () {
    print X ${(kv)hash_test}
    hash_test[in]=deeper
@@ -224,8 +223,8 @@ F:note "typeset" rather than "static" in output from outer
 
  typeset -A hash_test=(top level)
  () {
-  static -a array_test; array_test=(in function)
-  static -A hash_test; hash_test=($array_test)
+  local -Sa array_test=(in function)
+  local -SA hash_test=($array_test)
   () {
    print X ${(kv)hash_test}
    hash_test=(even deeper)
@@ -241,7 +240,7 @@ F:note "typeset" rather than "static" in output from outer
 
  typeset -A hash_test=(top level)
  () {
-  static -A hash_test; hash_test=(in function)
+  local -SA hash_test=(in function)
   () {
    print X ${(kv)hash_test}
    unset hash_test

Re: Proof of concept: "static" parameter scope

[Oliver Kiddle]

Bart wrote:
> 
> The patch below creates a module "zsh/param/static" which supports a
> single builtin command "static".  The included doc section describes
> "static" in detail, but briefly, it works like the "local" builtin
> except that variables so declared are not dynamically visible to any
> called functions.

Very nice.

> "Called functions" includes recursive calls to the function itself, so
> this doesn't work like C "static".  Therefore I'm in the market for a
> better name.

The trouble with "static" is that people with a C or C++ background will
expect something different. It might be better to find a new word in the
thesaurus that doesn't carry the baggage of another common meaning. Note
that, as you mention in the documentation patch, ksh93 has a typeset
-S option which does do C like static variables. Aside from confusing
users, the Zsh use of the term "parameter" seems even more tenuous when
applied to lexically scoped variables because they can't be used as
named parameters. So instead of "static" I would simply suggest "var".

This name does take the emphasis away from the scope but that could be a
good thing in terms of encouraging it's use. It is actually "local" with its
obscure dynamic scoping that really needs the more elaborate explanation
in the documentation. And "typeset" isn't a good name because
typesetting evokes things like LaTeX.

We might also want to hold on to -S for real static variables or perhaps
file backed "shared" variables (as someone recently requested).

> There are some tricks played to avoid creating new PM_* flag bits.  If
> this ceased to be a module and became a "real" builtin (something about
> which I'm ambivalent) there are parts that could be simplified by using
> another flag bit.

I like the fact that this hasn't added a load of extra complexity to
bin_typeset(). An extra flag wouldn't be too invasive if it helps the
implementation. I can't think of a reason to suggest that it should
cease to be a module if the implementation hasn't required it. It's also
worth having the reserved word form as per your later patch even if it
compromises the purity of the module separation.

Despite throwing quite a few odd things at it, I've not been able to
break it.

It'd be nice if typeset -p could tell you that it is a static variable.
Theres also ${(t)...}.

> +Although var(name)tt(=)var(value) sytax may be used in the argument list

typo: syntax

> +itemiz(An exported static remains in the environment of inner scopes but
> +appears unset for the current shell in those scopes.)
> +enditemize()

That doesn't seem ideal. It also seems that can be further affected if
the variable is declared local in some inner scope.

Oliver

Re: Proof of concept: "static" parameter scope

[Roman Neuhauser]

# okiddle@yahoo.co.uk / 2015-09-28 19:04:59 +0200:
> Bart wrote:
> > "Called functions" includes recursive calls to the function itself, so
> > this doesn't work like C "static".  Therefore I'm in the market for a
> > better name.
> 
> The trouble with "static" is that people with a C or C++ background will
> expect something different. It might be better to find a new word in the
> thesaurus that doesn't carry the baggage of another common meaning. Note
> that, as you mention in the documentation patch, ksh93 has a typeset
> -S option which does do C like static variables. Aside from confusing
> users, the Zsh use of the term "parameter" seems even more tenuous when
> applied to lexically scoped variables because they can't be used as
> named parameters. So instead of "static" I would simply suggest "var".

agree wholeheartedly with everything Oliver said.

one nitpick: "var" has no semantics regarding scope, which is not a
problem per se, but with 'local' doing something (naively) unexpected,
maybe a stronger signal would be in order?

i'd maybe suggest "private" instead, but the sweet succintness of "var"
is really attractive.  hmm, what about "my"?
 
-- 
roman

Re: Proof of concept: "static" parameter scope

[Mikael Magnusson]

On Mon, Sep 28, 2015 at 7:04 PM, Oliver Kiddle <okiddle@yahoo.co.uk> wrote:
> Bart wrote:
>>
>> The patch below creates a module "zsh/param/static" which supports a
>> single builtin command "static".  The included doc section describes
>> "static" in detail, but briefly, it works like the "local" builtin
>> except that variables so declared are not dynamically visible to any
>> called functions.
>
> Very nice.
>
>> "Called functions" includes recursive calls to the function itself, so
>> this doesn't work like C "static".  Therefore I'm in the market for a
>> better name.
>
> The trouble with "static" is that people with a C or C++ background will
> expect something different. It might be better to find a new word in the
> thesaurus that doesn't carry the baggage of another common meaning. Note
> that, as you mention in the documentation patch, ksh93 has a typeset
> -S option which does do C like static variables. Aside from confusing
> users, the Zsh use of the term "parameter" seems even more tenuous when
> applied to lexically scoped variables because they can't be used as
> named parameters. So instead of "static" I would simply suggest "var".
>
> This name does take the emphasis away from the scope but that could be a
> good thing in terms of encouraging it's use. It is actually "local" with its
> obscure dynamic scoping that really needs the more elaborate explanation
> in the documentation. And "typeset" isn't a good name because
> typesetting evokes things like LaTeX.

I think the "local" scope is actually extremely simple to understand,
it goes out of scope when the current function ends, period. In C it
is much more complicated, it goes out of scope when you leave a block,
it goes out of scope when you call a function but then goes back into
scope when you return, possibly multiple times, you can 'goto' out and
in to scope. I suppose it's also a matter of perspective.

Also, "static" already means at least 3 different things in C, it
doesn't make any sense to expect it to mean any particular one of
these things in zsh?

What about "private" instead of static though? If the intention is to
hide it from called functions, it is somewhat closer to that concept,
if not just the normal word.

-- 
Mikael Magnusson

Re: Proof of concept: "static" parameter scope

[Bart Schaefer]

Rearranging a bit ...

On Sep 28,  7:04pm, Oliver Kiddle wrote:
}
} Despite throwing quite a few odd things at it, I've not been able to
} break it.

I tried to make Test/V10* pretty comprehensive.

} It'd be nice if typeset -p could tell you that it is a static variable.

Cf. previous discussion about printparamnode(), which as it stands has
always gotten "local" wrong as well.  Or more accurately, given that
"typeset" is equivalent to "local" when called in function scope, it
is "typeset" of globals that "typeset -p" has always gotten wrong.

} Theres also ${(t)...}.

Yes, this could also be fixed with an actual flag bit -- or by adding
another callback to the gsu structure, which could also potentially be
used to fix printparamnode(), but is a fairly large change.

} > +itemiz(An exported static remains in the environment of inner scopes but
} > +appears unset for the current shell in those scopes.)
} > +enditemize()
} 
} That doesn't seem ideal.

Agreed, and the best way to handle it would be that statics can't be
exported, but since variables can be declared and exported in two
independent steps, I didn't want to meddle that deeply with "export".

At that point though, I hadn't had the idea of swapping out the actual
handler in the builtintab ... but I really think that particular hack
is to be strongly discouraged.

} It also seems that can be further affected if
} the variable is declared local in some inner scope.

Indeed, nothing prevents creating a local with the same name as a static
as long as the scope is not the same, and exported locals behave as they
always did.

As for naming ...

On Sep 28,  7:04pm, Oliver Kiddle wrote:
}
} The trouble with "static" is that people with a C or C++ background will
} expect something different. It might be better to find a new word in the
} thesaurus that doesn't carry the baggage of another common meaning.

On Sep 28,  9:42pm, Mikael Magnusson wrote:
} 
} Also, "static" already means at least 3 different things in C, it
} doesn't make any sense to expect it to mean any particular one of
} these things in zsh?

The meaning in this zsh case is obviously "not dynamic" (because all
other variables except the positional parameters have dynamic scope).
C/C++ "static" encompasses both scoping and memory management semantics
and is mostly derived from the latter.

On Sep 28,  7:04pm, Oliver Kiddle wrote:
}
} [...]  the Zsh use of the term "parameter" seems even more tenuous when
} applied to lexically scoped variables because they can't be used as
} named parameters. So instead of "static" I would simply suggest "var".

Possibly OK for a keyword, not so good for the name of a module.  One
ought to be able to get some idea of what's going to happen when the
module is loaded; "zsh/param/var" gives no hint, and looks redundant.

On Sep 28,  7:58pm, Roman Neuhauser wrote:
} Subject: Re: Proof of concept: "static" parameter scope
}
} one nitpick: "var" has no semantics regarding scope, which is not a
} problem per se, but with 'local' doing something (naively) unexpected,
} maybe a stronger signal would be in order?
} 
} i'd maybe suggest "private" instead, but the sweet succintness of "var"
} is really attractive.  hmm, what about "my"?

This is sort of like perl "my" but perl syntax lends itself to that as
a keyword a little better; "my -slew-of-option-letters thing" doesn't
roll so well and "zsh/param/my" is only a little better than "var".

On Sep 28,  9:42pm, Mikael Magnusson wrote:
} 
} What about "private" instead of static though? If the intention is to
} hide it from called functions, it is somewhat closer to that concept,
} if not just the normal word.

Once again it's not the same as C++ "private" if that's a concern, but
I do lean in this direction.

Make up a completely new term?  "limited" ?  "bounded" ?  "scoped" ?

Re: Proof of concept: "static" parameter scope

[Peter Stephenson]

On Mon, 28 Sep 2015 18:23:27 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 28,  9:42pm, Mikael Magnusson wrote:
> } 
> } What about "private" instead of static though? If the intention is to
> } hide it from called functions, it is somewhat closer to that concept,
> } if not just the normal word.
> 
> Once again it's not the same as C++ "private" if that's a concern, but
> I do lean in this direction.
> 
> Make up a completely new term?  "limited" ?  "bounded" ?  "scoped" ?

I quite like "private" and I think the C++ meaning is distant enough
that it's not really a confusion.  It's the only name that suggests the
scope is limited to right here.

"var" sounds a bit much like the name of a temporary variable.

If you really wanted to emphaise the C connection, the name would be
"auto", but that doesn't make sense for zsh since it doesn't distinguish
static from dynamic scope.

pws

Re: Proof of concept: "static" parameter scope

[Daniel Shahaf]

Bart Schaefer wrote on Mon, Sep 28, 2015 at 18:23:27 -0700:
> Make up a completely new term?  "limited" ?  "bounded" ?  "scoped" ?

disinherit?

Although the term "inherit" is usually found in object-oriented
contexts...

Re: Proof of concept: "static" parameter scope

[Andrew Janke]

On 9/28/15 1:58 PM, Roman Neuhauser wrote:
> # okiddle@yahoo.co.uk / 2015-09-28 19:04:59 +0200:
>> Bart wrote:
>>> "Called functions" includes recursive calls to the function itself, so
>>> this doesn't work like C "static".  Therefore I'm in the market for a
>>> better name.
>> The trouble with "static" is that people with a C or C++ background will
>> expect something different. It might be better to find a new word in the
>> thesaurus that doesn't carry the baggage of another common meaning. Note
>> that, as you mention in the documentation patch, ksh93 has a typeset
>> -S option which does do C like static variables. Aside from confusing
>> users, the Zsh use of the term "parameter" seems even more tenuous when
>> applied to lexically scoped variables because they can't be used as
>> named parameters. So instead of "static" I would simply suggest "var".
> agree wholeheartedly with everything Oliver said.
>
> one nitpick: "var" has no semantics regarding scope, which is not a
> problem per se, but with 'local' doing something (naively) unexpected,
> maybe a stronger signal would be in order?
>
> i'd maybe suggest "private" instead, but the sweet succintness of "var"
> is really attractive.  hmm, what about "my"?
>   
I'm a fan of "my". Perl's "my" and "local" are exactly the example I use 
when explaining the behavior of zsh "local" variables to new users. This 
new scope seems to correspond pretty well to Perl's "my" lexical scoping.

Or maybe "auto"? The C "auto" storage class (the default one, scoped to 
a block in a function activation) corresponds pretty well to this new 
static scope, since it's scoped to a single function activation / call 
stack frame (as opposed to a stack frame and all its child frames).

Or maybe "reallylocal"?

Re: Proof of concept: "static" parameter scope

[Bart Schaefer]

On Sep 29, 10:34pm, Daniel Shahaf wrote:
}
} Bart Schaefer wrote on Mon, Sep 28, 2015 at 18:23:27 -0700:
} > Make up a completely new term?  "limited" ?  "bounded" ?  "scoped" ?
} 
} disinherit?

Sounds like something you do after the fact, like "export".  Which I
suppose is a way it could have been implemented -- first declare the
local and then impose scoping.  That's approximately what happens
behind the scenes here anyway.  Hmm.  "isolate"?

On Sep 29,  7:31pm, Andrew Janke wrote:
}
} I'm a fan of "my". Perl's "my" and "local" are exactly the example I use 
} when explaining the behavior of zsh "local" variables to new users. This 
} new scope seems to correspond pretty well to Perl's "my" lexical scoping.

I like the concept of "my" but have never been thrilled with the name.

} Or maybe "auto"? The C "auto" storage class (the default one, scoped to 
} a block in a function activation) corresponds pretty well to this

Yes but again "auto" refers (as you note) to storage ... all zsh locals
are "auto" in that sense.  This needs to describe visibility.

Really, "local -S" (or some other letter if we're trying to save -S for
something) is the ideal thing.  The only reason we even need another
name is to set up the handler function.

Since PWS has voted in favor of "private" I think I'm going to revise
the patch in favor of that.  There's also no "-P" option of typeset in
either bash or ksh, so although I don't like the (dis)similarity with
"-p" I think we can go with "local -P" for "private".

Re: Proof of concept: "static" parameter scope

[Peter Stephenson]

On Fri, 25 Sep 2015 10:15:40 +0100
Peter Stephenson <p.stephenson@samsung.com> wrote:
> On Thu, 24 Sep 2015 19:23:05 -0700
> Bart Schaefer <schaefer@brasslantern.com> wrote:
> > Modules can't declare keywords, so this isn't able to support the new
> > array assignment syntax used by "local" et al.  However, the entry point
> > is coded to match that call signature.
> 
> I think the biggest issue with fixing this is it's going to change the
> "features" interface to modules as we'll have to add a feature type that
> defines a typeset-style interface (and a corresponding reserved word,
> but I think we can tie the two together).

I wonder if it's easier than that.  If the builtin has the BINF_ASSIGN
flag then the module code could do that automatically without the module
needing to declare a different sort of feature.  As long as the
interface is correct, simply adding a reserved word with the token
TYPESET should be enough.

The trouble is it needs to know this early, because anything using this
builtin with the assignment interface needs to be parsed properly.  In
particular, it would have to be registered as a reserved word at any
point it wsa marked for autoload.  But maybe an additional flag is good
enough for that.

pws

Re: Proof of concept: "static" parameter scope

[Bart Schaefer]

On Sep 30,  8:38pm, Peter Stephenson wrote:
} Subject: Re: Proof of concept: "static" parameter scope
}
} I wonder if it's easier than that.  If the builtin has the BINF_ASSIGN
} flag then the module code could do that automatically without the module
} needing to declare a different sort of feature.  As long as the
} interface is correct, simply adding a reserved word with the token
} TYPESET should be enough.

Indeed, simply calling reswdtab->addnode() from setup_() works fine.  It
doesn't really even need to be automated by the module code.

} The trouble is it needs to know this early, because anything using this
} builtin with the assignment interface needs to be parsed properly.  In
} particular, it would have to be registered as a reserved word at any
} point it wsa marked for autoload.

That is indeed a problem.  It actually works much better to have the
module overload "local" (my hack to add the -S [now -P] switch) because
then the order of parsing the calling code and loading the module are
independent.

Also there are some interesting side-effects if you load the module,
then define a function that uses the new keyword, and then unload the
module again and replace the keyword with a function or an external
command.

Here's a confusing thing that's indirectly related:

torch% disable -r local
torch% zxxx() { local x=(a b c); print $x }
torch% 

Eh?  Why is that not a syntax error?  The reserved word "local" has
been disabled, yet x=(a b c) is still parsed as an assignment.

torch% zxxx
zxxx: number expected

Enabling the builtin again does not fix this:

torch% enable -r local
torch% zxxx
zxxx: number expected

Re: Proof of concept: "static" parameter scope

[Peter Stephenson]

On Wed, 30 Sep 2015 17:27:48 -0700
Bart Schaefer <schaefer@brasslantern.com> wrote:
> torch% disable -r local
> torch% zxxx() { local x=(a b c); print $x }
> torch% 
> 
> Eh?  Why is that not a syntax error?  The reserved word "local" has
> been disabled, yet x=(a b c) is still parsed as an assignment.

What would make it a syntax error here?  You haven't disabled globbing
flags.  It's only an error when you run it and get...

> torch% zxxx
> zxxx: number expected

It's just the same as

% zyyy() { burble x=(a b c); print $x }
% zyyy
zyyy: number expected

as it should be.  Maybe you'd forgotten that parentheses are parsed so
as not to make white space work as a word separator?

> Enabling the builtin again does not fix this:
> 
> torch% enable -r local
> torch% zxxx
> zxxx: number expected

Obviously, since it parsed the function zxxx when local wasn't a
reserved word and you've done nothing to reparse it.  You need to parse
it again.

% enable -r local
% zxxx() { local x=(a b c); print $x }  
% zxxx
a b c

This is the point of the difference between reserved words (keywords that
cause parsing to behave differently) and builtins (names of commands that
are looked up when the command is about to be executed).

This certainly doesn't help you make a syntactic structure dynamically
loadable, but that's always going to be hairy.  I think our choices are
- document it and live with it
- decide it's too hairy and limit ourselves to autoloading an option to
local etc.
- provide adequate means, probably turned on by default, to load everything
early enough that it just works.  This means the distribution would enable
"private", or whatever, as a keyword out of the box.  If we don't I don't
think this is worth doing.  (Of course you can still use disable -r in am
initialisation file.)
- go a step even further than that and just compile it in (but you can *still*
disable -r).
- or, I suppose, we could compile it in with disable -r applied, but then
we're back with all the gotchas about needing to reparse if you use it.

pws

Autoloaded keywords (Re: Proof of concept: "static" parameter scope)

[Bart Schaefer]

On Oct 3,  8:19pm, Peter Stephenson wrote:
} Subject: Re: Proof of concept: "static" parameter scope
}
} It's just the same as
} 
} % zyyy() { burble x=(a b c); print $x }
} % zyyy
} zyyy: number expected
} 
} as it should be.  Maybe you'd forgotten that parentheses are parsed so
} as not to make white space work as a word separator?

Yes, that's sort of it.  I was expecting a different error in the event
that they'd been interpreted as glob qualifiers, so I assumed they had
not been.

} This certainly doesn't help you make a syntactic structure dynamically
} loadable, but that's always going to be hairy.  I think our choices are
} - provide adequate means, probably turned on by default, to load everything
} early enough that it just works.  This means the distribution would enable
} "private", or whatever, as a keyword out of the box.  If we don't I don't
} think this is worth doing.  (Of course you can still use disable -r in am
} initialisation file.)
} - go a step even further than that and just compile it in (but you can *still*
} disable -r).

The latter would be fine if we're just talking about "private" and as
has been mentioned we could pick up some efficiencies by adding a real
PM_PRIVATE flag bit (though we are running out of PM_* bit positions).

However, if we're talking about generalizing the mechanism, we can't
reasonably assert that we'll enable every module-defined keyword via a
change to the default distribution.

I'm leaning toward

} - document it and live with it

but another thought is to in fact add autoloadable keywords as a module
"feature" type and check every word as it's parsed to see if it needs to
trigger an autoload.  This shouldn't be that much worse than checking
every word to see if it's an alias (and has the same pitfalls ... in
fact in ksh many of what we have harcoded as builtins are implemented
[or at least documented] as aliases).

} - decide it's too hairy and limit ourselves to autoloading an option to
} local etc.

I hadn't considered autoloadable *options* ... how would that work?

Here's yet another idea -- perhaps an extension of "autoload" that
specifies which modules have to be zmodload'd before the function can
be autoloaded and run.  Correspondingly the in-memory function image
would record the set of modules that were loaded when it was parsed,
sort of like sticky options, and would return a runtime error if
those modules were no longer loaded.  (Or perhaps only if it were
unable to REload them, but that might be too much automation.)  We'd
have to be sure that zcompile wrote out that state.  (Does it now
preserve sticky options?)

Re: Proof of concept: "static" parameter scope

[Daniel Shahaf]

Peter Stephenson wrote on Sat, Oct 03, 2015 at 20:19:36 +0100:
> On Wed, 30 Sep 2015 17:27:48 -0700
> Bart Schaefer <schaefer@brasslantern.com> wrote:
> > Enabling the builtin again does not fix this:
> > 
> > torch% enable -r local
> > torch% zxxx
> > zxxx: number expected
> 
> Obviously, since it parsed the function zxxx when local wasn't a
> reserved word and you've done nothing to reparse it.  You need to parse
> it again.
> 
> % enable -r local
> % zxxx() { local x=(a b c); print $x }  
> % zxxx
> a b c

Related issue:

% disable -r local
% zxxx() { local x=(a b c); print $x }
% enable -r local  
% which zxxx
zxxx () {
	local x=(a b c)
	print $x
}

Should it output 'builtin local x=(a b c)'?

Re: Proof of concept: "static" parameter scope

[Bart Schaefer]

On Oct 5,  9:55pm, Daniel Shahaf wrote:
}
} Related issue:
} 
} % disable -r local
} % zxxx() { local x=(a b c); print $x }
} % enable -r local  
} % which zxxx
} zxxx () {
} 	local x=(a b c)
} 	print $x
} }
} 
} Should it output 'builtin local x=(a b c)'?

No, we don't want to go down that road, because "disable local" doesn't
(shouldn't) cause it to start emitting "command local" either.

However, it is worth noting that it's now necessary to disable BOTH the
keyword AND the corresponding builtin if you actually want to find an
external command named "local" by default.

Re: Proof of concept: "static" parameter scope

[Daniel Shahaf]

Bart Schaefer wrote on Mon, Oct 05, 2015 at 15:17:39 -0700:
> On Oct 5,  9:55pm, Daniel Shahaf wrote:
> }
> } Related issue:
> } 
> } % disable -r local
> } % zxxx() { local x=(a b c); print $x }
> } % enable -r local  
> } % which zxxx
> } zxxx () {
> } 	local x=(a b c)
> } 	print $x
> } }
> } 
> } Should it output 'builtin local x=(a b c)'?
> 
> No, we don't want to go down that road, because "disable local" doesn't
> (shouldn't) cause it to start emitting "command local" either.
> 

Okay, I understand the consistency point, but could you explain why
"disable local" shouldn't emit "command local"?

I'm not saying it should do something else; I'm just trying to understand
why the modifier ('builtin' or 'command') should be output in neither case.

> However, it is worth noting that it's now necessary to disable BOTH the
> keyword AND the corresponding builtin if you actually want to find an
> external command named "local" by default.

Cheers,

Daniel

Re: Proof of concept: "static" parameter scope

[Bart Schaefer]

On Oct 5, 10:36pm, Daniel Shahaf wrote:
}
} Okay, I understand the consistency point, but could you explain why
} "disable local" shouldn't emit "command local"?

The task of  which name / functions name / print -R $functions[name]
is to reproduce the text of the function, not its semantics.

Otherwise the next logical step would be for

    emulate zsh -c 'foo() { ... }'
    functions foo

to emit a "setopt" command along with the function definition.  Oh,
but why stop there?  I might have defined the function with all sorts
of setopts in effect, even if I didn't say "emulate" to make them
sticky.  Maybe those should be output as well ...

Re: Proof of concept: "static" parameter scope

[Peter Stephenson]

On Mon, 5 Oct 2015 21:55:14 +0000
Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> % disable -r local
> % zxxx() { local x=(a b c); print $x }
> % enable -r local  
> % which zxxx
> zxxx () {
> 	local x=(a b c)
> 	print $x
> }
> 
> Should it output 'builtin local x=(a b c)'?

There's also the opposite case of loading some standard functions with
the full interface and disabling it to run some stuff of your own in
compatibility mode, for which there's no obvous fix without outputting
an "enable" (and hoping the user hasn't disabled enable to get the
printer control command...)

I'm not sure it's worth a code fix, though a well-placed note might be
useful.

pws

-- 
Peter Stephenson | Principal Engineer Samsung Cambridge Solution Centre
Email: p.stephenson@samsung.com | Phone: +44 1223 434724 |
www.samsung.com