static vs. dynamic scoping

static vs. dynamic scoping

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 2383 bytes --]

On the Austin Group mailing list, David Korn (of ksh93 fame)
complained[1] that bash's 'local' uses dynamic scoping, but that ksh's
'typeset' uses static scoping, and argued that static scoping is saner
since it matches the behavior of declarative languages like C and Java
(dynamic scoping mainly matters in functional languages like lisp):

[1]
https://www.opengroup.org/sophocles/show_mail.tpl?CALLER=show_archive.tpl&source=L&listname=austin-group-l&id=14951

I'm trying to standardize the notion of local variables for the next
revision of POSIX, but before I can do so, I need some feedback on two
general aspects:

1. Implementation aspect:
  How hard would it be to add static scoping to zsh?
  Is it something that can be added in addition to dynamic scoping, via
the use of an option to select the non-default mode (for example, 'local
-d' to force dynamic, 'local -s' to force static, and 'local' to go with
default scoping)?
  If both scoping forms are supported, is it worth making the default
scoping dependent on posix compliance (for example, 'local' means
dynamic scoping for 'emulate zsh' but static scoping for 'emulate sh'),
or should it be the same default for both modes?

2. User aspect:
  Is anyone aware of a script that intentionally uses the full power of
dynamic scoping available through 'local' which would break if scoping
switched to static?

Here's a sample shell script that illustrates the difference between the
two scoping methods.

$ ksh -c 'function f1 { typeset a=local; f2; echo $a; };
  function f2 { echo $a; a=changed; };
  a=global; f1; echo $a'
global
local
changed

$ zsh -c 'function f1 { typeset a=local; f2; echo $a; };
  function f2 { echo $a; a=changed; };
  a=global; f1; echo $a'
local
changed
global

In static scoping, function f2 does not shadow a declaration of a, so
references to $a within f2 refer to the global variable.  The local
variable a of f1 can only be accessed within f1; the behavior of f2 is
the same no matter how it was reached.

In dynamic scoping, function f2 looks up its call stack for the closest
enclosing scope of a variable named a, and finds the local one declared
in f1.  Therefore, the behavior of f2 depends on how f2 is called.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: static vs. dynamic scoping

[Peter Stephenson]

I might as well copy what I sent to the Austin group list this
morning for the majority not following that...

The "other" matters referred to are, to summarise very briefly,
basically whether

  foo="a b"
  export x=$foo

causes x to be set to "a b".  Currently POSIX implies it doesn't.


From: Peter Stephenson <pws@csr.com>
To: austin-group-l@opengroup.org
Subject: Re: Word splitting in 'export' arguments and adding local
Date: Wed, 10 Nov 2010 10:13:20 +0000

On Tue, 09 Nov 2010 22:34:22 -0500
Chet Ramey <chet.ramey@case.edu> wrote:
> Bash uses dynamic scoping and, as I have existing users and scripts
> that take advantage of it, I have no intention of changing that.  I
> know zsh uses dynamic scoping and there are scripts that make
> extensive use of it.  

That's correct about zsh: the completion functions supplied with the
shell rely (very) heavily on dynamic scoping and we certainly wouldn't
be changing that.  I can't see any likelihood of writing a completely
different additional implementation of local variables with static
scoping for POSIX mode.

It might be possible as an option to hide and uncover variables when
entering nested functions to simulate static scoping without a complete
rewrite, but I really haven't thought it through and it's not likely to
be a high priority.  It might also be worth reminding people that POSIX
support was rather an afterthought for zsh but we try to keep that mode
working where possible.

I don't think any of the other matters being discusses here are a big
issue for zsh in POSIX mode, however.  If you happen to be investigating
this in zsh, the following options are relevant:

SH_WORD_SPLIT
   turn on splitting of substitutions as done by default in other shells

KSH_TYPESET
   turn off argument splitting on assignments for typeset, local, etc.
   if SH_WORD_SPLIT is on.  Note this is not turned on automatically
   in POSIX mode (for reasons anyone following this thread will realise).

MAGIC_EQUAL_SUBST
   ~-expansion (and zsh-specific =-expansion, if applicable) are active
   after all "=", and also after following :, regardless of the command.
   The example in the manual is
     echo foo=~/bar:~/rod
   expands both ~'s.  The effect of this option has expanded somewhat
   during the life of zsh.  (I'm not suggesting this would ever be
   POSIX-compatible.)

pws


Member of the CSR plc group of companies. CSR plc registered in England and Wales, registered number 4187346, registered office Churchill House, Cambridge Business Park, Cowley Road, Cambridge, CB4 0WZ, United Kingdom

Re: static vs. dynamic scoping

[Bart Schaefer]

On Nov 9,  3:08pm, Eric Blake wrote:
}
} 1. Implementation aspect:
}   How hard would it be to add static scoping to zsh?

Just to elaborate on this, as I've been fooling with parameter scopes a
bit lately (http://www.zsh.org/mla/workers//2010/msg00719.html) ...

Zsh currently maintains scopes as [what amounts to] a parallel stack
alongside the function call stack, with scope depth indexed by the
call stack depth.  A variable expansions finds its object at the
uppermost frame of the scope stack in which that variable's name has
been "mentioned" with either typeset/local/declare or undef; if it is
not found at all, assignment creates it in the global frame.  Note,
though, that the global frame is simply the bottom of the stack, not
handled separately except for some special cases for stack depth 0.

Therefore it's at least moderately difficult to add static scoping.

}   Is it something that can be added in addition to dynamic scoping, via
} the use of an option to select the non-default mode (for example, 'local
} -d' to force dynamic, 'local -s' to force static, and 'local' to go with
} default scoping)?

It might be possible to flag a variable in a given frame as static and
thereby cause it to be skipped when the current call depth does not
match the stack depth where the variable was "mentioned", but due to
the details of the implementation [*] I can't think of any reasonable
way to mark entire frames static or dynamic as implied by having two
different function styles.  Maybe that doesn't matter as long as all
undeclared variables in either kind of scope are global, because then
the state of the frame matters only when "local" is used in that frame.

[*] Cf. "what amounts to" -- in reality, every variable has its own
independent stack which comes into being only when the variable is
"mentioned", so that it's not necessary to walk the stack to find
the uppermost frame for a given variable.

}   If both scoping forms are supported, is it worth making the default
} scoping dependent on posix compliance (for example, 'local' means
} dynamic scoping for 'emulate zsh' but static scoping for 'emulate sh'),
} or should it be the same default for both modes?

As it'd probably be controlled by an option, this probably doesn't
change the difficulty either way.

Re: static vs. dynamic scoping

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 5025 bytes --]

On 11/10/2010 09:50 AM, Bart Schaefer wrote:
> On Nov 9,  3:08pm, Eric Blake wrote:
> }
> } 1. Implementation aspect:
> }   How hard would it be to add static scoping to zsh?
> 
> Just to elaborate on this, as I've been fooling with parameter scopes a
> bit lately (http://www.zsh.org/mla/workers//2010/msg00719.html) ...
> 
> Zsh currently maintains scopes as [what amounts to] a parallel stack
> alongside the function call stack, with scope depth indexed by the
> call stack depth.  A variable expansions finds its object at the
> uppermost frame of the scope stack in which that variable's name has
> been "mentioned" with either typeset/local/declare or undef; if it is
> not found at all, assignment creates it in the global frame.  Note,
> though, that the global frame is simply the bottom of the stack, not
> handled separately except for some special cases for stack depth 0.
> 
> Therefore it's at least moderately difficult to add static scoping.
> 
> }   Is it something that can be added in addition to dynamic scoping, via
> } the use of an option to select the non-default mode (for example, 'local
> } -d' to force dynamic, 'local -s' to force static, and 'local' to go with
> } default scoping)?
> 
> It might be possible to flag a variable in a given frame as static and
> thereby cause it to be skipped when the current call depth does not
> match the stack depth where the variable was "mentioned", but due to
> the details of the implementation [*] I can't think of any reasonable
> way to mark entire frames static or dynamic as implied by having two
> different function styles.  Maybe that doesn't matter as long as all
> undeclared variables in either kind of scope are global, because then
> the state of the frame matters only when "local" is used in that frame.
>
> [*] Cf. "what amounts to" -- in reality, every variable has its own
> independent stack which comes into being only when the variable is
> "mentioned", so that it's not necessary to walk the stack to find
> the uppermost frame for a given variable.

You are correct that ksh93 currently has two flavors of functions:

POSIX functions 'f () {...}', currently no scoping support
ksh functions 'function f {...}', supports static scoping

But I see no need to standardize on this.  My current thoughts for POSIX
standardization are:

keep a single function syntax (ksh can continue to use 'function' as a
reserved word, and with alternate function syntax, but that would be an
extension and not mandatory; bash can continue to use function as an
alternate spelling for POSIX functions without the split in
functionality used by ksh)

provide the 'typeset' special built-in, which is required to obey XBD
12.2 (that is, it must honor --), so as to allow implementations to
support extensions such as 'typeset -a' for declaring array variables

require that 'typedef' used without options and within a function body
provide a statically scoped local variable.  Possible require a few
other commonly implemented 'typedef -X' options, such as 'typedef -p'
for printing the current set of locally scoped variables (much like
'export -p).

only require static scoping (for ksh93), but additionally allow dynamic
scoping as an extension (this would be the course used by bash and zsh,
to preserve their current user base of tab-completion that expects
dynamic scoping)

continue to allow the ksh extension of reference variables (since ksh
uses this in in lieu of dynamic scoping for exposing local variables to
secondary functions), although I don't see this implemented in bash or zsh

For implementations that want to provide static and dynamic scoping
simultaneously (bash, zsh), I think the following implementation would
do just fine:
 + Create two variants of 'local': 'local -d' for dynamic scoping, and
'local -s' for static scoping (which version was used determines whether
a variable is marked as static on the variable stack).  For backwards
compatibility, 'local' becomes 'local -d'.
 + Make 'typedef' become a synonym for 'local -s' (or keep 'typedef' as
a synonym for 'local', and add a a shell option that determines whether
'local' defaults to 'local -d' or 'local -s', where that option is
auto-set according to posix mode)
 + Within a function, when referencing a variable:
   + if the variable was declared within that function, use it (doesn't
matter whether it was declared with 'local -s' or 'local -d')
   + if the variable was not declared within that function, then travel
up the variable declaration stack to the first instance of the variable
that was not marked static (if the script only uses static scoping, then
this requires traversing the entire stack of variables, but will settle
on the global scope; if the script only uses dynamic scoping, then this
will find the very first variable on the stack)

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: static vs. dynamic scoping

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 736 bytes --]

On 11/10/2010 10:28 AM, Eric Blake wrote:
> provide the 'typeset' special built-in, which is required to obey XBD
> 12.2 (that is, it must honor --), so as to allow implementations to
> support extensions such as 'typeset -a' for declaring array variables
> 
> require that 'typedef' used without options and within a function body
> provide a statically scoped local variable.  Possible require a few
> other commonly implemented 'typedef -X' options, such as 'typedef -p'
> for printing the current set of locally scoped variables (much like
> 'export -p).

'typeset', not 'typedef', throughout that paragraph.

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: static vs. dynamic scoping

[Thorsten Glaser]

Eric Blake dixit:

>require that 'typedef' used without options and within a function body
>provide a statically scoped local variable.  Possible require a few

I'd very much like to avoid changing the scoping model of mksh,
and request that this not be standardised on. This is both from
an implementors and users point of view.

Thanks, (and thanks to pgas for pointing out this thread)
//mirabilos
-- 
22:20⎜<asarch> The crazy that persists in his craziness becomes a master
22:21⎜<asarch> And the distance between the craziness and geniality is
only measured by the success 18:35⎜<asarch> "Psychotics are consistently
inconsistent. The essence of sanity is to be inconsistently inconsistent

Re: static vs. dynamic scoping

[Bart Schaefer]

One additional thought on this ...

Zsh does have the "-g" option of "typeset" to allow one declare or
to change the properties of a variable that is not in the scope of
the current function.  However, "-g" doesn't really mean "global";
it means only "the nearest dynamic scope where this variable is
already declared" which is the global scope if the parameter has
never been declared, but might even be the current scope if there
has previously been a "local" delcaration of the variable.

Does ksh93 have any mechanism for explicitly declaring a variable to
be global from inside a function scope?

-- 

Re: static vs. dynamic scoping

[Eric Blake]

[-- Attachment #1: Type: text/plain, Size: 1291 bytes --]

On 11/10/2010 10:22 AM, Bart Schaefer wrote:
> One additional thought on this ...
> 
> Zsh does have the "-g" option of "typeset" to allow one declare or
> to change the properties of a variable that is not in the scope of
> the current function.  However, "-g" doesn't really mean "global";
> it means only "the nearest dynamic scope where this variable is
> already declared" which is the global scope if the parameter has
> never been declared, but might even be the current scope if there
> has previously been a "local" delcaration of the variable.
> 
> Does ksh93 have any mechanism for explicitly declaring a variable to
> be global from inside a function scope?

I'm not that familiar with ksh93; you'd have to ask David Korn for
precise details.  But my understanding is that with static scoping,
every variable reference which was not explicitly declared local is
inherently global, because there are no other scopes to worry about.

That is (assuming ksh were to support local scoping with posix syntax
functions):

f () {
  a=1  # modify the global variable a
  typeset a;
  a=2  # modify the function-local a
}
a=0; f; echo $a # outputs 1

-- 
Eric Blake   eblake@redhat.com    +1-801-349-2682
Libvirt virtualization library http://libvirt.org


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 619 bytes --]

Re: static vs. dynamic scoping

[Bart Schaefer]

On Nov 10, 10:30am, Eric Blake wrote:
>
> I'm not that familiar with ksh93; you'd have to ask David Korn for
> precise details.  But my understanding is that with static scoping,
> every variable reference which was not explicitly declared local is
> inherently global, because there are no other scopes to worry about.

Right, and that works fine for simple assignments and for expansion.
I'm curious about what happens when you want to declare something
not-static but also not a simple scalar, e.g.

f() {
    # Is there any way to make these global?
    typeset -i4 an_int
    typeset -u all_upper
}