zsh-workers
 help / color / mirror / code / Atom feed
* More rabbit-holes with unset variables
@ 2020-11-25  7:02 Bart Schaefer
  2020-11-25 13:19 ` Stephane Chazelas
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-25  7:02 UTC (permalink / raw)
  To: Zsh hackers list

I was experimenting with ksh to look deeper into adding behavior to
ksh_typeset in zsh, and discovered that this example:

function ff {
  echo start
  typeset -i i
  typeset -p i
  echo ${i-nil}
  unset i
  typeset -p i
  echo end
}

Outputs in ksh93:

start
typeset -i i
nil
end

And bash:

start
declare -i i
nil
declare -- i
end

Although both of them have a way to represent "declared but not set",
neither of them treats "explicitly unset" as equivalent to that, and
neither of them preserves any type information across unset.  Only
bash regurgitates the "declare" after unset, but that's an effect of
the implementation of -p rather than of the state of the variable,
because function scope still applies to $a in both cases.

(I had to remind myself that "ff() { ... }" in ksh is NOT a "function"
in the sense of scope locality.)

Therefore, this isn't as simple as having zsh create an unset variable
when typeset is given no assignment, because subsequent assignment has
to preserve the type of the variable, which normally does not apply
after unset.


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-25  7:02 More rabbit-holes with unset variables Bart Schaefer
@ 2020-11-25 13:19 ` Stephane Chazelas
  2020-11-25 22:17   ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Stephane Chazelas @ 2020-11-25 13:19 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

2020-11-24 23:02:05 -0800, Bart Schaefer:
> I was experimenting with ksh to look deeper into adding behavior to
> ksh_typeset in zsh, and discovered that this example:
[...]

Sorry I didn't read the full discussion as I just subscribed to
the list, but I did have a look at the differences between local
scope implementations at some point and there are many many
variations between shells.

I did find zsh was far from being the worst one there.

They have been discussed at length on the austin group (POSIX)
mailing list as they've been trying on and off to specify a
local scope for sh variables.

I did write a quick summary of some of those findings at:
https://unix.stackexchange.com/questions/493729/list-of-shells-that-support-local-keyword-for-defining-local-variables/493743#493743
which is very relevant to this discussion.

See https://www.austingroupbugs.net/bug_view_page.php?bug_id=767
for the POSIX attempt. You'll find a number of lengthy related
discussions on their mailing archive.

It's after one of those discussions that bash added the
localvar_inherit and localvar_unset options, and NetBSD sh added
-I and -N options to "local" I beleive (the mantainers of bash,
NetBSD sh, bosh are regulars on the austin group mailing list,
FreeBSD sh maintainer is seen occasionally).

-- 
Stephane


^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-25 13:19 ` Stephane Chazelas
@ 2020-11-25 22:17   ` Felipe Contreras
  2020-11-26  6:10     ` Stephane Chazelas
  2020-11-26 20:41     ` Bart Schaefer
  0 siblings, 2 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-11-25 22:17 UTC (permalink / raw)
  To: Bart Schaefer, Zsh hackers list, Stephane Chazelas

On Wed, Nov 25, 2020 at 7:19 AM Stephane Chazelas <stephane@chazelas.org> wrote:
>
> 2020-11-24 23:02:05 -0800, Bart Schaefer:
> > I was experimenting with ksh to look deeper into adding behavior to
> > ksh_typeset in zsh, and discovered that this example:
> [...]
>
> Sorry I didn't read the full discussion as I just subscribed to
> the list, but I did have a look at the differences between local
> scope implementations at some point and there are many many
> variations between shells.
>
> I did find zsh was far from being the worst one there.

Yes, but the discussion was about inheritance and the behavior of
unset, not about what we are discussing here.

They did mention the zsh behavior, but that's that's it. I don't think
anyone contested what in my opinion is the natural behavior, which is
described initially in the description:

  The variable whose name is specified shall be created as a local
variable with name "name". It
  shall inherit its initial value, as well as the exported and
readonly flags, from the variable
  with the same name in the surrounding dynamic scope if there is one;
otherwise, the variable is
  initially unset. Then, if "=word" is provided, the value of that
local variable shall then be set
  to word.

So, the variable is initially unset, *unless* "=word" is provided.

Do you think anyone objected to that behavior?

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-25 22:17   ` Felipe Contreras
@ 2020-11-26  6:10     ` Stephane Chazelas
  2020-11-26  7:20       ` Felipe Contreras
  2020-11-26 22:07       ` Bart Schaefer
  2020-11-26 20:41     ` Bart Schaefer
  1 sibling, 2 replies; 40+ messages in thread
From: Stephane Chazelas @ 2020-11-26  6:10 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Bart Schaefer, Zsh hackers list

2020-11-25 16:17:46 -0600, Felipe Contreras:
[...]
>   The variable whose name is specified shall be created as a local
> variable with name "name". It
>   shall inherit its initial value, as well as the exported and
> readonly flags, from the variable
>   with the same name in the surrounding dynamic scope if there is one;
> otherwise, the variable is
>   initially unset. Then, if "=word" is provided, the value of that
> local variable shall then be set
>   to word.
[...]

You'd describing the ash/bosh behaviour. Most other shells
behave differently. As mentioned at that SE Q&A or in that POSIX
issue, there is a strong case for that variable to be a brand
new variable with an initial unset or empty/0/empty-list value
depending on the type.

In reality, depending on the shell you get:

- variable left as is from the parent scope (inherit, value,
  type, attributes), but restored upon return of function.
  Useful in things like "local PATH" where you want to keep the
  current value of PATH, be able to do modifications and restore
  the original value upon return of the function.

  That approach is not acceptable for a shell with other types
  of variables (array, hash, compound) and with variable
  attributes (integer, float, padding, uppercase...). Because
  that would invalidate all the uses like:

  f() { local i; for i do ...; done; }

  Which would be broken if i remained a hash/integer/padded...

  In ash, that function doesn't work in contexts where i has
  been made read-only in a parent scope, and leaks the value of
  i to executed commands if it was exported. NetBSD sh added
  "local -N i" for that. Though you could also do
  "local var; unset var" (not portable as it doesn't work in
  mksh/yash)

- variable created unset but inheriting some of the attributes
  (bash) that's the worst of both world as you don't know what
  you're going to get. (localvar_unset doesn't address that).

- variable created anew initially unset

- variable created anew with initial value (zsh).

As to unset vs initial value, both have merits. While I'd tend
to prefer "initially unset", it makes sense that "local -i i;
echo "$i"" outputs an integer. Or that "local -Z2 v; echo $#v"
outputs 2.

In any case, I don't think zsh can change its default behaviour
as it would break backward compatibility.

It could add -N/-I à la NetBSD sh if people found it was useful
enough.

It could try and emulate ksh in ksh emulation, but which ksh?
ksh88, ksh93 and mksh behave radically differently in that
regard. Also few people use ksh these days, so I'm not sure it's
worth the effort. While the ksh emulation mode can help with
bash compatibility, bash's behaviour in this instent is also
very different.

-- 
Stephane



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26  6:10     ` Stephane Chazelas
@ 2020-11-26  7:20       ` Felipe Contreras
  2020-11-26 11:21         ` Oliver Kiddle
  2020-11-26 22:07       ` Bart Schaefer
  1 sibling, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26  7:20 UTC (permalink / raw)
  To: Felipe Contreras, Bart Schaefer, Zsh hackers list

On Thu, Nov 26, 2020 at 12:10 AM Stephane Chazelas
<stephane@chazelas.org> wrote:

> As to unset vs initial value, both have merits. While I'd tend
> to prefer "initially unset", it makes sense that "local -i i;
> echo "$i"" outputs an integer. Or that "local -Z2 v; echo $#v"
> outputs 2.

Yes, when you are specifying a type, it's obvious what default value
you would want, and at least to me it makes sense that "typeset -a
arr" initializes $arr with an empty array. But that's not what was
discussed here.

The discussion is about what "typeset var" (with no type) should do.

> In any case, I don't think zsh can change its default behaviour
> as it would break backward compatibility.

That's what major versions are for (maybe zsh 6.0?), and backwards
compatibility has been broken before.

Anyway, in the current state of the discussion, the idea is to add the
unset behavior to "setopt ksh_typeset", so no backwards compatibility
would be broken.

> It could try and emulate ksh in ksh emulation, but which ksh?
> ksh88, ksh93 and mksh behave radically differently in that
> regard. Also few people use ksh these days, so I'm not sure it's
> worth the effort. While the ksh emulation mode can help with
> bash compatibility, bash's behaviour in this instent is also
> very different.

When no type is specified, both ksh and bash do the same thing: no
initial value is assigned.

I don't want to say "unset", since as you explained; that can entail a
lot of different behaviors depending on the shell. But rather: nil
value.

Can we agree that it makes sense that "typeset var" (with no type)
assigns $var the nil value (in other words: no value) (or at the very
least no value in the case that no global $var has been previously
set)?

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26  7:20       ` Felipe Contreras
@ 2020-11-26 11:21         ` Oliver Kiddle
  2020-11-26 11:29           ` Roman Perepelitsa
                             ` (2 more replies)
  0 siblings, 3 replies; 40+ messages in thread
From: Oliver Kiddle @ 2020-11-26 11:21 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

The original Bourne shell did I believe have both export and the
${...-...} form. Using /bin/sh from Solaris (available for Linux as
heirloom sh):
  export FOO
  echo ${FOO-replacement}
Will output the replacement.

It appears that export VAR will not export an empty VAR even in zsh
(unless you do VAR=""). So zsh's not exactly consistent.
typeset [-x] is arguably just a variant of export.

This does change how I regard zsh's behaviour. It isn't zsh taking a
different but equally valid approach on an extension but an sh
incompatibility. It once was a bug even if now too entrenched.

Zsh's behaviour is long-standing - I've even checked 2.5.03. And it is
well-defined behaviour so declaring it a feature is an option.

It does make the KSH_TYPESET option somewhat less appropriate because
this effect also applies to sh emulation.

But if we decide to "fix" it for zsh, I don't think it'll break too much.
Given that empty values are rarely especially useful, I've always used
the :- form. If the change sits in git for a reasonable time, we might
find out how much it might break.

I did reply on the initial thread but the References: header hit
a mail server limit for line length and it failed.
So I've covered my main point but just to pick out something from this
thread:

Felipe Contreras wrote:
> The discussion is about what "typeset var" (with no type) should do.

That is not "with no type", it has the default type which effectively is
a string. Other shells also treat such variables as an empty string for
the purposes of other features. Try += for example.

Oliver



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 11:21         ` Oliver Kiddle
@ 2020-11-26 11:29           ` Roman Perepelitsa
  2020-11-26 19:08             ` Felipe Contreras
  2020-11-26 19:29           ` Felipe Contreras
  2020-11-26 21:13           ` Bart Schaefer
  2 siblings, 1 reply; 40+ messages in thread
From: Roman Perepelitsa @ 2020-11-26 11:29 UTC (permalink / raw)
  To: Oliver Kiddle; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 12:22 PM Oliver Kiddle <opk@zsh.org> wrote:
>
> But if we decide to "fix" it for zsh, I don't think it'll break too much.

What kind of change do you envision? Will it change the output of this command?

  zsh -c 'f() { local -i x; echo $x; }; f'

If yes, it'll likely break a lot of code (it'll surely break my code).
If no, then the behavior of the following command should also stay
unchanged for consistency:

  zsh -c 'f() { local x; echo ${x-y}; }; f'

Roman.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 11:29           ` Roman Perepelitsa
@ 2020-11-26 19:08             ` Felipe Contreras
  0 siblings, 0 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26 19:08 UTC (permalink / raw)
  To: Roman Perepelitsa; +Cc: Oliver Kiddle, Zsh hackers list

On Thu, Nov 26, 2020 at 5:30 AM Roman Perepelitsa
<roman.perepelitsa@gmail.com> wrote:
>
> On Thu, Nov 26, 2020 at 12:22 PM Oliver Kiddle <opk@zsh.org> wrote:
> >
> > But if we decide to "fix" it for zsh, I don't think it'll break too much.
>
> What kind of change do you envision? Will it change the output of this command?
>
>   zsh -c 'f() { local -i x; echo $x; }; f'
>
> If yes, it'll likely break a lot of code (it'll surely break my code).
> If no, then the behavior of the following command should also stay
> unchanged for consistency:
>
>   zsh -c 'f() { local x; echo ${x-y}; }; f'

I disagree. The first one is an orange, the second is an apple.

In my opinion it's perfectly reasonable to return 0 in the first, and
nil in the second.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 11:21         ` Oliver Kiddle
  2020-11-26 11:29           ` Roman Perepelitsa
@ 2020-11-26 19:29           ` Felipe Contreras
  2020-11-26 21:13           ` Bart Schaefer
  2 siblings, 0 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26 19:29 UTC (permalink / raw)
  To: Oliver Kiddle; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 5:21 AM Oliver Kiddle <opk@zsh.org> wrote:

> It appears that export VAR will not export an empty VAR even in zsh
> (unless you do VAR=""). So zsh's not exactly consistent.
> typeset [-x] is arguably just a variant of export.

That is a good point. Why does zsh return nil in this case?

  typeset -x FOO
  sh -c 'echo ${FOO-nil}'

> Felipe Contreras wrote:
> > The discussion is about what "typeset var" (with no type) should do.
>
> That is not "with no type", it has the default type which effectively is
> a string. Other shells also treat such variables as an empty string for
> the purposes of other features. Try += for example.

I mean the user specified no type.

What each shell implementation decides to do with that depends, but
the user specified no type. Yes, it makes sense to use string as the
default type, but some weird shell implementation might decide not to.

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-25 22:17   ` Felipe Contreras
  2020-11-26  6:10     ` Stephane Chazelas
@ 2020-11-26 20:41     ` Bart Schaefer
  2020-11-26 20:49       ` Stephane Chazelas
  2020-11-26 21:20       ` Felipe Contreras
  1 sibling, 2 replies; 40+ messages in thread
From: Bart Schaefer @ 2020-11-26 20:41 UTC (permalink / raw)
  To: Zsh hackers list

On Wed, Nov 25, 2020 at 2:17 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> [quoting an article from the posix tracker]
>   The variable whose name is specified shall be created as a local
> variable with name "name". It
>   shall inherit its initial value, as well as the exported and
> readonly flags, from the variable
>   with the same name in the surrounding dynamic scope if there is one;
> otherwise, the variable is
>   initially unset. Then, if "=word" is provided, the value of that
> local variable shall then be set
>   to word.
>
> So, the variable is initially unset, *unless* "=word" is provided.
>
> Do you think anyone objected to that behavior?

Is there a newer version of the standard than V4 from 2018, which is
the latest I can find online?

"Local variables within a function were considered and included in
another early proposal (controlled by the special built-in local), but
were removed because they do not fit the simple model developed for
functions and because there was some opposition to adding yet another
new special built-in that was not part of historical practice.
Implementations should reserve the identifier local (as well as
typeset, as used in the KornShell) in case this local variable
mechanism is adopted in a future version of this standard."

So the language you're quoting was rejected, though not for that
specific reason.

In fact, unless the notion of typed variables (integer, array, etc.)
is excluded, that language is ambiguous, because (as I've mentioned
elsewhere) the behavior of an "unset" variable in other contexts (such
as assignment) is that it takes on the attributes from the assignment;
so a the only useful "declared but not set" variable is a simple
scalar.

I'll note that posix defines null as "" (double-quoted empty string)
although it requires a careful reading to find that.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 20:41     ` Bart Schaefer
@ 2020-11-26 20:49       ` Stephane Chazelas
  2020-11-26 21:20       ` Felipe Contreras
  1 sibling, 0 replies; 40+ messages in thread
From: Stephane Chazelas @ 2020-11-26 20:49 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

2020-11-26 12:41:31 -0800, Bart Schaefer:
[...]
> Is there a newer version of the standard than V4 from 2018, which is
> the latest I can find online?
[...]

To answer that specific question, the next major version
(issue8) of the standard is currently in draft stage. A draft1
has been released and undergone a review stage now completed.
There is a draft1.1 which can be downloaded from the opengroup
once you create an account, see
https://www.mail-archive.com/austin-group-l@opengroup.org/msg06965.html
for details.

In any case, sh "local" scope won't be included in that version
of the standard. We'll probably need to wait another decade or
so for that.

-- 
Stephane



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 11:21         ` Oliver Kiddle
  2020-11-26 11:29           ` Roman Perepelitsa
  2020-11-26 19:29           ` Felipe Contreras
@ 2020-11-26 21:13           ` Bart Schaefer
  2020-11-26 21:31             ` Felipe Contreras
  2 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-26 21:13 UTC (permalink / raw)
  To: Zsh hackers list

On Thu, Nov 26, 2020 at 3:22 AM Oliver Kiddle <opk@zsh.org> wrote:
>
> It appears that export VAR will not export an empty VAR even in zsh
> (unless you do VAR=""). So zsh's not exactly consistent.
> typeset [-x] is arguably just a variant of export.

Neither is a variant of the other; "export" is an alias for "typeset
-xg" that happens to be implemented as a reserved word.

> This does change how I regard zsh's behaviour. It isn't zsh taking a
> different but equally valid approach on an extension but an sh
> incompatibility. It once was a bug even if now too entrenched.

It doesn't export the empty string at the time the parameter is
declared, but it does consistently set it to empty string internally:

% Src/zsh -f
% export FOOBAR
% typeset -p FOOBAR
export FOOBAR=''
%

If it were NOT set, as after "unset FOOBAR", then "typeset -p" would
print nothing.

> It does make the KSH_TYPESET option somewhat less appropriate because
> this effect also applies to sh emulation.

We could do this when EMULATION(EMULATE_KSH|EMULATE_SH) instead of on an option.

If we are in fact limiting it to variables with no "strong type" type
then it really is sufficient to create an unset variable, except that
we somehow have to signal to "typeset -p" that it should print the
declaration even though the variable appears to be unset.

The latter raises the question of whether we want the Bash behavior of
continuing to print the declaration even after an explicit "unset",
which differs from ksh (cf. my first message in this thread).



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 20:41     ` Bart Schaefer
  2020-11-26 20:49       ` Stephane Chazelas
@ 2020-11-26 21:20       ` Felipe Contreras
  2020-11-26 22:41         ` Bart Schaefer
  1 sibling, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26 21:20 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 2:42 PM Bart Schaefer <schaefer@brasslantern.com> wrote:

> So the language you're quoting was rejected, though not for that
> specific reason.

Yes, but that's not what I asked.

I asked who objected to the specific behavior I quoted.

> In fact, unless the notion of typed variables (integer, array, etc.)
> is excluded, that language is ambiguous, because (as I've mentioned
> elsewhere) the behavior of an "unset" variable in other contexts (such
> as assignment) is that it takes on the attributes from the assignment;
> so a the only useful "declared but not set" variable is a simple
> scalar.

And by scalar I suppose you mean string.

Yes, I agree in the context of typed variables it's much more
debatable what the initial value should be.

Which is why since the start I only mentioned variables for which the
user didn't specify a type.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 21:13           ` Bart Schaefer
@ 2020-11-26 21:31             ` Felipe Contreras
  2020-11-26 23:29               ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26 21:31 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 3:13 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Thu, Nov 26, 2020 at 3:22 AM Oliver Kiddle <opk@zsh.org> wrote:

> > This does change how I regard zsh's behaviour. It isn't zsh taking a
> > different but equally valid approach on an extension but an sh
> > incompatibility. It once was a bug even if now too entrenched.
>
> It doesn't export the empty string at the time the parameter is
> declared, but it does consistently set it to empty string internally:

And you don't find it inconsistent that the internal value is
*different* from the exported value?

Plus this:

  typeset -x FOO

Is different than this:

  typeset -x FOO=""

?

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26  6:10     ` Stephane Chazelas
  2020-11-26  7:20       ` Felipe Contreras
@ 2020-11-26 22:07       ` Bart Schaefer
  1 sibling, 0 replies; 40+ messages in thread
From: Bart Schaefer @ 2020-11-26 22:07 UTC (permalink / raw)
  To: Zsh hackers list

On Wed, Nov 25, 2020 at 10:10 PM Stephane Chazelas
<stephane@chazelas.org> wrote:
>
> In any case, I don't think zsh can change its default behaviour
> as it would break backward compatibility.
>
> It could add -N/-I ą la NetBSD sh if people found it was useful
> enough.

Zsh's "typeset -h" and "typeset +h" have some similar properties.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 21:20       ` Felipe Contreras
@ 2020-11-26 22:41         ` Bart Schaefer
  2020-11-26 23:45           ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-26 22:41 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 1:21 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> I asked who objected to the specific behavior I quoted.

I'm not going to dig through the austin-group archives to try to
distinguish which of the two parts of the quoted behavior (inherit, or
otherwise not set) was the source of anyone's objection.  I concede
that the explanatory paragraph in the rationale did not call out
either of those behaviors.

The point is that none of what we're discussing is an agreed-upon standard.

> On Thu, Nov 26, 2020 at 2:42 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> > so a the only useful "declared but not set" variable is a simple
> > scalar.
>
> And by scalar I suppose you mean string.

I mean the simplest form of what zsh refers to as "scalar" internally.
Yes, that's ordinarily representable in C by "char[]", but in the
abstract that doesn't matter.

On the other hand, the shell language doesn't really deal in anything
else.  Everything is designed around file descriptors and argv
(char[][]) and performing implicit string splitting/catenation, and
except for nameref in ksh there's no way to pass pointers around.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 21:31             ` Felipe Contreras
@ 2020-11-26 23:29               ` Bart Schaefer
  2020-11-26 23:32                 ` Bart Schaefer
  2020-11-26 23:53                 ` Felipe Contreras
  0 siblings, 2 replies; 40+ messages in thread
From: Bart Schaefer @ 2020-11-26 23:29 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 1:31 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> And you don't find it inconsistent that the internal value is
> *different* from the exported value?

That's a loaded question, because to answer it requires conceding that
there exists an exported value from which the internal value differs.
There's no way to "see" the export namespace without forking an
external process, so only the internal value matters.

That internal value exists for the sole purpose of recording the fact
that two different namespaces need to be synchronized whenever either
of them changes.  The "difference" occurs because neither namespace
has changed yet.  The fact that there is no first-class "not defined"
value that can be employed for this purpose is the same issue that got
us into this whole discussion to begin with.

We're back to the point that from inside the shell, the only place
where that difference is a distinction is when you explicitly inquire
about the "not defined" state, and I'm already looking for a solution
to that.

> Plus this:
>
>   typeset -x FOO
>
> Is different than this:
>
>   typeset -x FOO=""
>
> ?

It also differs from

typeset -i FOO

which internally sets FOO=0 but doesn't export it until something
changes.  Diving into how zsh represents the relationship between the
internal and exported namespaces is a whole other ball of wax and is
why I tried to keep it out of scope before Oliver brought it up.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 23:29               ` Bart Schaefer
@ 2020-11-26 23:32                 ` Bart Schaefer
  2020-11-26 23:53                 ` Felipe Contreras
  1 sibling, 0 replies; 40+ messages in thread
From: Bart Schaefer @ 2020-11-26 23:32 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 3:29 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> It also differs from
>
> typeset -i FOO
>
> which internally sets FOO=0 but doesn't export it

Of course I meant

typeset -xi FOO

aka

export -i FOO



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 22:41         ` Bart Schaefer
@ 2020-11-26 23:45           ` Felipe Contreras
  2020-11-27  0:09             ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26 23:45 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 4:41 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Thu, Nov 26, 2020 at 1:21 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > I asked who objected to the specific behavior I quoted.
>
> I'm not going to dig through the austin-group archives to try to
> distinguish which of the two parts of the quoted behavior (inherit, or
> otherwise not set) was the source of anyone's objection.  I concede
> that the explanatory paragraph in the rationale did not call out
> either of those behaviors.

The question was not addressed to you, but to the person that brought
up the bug report.

A quick look at the bug report doesn't show anyone objecting to that.

> > On Thu, Nov 26, 2020 at 2:42 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
> >
> > > so a the only useful "declared but not set" variable is a simple
> > > scalar.
> >
> > And by scalar I suppose you mean string.
>
> I mean the simplest form of what zsh refers to as "scalar" internally.
> Yes, that's ordinarily representable in C by "char[]", but in the
> abstract that doesn't matter.

Strings in C are typically declared as "char *", not "char []".

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 23:29               ` Bart Schaefer
  2020-11-26 23:32                 ` Bart Schaefer
@ 2020-11-26 23:53                 ` Felipe Contreras
  2020-11-27  0:23                   ` Bart Schaefer
  1 sibling, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-26 23:53 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 5:29 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Thu, Nov 26, 2020 at 1:31 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > And you don't find it inconsistent that the internal value is
> > *different* from the exported value?
>
> That's a loaded question, because to answer it requires conceding that
> there exists an exported value from which the internal value differs.
> There's no way to "see" the export namespace without forking an
> external process, so only the internal value matters.

No. The exported value exists whether you decide to look at it or not.

> > Plus this:
> >
> >   typeset -x FOO
> >
> > Is different than this:
> >
> >   typeset -x FOO=""
> >
> > ?

If this is not inconsistent, then nothing is.

So I guess nothing zsh ever does can be inconsistent, by definition.

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 23:45           ` Felipe Contreras
@ 2020-11-27  0:09             ` Bart Schaefer
  2020-11-27  0:30               ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27  0:09 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 3:45 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> On Thu, Nov 26, 2020 at 4:41 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
> >
> > I mean the simplest form of what zsh refers to as "scalar" internally.
> > Yes, that's ordinarily representable in C by "char[]", but in the
> > abstract that doesn't matter.
>
> Strings in C are typically declared as "char *", not "char []".

How strings in C are typically declared and how scalars are
represented in zsh are not the same thing.

I deliberately chose to write char[] to demonstrate that zsh scalars
are never null pointers.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-26 23:53                 ` Felipe Contreras
@ 2020-11-27  0:23                   ` Bart Schaefer
  2020-11-27  1:51                     ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27  0:23 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 3:53 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> > There's no way to "see" the export namespace without forking an
> > external process, so only the internal value matters.
>
> No. The exported value exists whether you decide to look at it or not.

The point is that the exported value DOES NOT exist in this example;
if you were to look at the C global "environ" array following "export
FOO", it would not have (a pointer to a string containing) "FOO" in
it.

> > >   typeset -x FOO
> > >
> > > Is different than this:
> > >
> > >   typeset -x FOO=""
>
> If this is not inconsistent, then nothing is.

Now I'm confused.  All along you've been arguing that { typeset FOO }
SHOULD differ from { typeset FOO="" }.  Why does adding -x invert your
argument?



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27  0:09             ` Bart Schaefer
@ 2020-11-27  0:30               ` Felipe Contreras
  2020-11-27  0:51                 ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-27  0:30 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 6:09 PM Bart Schaefer <schaefer@brasslantern.com> wrote:

> How strings in C are typically declared and how scalars are
> represented in zsh are not the same thing.
>
> I deliberately chose to write char[] to demonstrate that zsh scalars
> are never null pointers.

But we were not talking about zsh, we were talking about shell in
general. Specifically the POSIX standard.

You are mixing and matching. First speaking about POSIX, then about
scalars, then about "char []", then you say "the shell language
doesn't really deal in anything else."

In the context of shell in general they are strings (scalars and "char
[]" are zsh-specific).

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27  0:30               ` Felipe Contreras
@ 2020-11-27  0:51                 ` Bart Schaefer
  2020-11-27  1:30                   ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27  0:51 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 4:31 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> You are mixing and matching. First speaking about POSIX, then about
> scalars, then about "char []", then you say "the shell language
> doesn't really deal in anything else."

This is now beyond silly, but:

What was or not POSIX is a indirectly-related subtopic.

I mentioned scalars.

You "supposed" that meant strings.

I clarified strings were not what I meant.

When after that I said "anything else", the word "else" was referring
back ("on the other hand") to your reference to strings, not to my
reference to scalars.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27  0:51                 ` Bart Schaefer
@ 2020-11-27  1:30                   ` Felipe Contreras
  2020-11-27 20:54                     ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-27  1:30 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 6:52 PM Bart Schaefer <schaefer@brasslantern.com> wrote:

> I mentioned scalars.
>
> You "supposed" that meant strings.
>
> I clarified strings were not what I meant.

You said this:

> > I mean the simplest form of what zsh refers to as "scalar" internally.

What zsh refers to as "scalar" internally is a string:

  char *str; /* value if declared string  (PM_SCALAR)  */

From Src/zsh.h (struct param).

So if you didn't mean string, what did you mean?

And what did you mean by 'so a the only useful "declared but not set"
variable is a simple scalar'?

What simple scalar other than a string is useful "declared but not set"?

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27  0:23                   ` Bart Schaefer
@ 2020-11-27  1:51                     ` Felipe Contreras
  2020-11-27 20:01                       ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-27  1:51 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 6:23 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Thu, Nov 26, 2020 at 3:53 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > > There's no way to "see" the export namespace without forking an
> > > external process, so only the internal value matters.
> >
> > No. The exported value exists whether you decide to look at it or not.
>
> The point is that the exported value DOES NOT exist in this example;
> if you were to look at the C global "environ" array following "export
> FOO", it would not have (a pointer to a string containing) "FOO" in
> it.

How do you know?

The example brought by Oliver was this:

  export FOO
  echo ${FOO-replacement}

How do you know FOO doesn't have a value beforehand?

That's precisely the reason the "-replacement" part is there: FOO may
or may not have a value.

You used precisely this argument when I brought up this example:

  func () {
    [[ -n "$1" ]] && var=$1
    dosomething ${var-other}
  }

You said: "It's not possible to write deterministic code."

It is exactly the same thing here.

If we have this example:

  [[ -n "${1+set}" ]] && FOO=$1
  func () {
    export FOO
    sh -c 'echo "external: \"${FOO-nil}\""'
    echo "internal: \"${FOO-nil}\""
  }
  func

  % sh example.sh bar
  external: "bar"
  internal: "bar"

All shells I have return the same.

But when I do the same without argument:

  % sh example.sh
  external: "nil"
  internal: "nil"

In all shells, except:

  % zsh example.sh
  external: "nil"
  internal: ""

The inconsistency between the internal and external value *only*
happens in zsh, and it most definitely exists.

> > > >   typeset -x FOO
> > > >
> > > > Is different than this:
> > > >
> > > >   typeset -x FOO=""
> >
> > If this is not inconsistent, then nothing is.
>
> Now I'm confused.  All along you've been arguing that { typeset FOO }
> SHOULD differ from { typeset FOO="" }.  Why does adding -x invert your
> argument?

It doesn't.

To be consistent, either these two are the same:

  typeset -x FOO
  typeset -x FOO=""

Or these two are different:

  typeset FOO
  typeset FOO=""

As long as in zsh none of these are changed, zsh is objectively
inconsistent. It is *two* inconsistencies.

What I think is obvious should be changed is the latter, so that:

  typeset -x FOO
  typeset FOO

In both cases FOO is "unset", both locally and externally.

And.

  typeset -x FOO=""
  typeset FOO=""

In both cases FOO would have an empty string both locally and externally.

I don't see how this *second* inconsistency isn't obvious.

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27  1:51                     ` Felipe Contreras
@ 2020-11-27 20:01                       ` Bart Schaefer
  2020-11-27 21:49                         ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27 20:01 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

My use of "rabbit holes" has been even more prophetic than I expected.
(Ironically, spellcheck wants to correct that to "pathetic".)

On Thu, Nov 26, 2020 at 5:51 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> On Thu, Nov 26, 2020 at 6:23 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
> >
> > The point is that the exported value DOES NOT exist in this example;
> > if you were to look at the C global "environ" array following "export
> > FOO", it would not have (a pointer to a string containing) "FOO" in
> > it.
>
> How do you know?

By paying attention to context?

We're not talking about Oliver's example at this point, we're talking
specifically about the case where "export FOOBAR" creates a global
variable that did not exist before.  If it did previously exist, then
the internal and environment namespaces would be the same.  The only
way for them to differ is when FOOBAR did not already exist.

> You used precisely this argument when I brought up this example:

Not the same, because the behavior of export vs. local (typeset in
function context) is not the same; the latter always creates a new
variable, and that variable doesn't inherit from scope.

> The inconsistency between the internal and external value *only*
> happens in zsh, and it most definitely exists.

I have never denied that.  What I said was that if you never leave
zsh, you can't tell.  That you can tell by forking off /bin/sh is
because of the way the internal and external namespaces are managed,
not because of the way the internal namespace works, and I'm
unsuccessfully attempting to keep this thread focused on the latter.

> To be consistent, either these two are the same:
>
>   typeset -x FOO
>   typeset -x FOO=""
>
> Or these two are different:
>
>   typeset FOO
>   typeset FOO=""

In the internal namespace, and starting from a name FOO that's never
been used/does not appear in the process environment, in zsh all of
these do the same thing:

declare FOO
local FOO
export FOO

To use Daniel's "${verb}s a variable" from the other thread, ${verb}
is 'creates an internal parameter to represent'.  The entirety of
these two discussion threads is supposed to be about when it is
appropriate for that to have a default value, and what it means for it
not to; currently it always has one.

Specifically for "export" there are two additional requirements:
1) If the variable already exists with a value, then variable=value is
added to the environment.
2) If a value is later assigned to the variable, then variable=value
is added to the environment.

Neither of those requirements is met under the stated conditions, so
nothing is added to the environment.  This is entirely separate from
whether there is a default.

Again starting from a previously nonexistent FOO, all of these are
also the same:

declare FOO=anything
local FOO=anything
export FOO=anything

These explicitly do two things:  First ${verb}, and then assign.  So
for export, the second additional requirement is met, and
variable=value is added to the environment.

Is there an inconsistency from the viewpoint of an omniscient
observer?  Yes.  As a practical matter, can either a script written
entirely in zsh, or an external program invoked from zsh,
independently discern this inconsistency?  No.

I'm done with responses about export behavior on this thread except
when directly related to the treatment of default values.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27  1:30                   ` Felipe Contreras
@ 2020-11-27 20:54                     ` Bart Schaefer
  2020-11-27 22:10                       ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27 20:54 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Thu, Nov 26, 2020 at 5:30 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> What zsh refers to as "scalar" internally is a string:
>
>   char *str; /* value if declared string  (PM_SCALAR)  */
>
> From Src/zsh.h (struct param).

This is exactly the discussion I was trying to avoid when I said "in
the abstract that doesn't matter".

You can't just pull one field out of a union inside a struct and
ignore the struct itself and the API for field access that goes with
it.

> So if you didn't mean string, what did you mean?

I meant a struct param, containing the least specific thing so
represented, as interpreted through all the layers of code that
implement a dereference of its value when you write $var or any of its
variations.  Again this doesn't actually matter, which is why I didn't
spell it out.

> And what did you mean by 'so a the only useful "declared but not set"
> variable is a simple scalar'?

As the very first message in this thread demonstrated, in both bash
and ksh (call this "example one", and to be pedantic assume that X is
not inheriting its name or value from somewhere):

typeset -i X
echo ${X-nil}
X="garbage"
echo ${X-nil}

will output

nil
0

However (call this "example two"):

typeset -i X
unset X
X="garbage"
echo ${X-nil}

outputs

garbage

The language you quoted from the posix proposal says "otherwise, the
variable is initially unset".  Given that proposed language, example
one is incorrect, because an "unset" variable should not retain its
(in this example) integer properties when assigned a string.

> What simple scalar other than a string is useful "declared but not set"?

Under this interpretation, there isn't any.  That's what I said.  In
fact the last paragraph of the very first message in this thread:

"Therefore, this isn't as simple as having zsh create an unset
variable when typeset is given no assignment, because subsequent
assignment has to preserve the type of the variable, which normally
does not apply after unset."



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27 20:01                       ` Bart Schaefer
@ 2020-11-27 21:49                         ` Felipe Contreras
  2020-11-27 22:06                           ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-27 21:49 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 2:01 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> My use of "rabbit holes" has been even more prophetic than I expected.
> (Ironically, spellcheck wants to correct that to "pathetic".)

There's no rabbit hole here. It's obvious that zsh's behavior is inconsistent.

If trying to defend that it's not seems like an impossible endless
endeavor, well, maybe there's a reason for that.

> On Thu, Nov 26, 2020 at 5:51 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > On Thu, Nov 26, 2020 at 6:23 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
> > >
> > > The point is that the exported value DOES NOT exist in this example;
> > > if you were to look at the C global "environ" array following "export
> > > FOO", it would not have (a pointer to a string containing) "FOO" in
> > > it.
> >
> > How do you know?
>
> By paying attention to context?
>
> We're not talking about Oliver's example at this point, we're talking
> specifically about the case where "export FOOBAR" creates a global
> variable that did not exist before.  If it did previously exist, then
> the internal and environment namespaces would be the same.  The only
> way for them to differ is when FOOBAR did not already exist.

You should not be so assuming as to what "we" are talking about, maybe
you were not talking about that, I was, and it should be clear from
where we came from:

Oliver:
> > > > > > It appears that export VAR will not export an empty VAR even in zsh
(unless you do VAR=""). So zsh's not exactly consistent.  typeset [-x] is
arguably just a variant of export.

> > > > > > This does change how I regard zsh's behaviour. It isn't zsh taking a
different but equally valid approach on an extension but an sh incompatibility.
It once was a bug even if now too entrenched.

Bart:
> > > > > It doesn't export the empty string at the time the parameter is
declared, but it does consistently set it to empty string internally:

Felipe:
> > > > And you don't find it inconsistent that the internal value is
*different* from the exported value?

Bart:
> > > That's a loaded question, because to answer it requires conceding that
there exists an exported value from which the internal value differs.  There's
no way to "see" the export namespace without forking an external process, so
only the internal value matters.

Felpe:
> > No. The exported value exists whether you decide to look at it or not.

Bart:
> The point is that the exported value DOES NOT exist in this example

Cleary I am talking about Oliver's example: echo ${FOO-replacement}.

> > You used precisely this argument when I brought up this example:
>
> Not the same, because the behavior of export vs. local (typeset in
> function context) is not the same; the latter always creates a new
> variable, and that variable doesn't inherit from scope.

Analogies don't have to be the same.

> > The inconsistency between the internal and external value *only*
> > happens in zsh, and it most definitely exists.
>
> I have never denied that.  What I said was that if you never leave
> zsh, you can't tell.  That you can tell by forking off /bin/sh is
> because of the way the internal and external namespaces are managed,
> not because of the way the internal namespace works, and I'm
> unsuccessfully attempting to keep this thread focused on the latter.

It doesn't matter if you can tell or not; the two values exist, and
they are different, and they are different *only* in zsh.

> > To be consistent, either these two are the same:
> >
> >   typeset -x FOO
> >   typeset -x FOO=""
> >
> > Or these two are different:
> >
> >   typeset FOO
> >   typeset FOO=""
>
> In the internal namespace, and starting from a name FOO that's never
> been used/does not appear in the process environment, in zsh all of
> these do the same thing:
>
> declare FOO
> local FOO
> export FOO
>
> To use Daniel's "${verb}s a variable" from the other thread, ${verb}
> is 'creates an internal parameter to represent'.  The entirety of
> these two discussion threads is supposed to be about when it is
> appropriate for that to have a default value, and what it means for it
> not to; currently it always has one.
>
> Specifically for "export" there are two additional requirements:
> 1) If the variable already exists with a value, then variable=value is
> added to the environment.
> 2) If a value is later assigned to the variable, then variable=value
> is added to the environment.
>
> Neither of those requirements is met under the stated conditions, so
> nothing is added to the environment.

You are just describing existing behavior.

> This is entirely separate from whether there is a default.

No, it isn't. This is an unsubstantiated claim.

*If* you are going to assign a default value, then this is the value
that should be exported. Otherwise the values will be different, and
there would be an inconsistency.

*Or* just don't assign any default value, and this way both the local
and exported values are also the same.

So clearly the inconsistency has to do with the default.

> Again starting from a previously nonexistent FOO, all of these are
> also the same:
>
> declare FOO=anything
> local FOO=anything
> export FOO=anything
>
> These explicitly do two things:  First ${verb}, and then assign.  So
> for export, the second additional requirement is met, and
> variable=value is added to the environment.

Again, explaining current behavior.

> Is there an inconsistency from the viewpoint of an omniscient
> observer?  Yes.

Good. That's all I'm saying, and that's what Oliver said (zsh's not
exactly consistent).

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27 21:49                         ` Felipe Contreras
@ 2020-11-27 22:06                           ` Bart Schaefer
  2020-11-27 23:35                             ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27 22:06 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 1:49 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> There's no rabbit hole here. It's obvious that zsh's behavior is inconsistent.

The rabbit holes are all the possible tangents that may or may not be
additional inconsistencies, into which we keep plunging, rather than
focusing on the most reasonable way to address the one we started
with.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27 20:54                     ` Bart Schaefer
@ 2020-11-27 22:10                       ` Felipe Contreras
  2020-11-27 22:39                         ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-27 22:10 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 2:54 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Thu, Nov 26, 2020 at 5:30 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > What zsh refers to as "scalar" internally is a string:
> >
> >   char *str; /* value if declared string  (PM_SCALAR)  */
> >
> > From Src/zsh.h (struct param).
>
> This is exactly the discussion I was trying to avoid when I said "in
> the abstract that doesn't matter".
>
> You can't just pull one field out of a union inside a struct and
> ignore the struct itself and the API for field access that goes with
> it.

I am not. I have been looking at the code and fixing some inconsistencies.

From what I can see PM_SCALAR is considered a string, and the thing
I'm putting more emphasis on is the comment: "value if declared
string".

> > So if you didn't mean string, what did you mean?
>
> I meant a struct param, containing the least specific thing so
> represented, as interpreted through all the layers of code that
> implement a dereference of its value when you write $var or any of its
> variations.  Again this doesn't actually matter, which is why I didn't
> spell it out.

The least specific thing--which happens when you do "static struct
param p"--sets the type to PM_SCALAR (0), which from what I can see
for all intents and purposes is a string.

> > And what did you mean by 'so a the only useful "declared but not set"
> > variable is a simple scalar'?
>
> As the very first message in this thread demonstrated, in both bash
> and ksh (call this "example one", and to be pedantic assume that X is
> not inheriting its name or value from somewhere):
>
> typeset -i X
> echo ${X-nil}
> X="garbage"
> echo ${X-nil}
>
> will output
>
> nil
> 0
>
> However (call this "example two"):
>
> typeset -i X
> unset X
> X="garbage"
> echo ${X-nil}
>
> outputs
>
> garbage
>
> The language you quoted from the posix proposal says "otherwise, the
> variable is initially unset".  Given that proposed language, example
> one is incorrect, because an "unset" variable should not retain its
> (in this example) integer properties when assigned a string.

I agree. I would say "imprecise" rather than "incorrect", since if all
these threads taught us anything is that it's not so clear to define
what we mean by "unset", "nil value", or "no value".

> > What simple scalar other than a string is useful "declared but not set"?
>
> Under this interpretation, there isn't any.  That's what I said.  In
> fact the last paragraph of the very first message in this thread:
>
> "Therefore, this isn't as simple as having zsh create an unset
> variable when typeset is given no assignment, because subsequent
> assignment has to preserve the type of the variable, which normally
> does not apply after unset."

So we can interpret what you said as 'so the only useful "declared but
not set" variable is a string' without losing any meaning.

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27 22:10                       ` Felipe Contreras
@ 2020-11-27 22:39                         ` Bart Schaefer
  2020-11-28  0:00                           ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-27 22:39 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 2:10 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> So we can interpret what you said as 'so the only useful "declared but
> not set" variable is a string' without losing any meaning.

That's not correct, but again I was hoping it wasn't necessary to
spell all this out.

All of the following create parameters representing strings, none of
which are useful unless they either have a default or the "preserve
when not set" behavior applies:

typeset -L FOO
typeset -R FOO
typeset -Z FOO
typeset -H FOO
typeset -l FOO
typeset -u FOO



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27 22:06                           ` Bart Schaefer
@ 2020-11-27 23:35                             ` Felipe Contreras
  0 siblings, 0 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-11-27 23:35 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 4:06 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Fri, Nov 27, 2020 at 1:49 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > There's no rabbit hole here. It's obvious that zsh's behavior is inconsistent.
>
> The rabbit holes are all the possible tangents that may or may not be
> additional inconsistencies, into which we keep plunging, rather than
> focusing on the most reasonable way to address the one we started
> with.

Right. But solving the first one (with setopt KSH_TYPESET or
whatever), solves the second one.

So to me they are not unrelated.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-27 22:39                         ` Bart Schaefer
@ 2020-11-28  0:00                           ` Felipe Contreras
  2020-11-28  0:04                             ` Bart Schaefer
  2020-11-28  0:36                             ` The emulation rabbit-hole RE typeset/unset Bart Schaefer
  0 siblings, 2 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-11-28  0:00 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 4:39 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Fri, Nov 27, 2020 at 2:10 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > So we can interpret what you said as 'so the only useful "declared but
> > not set" variable is a string' without losing any meaning.
>
> That's not correct, but again I was hoping it wasn't necessary to
> spell all this out.
>
> All of the following create parameters representing strings, none of
> which are useful unless they either have a default or the "preserve
> when not set" behavior applies:

These are not contradictory unless you commit the fallacy of the converse.

"All animals that bark are mammals" doesn't mean "all mammals bark".

Analogously:

"All unset variables that are useful are strings" doesn't mean "all
strings are unset variables that are useful".

To be pedantic: param_is_useful_unset(p) -> param_type_is_string(p).
Not the other way around.

This is a bit tautological, because if the variable is unset, it can't
be anything else but a string: param_is_unset(p) ->
param_type_is_string(p).

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-28  0:00                           ` Felipe Contreras
@ 2020-11-28  0:04                             ` Bart Schaefer
  2020-11-28 10:52                               ` Felipe Contreras
  2020-11-28  0:36                             ` The emulation rabbit-hole RE typeset/unset Bart Schaefer
  1 sibling, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-28  0:04 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 4:00 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> These are not contradictory unless you commit the fallacy of the converse.

I didn't say they were contradictory.  I said one of them lost meaning.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* The emulation rabbit-hole RE typeset/unset
  2020-11-28  0:00                           ` Felipe Contreras
  2020-11-28  0:04                             ` Bart Schaefer
@ 2020-11-28  0:36                             ` Bart Schaefer
  2020-11-28 11:35                               ` Felipe Contreras
  1 sibling, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-28  0:36 UTC (permalink / raw)
  To: Zsh hackers list

On Fri, Nov 27, 2020 at 4:00 PM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> This is a bit tautological, because if the variable is unset, it can't
> be anything else but a string: param_is_unset(p) ->
> param_type_is_string(p).

This is manifestly not true in ksh and bash, because:

[[ ${foo-nil} = nil ]] implies foo is unset
{ typeset -i foo } declares foo is an integer, not a string
function ff { typeset -i foo; echo ${foo-nil}; } outputs nil

Do we care whether zsh emulates this?



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: More rabbit-holes with unset variables
  2020-11-28  0:04                             ` Bart Schaefer
@ 2020-11-28 10:52                               ` Felipe Contreras
  0 siblings, 0 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-11-28 10:52 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 6:04 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Fri, Nov 27, 2020 at 4:00 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > These are not contradictory unless you commit the fallacy of the converse.
>
> I didn't say they were contradictory.  I said one of them lost meaning.

You said it was not correct. But it is.

You can argue meaning was lost. But it is not incorrect.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: The emulation rabbit-hole RE typeset/unset
  2020-11-28  0:36                             ` The emulation rabbit-hole RE typeset/unset Bart Schaefer
@ 2020-11-28 11:35                               ` Felipe Contreras
  2020-11-28 16:56                                 ` Bart Schaefer
  0 siblings, 1 reply; 40+ messages in thread
From: Felipe Contreras @ 2020-11-28 11:35 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Fri, Nov 27, 2020 at 6:37 PM Bart Schaefer <schaefer@brasslantern.com> wrote:
>
> On Fri, Nov 27, 2020 at 4:00 PM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > This is a bit tautological, because if the variable is unset, it can't
> > be anything else but a string: param_is_unset(p) ->
> > param_type_is_string(p).
>
> This is manifestly not true in ksh and bash, because:
>
> [[ ${foo-nil} = nil ]] implies foo is unset
> { typeset -i foo } declares foo is an integer, not a string
> function ff { typeset -i foo; echo ${foo-nil}; } outputs nil

All right, that is true.

But then, if we have this:

  typeset -i foo

At this point "foo" is unset (in bash and ksh). It does not have a
value, and ${foo-nil} is nil. So for all intents and purposes it is
nil.

Then, we decide to add 1 to it:

  ((foo += 1))

The result is 1 (in bash and ksh).

This is interesting.

Irrespective of the fact that foo is unset, you can still add 1 to it,
so in bash and ksh it doesn't matter that foo initially is *not* 0;
you can still add to it.

So this:

  f() { ((foo +=1 )); echo ${foo-nil}; }

Returns the same in all shells: 1. You don't need to define "foo" first.

So the statement 'the only useful "declared but not set" variable is a
simple scalar' does not seem to be true.

An integer is not a "simple scalar", and seems to be useful unset.

Or am I missing something?

Cheers.

-- 
Felipe Contreras



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: The emulation rabbit-hole RE typeset/unset
  2020-11-28 11:35                               ` Felipe Contreras
@ 2020-11-28 16:56                                 ` Bart Schaefer
  2020-12-01  8:49                                   ` Felipe Contreras
  0 siblings, 1 reply; 40+ messages in thread
From: Bart Schaefer @ 2020-11-28 16:56 UTC (permalink / raw)
  To: Felipe Contreras; +Cc: Zsh hackers list

On Sat, Nov 28, 2020 at 3:36 AM Felipe Contreras
<felipe.contreras@gmail.com> wrote:
>
> An integer is not a "simple scalar", and seems to be useful unset.
>
> Or am I missing something?

Two things.  Possibly three.

One, 'the only useful "declared but not set" variable is a simple
scalar' was a statement on the ambiguity of the austin-group proposal
that you excerpted, which explicitly made "declared but not set"
equivalent to "unset".

Two, that neither bash nor ksh actually does make those two things
equivalent.  Variables in bash and ksh can either have both properties
and values, or only properties, or neither.  Variables in zsh
currently have only the two states of both or neither, because the
latter is the definition of being unset.

This is what we've been saying all along -- zsh currently has no
provision for representing "only properties", and consequently the
only way to get any of the behavior of the properties settable by
typeset options is to provide a default value.  The only thing zsh can
currently represent independent of some value is function scope.

Three, maybe, is that math expression context has a special definition
of the meaning of an undeclared name, which is not the same as the
definition in the rest of the shell.  It's not a parameter expansion
in the normal sense.



^ permalink raw reply	[flat|nested] 40+ messages in thread

* Re: The emulation rabbit-hole RE typeset/unset
  2020-11-28 16:56                                 ` Bart Schaefer
@ 2020-12-01  8:49                                   ` Felipe Contreras
  0 siblings, 0 replies; 40+ messages in thread
From: Felipe Contreras @ 2020-12-01  8:49 UTC (permalink / raw)
  To: Bart Schaefer; +Cc: Zsh hackers list

On Sat, Nov 28, 2020 at 10:56 AM Bart Schaefer
<schaefer@brasslantern.com> wrote:
>
> On Sat, Nov 28, 2020 at 3:36 AM Felipe Contreras
> <felipe.contreras@gmail.com> wrote:
> >
> > An integer is not a "simple scalar", and seems to be useful unset.
> >
> > Or am I missing something?
>
> Two things.  Possibly three.
>
> One, 'the only useful "declared but not set" variable is a simple
> scalar' was a statement on the ambiguity of the austin-group proposal
> that you excerpted, which explicitly made "declared but not set"
> equivalent to "unset".

Yes, but this is a statement of fact. It either is true or it isn't.
And to me it looks like it isn't.

> Two, that neither bash nor ksh actually does make those two things
> equivalent.  Variables in bash and ksh can either have both properties
> and values, or only properties, or neither.  Variables in zsh
> currently have only the two states of both or neither, because the
> latter is the definition of being unset.
>
> This is what we've been saying all along -- zsh currently has no
> provision for representing "only properties", and consequently the
> only way to get any of the behavior of the properties settable by
> typeset options is to provide a default value.  The only thing zsh can
> currently represent independent of some value is function scope.

I'm not talking about what is currently the case in zsh. I'm talking
about what should ideally be the case. For bash, for ksh, for POSIX,
and consequently for zsh.

> Three, maybe, is that math expression context has a special definition
> of the meaning of an undeclared name, which is not the same as the
> definition in the rest of the shell.  It's not a parameter expansion
> in the normal sense.

No, but it's still useful.

Cheers.

-- 
Felipe Contreras


^ permalink raw reply	[flat|nested] 40+ messages in thread

end of thread, other threads:[~2020-12-01  8:50 UTC | newest]

Thread overview: 40+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-11-25  7:02 More rabbit-holes with unset variables Bart Schaefer
2020-11-25 13:19 ` Stephane Chazelas
2020-11-25 22:17   ` Felipe Contreras
2020-11-26  6:10     ` Stephane Chazelas
2020-11-26  7:20       ` Felipe Contreras
2020-11-26 11:21         ` Oliver Kiddle
2020-11-26 11:29           ` Roman Perepelitsa
2020-11-26 19:08             ` Felipe Contreras
2020-11-26 19:29           ` Felipe Contreras
2020-11-26 21:13           ` Bart Schaefer
2020-11-26 21:31             ` Felipe Contreras
2020-11-26 23:29               ` Bart Schaefer
2020-11-26 23:32                 ` Bart Schaefer
2020-11-26 23:53                 ` Felipe Contreras
2020-11-27  0:23                   ` Bart Schaefer
2020-11-27  1:51                     ` Felipe Contreras
2020-11-27 20:01                       ` Bart Schaefer
2020-11-27 21:49                         ` Felipe Contreras
2020-11-27 22:06                           ` Bart Schaefer
2020-11-27 23:35                             ` Felipe Contreras
2020-11-26 22:07       ` Bart Schaefer
2020-11-26 20:41     ` Bart Schaefer
2020-11-26 20:49       ` Stephane Chazelas
2020-11-26 21:20       ` Felipe Contreras
2020-11-26 22:41         ` Bart Schaefer
2020-11-26 23:45           ` Felipe Contreras
2020-11-27  0:09             ` Bart Schaefer
2020-11-27  0:30               ` Felipe Contreras
2020-11-27  0:51                 ` Bart Schaefer
2020-11-27  1:30                   ` Felipe Contreras
2020-11-27 20:54                     ` Bart Schaefer
2020-11-27 22:10                       ` Felipe Contreras
2020-11-27 22:39                         ` Bart Schaefer
2020-11-28  0:00                           ` Felipe Contreras
2020-11-28  0:04                             ` Bart Schaefer
2020-11-28 10:52                               ` Felipe Contreras
2020-11-28  0:36                             ` The emulation rabbit-hole RE typeset/unset Bart Schaefer
2020-11-28 11:35                               ` Felipe Contreras
2020-11-28 16:56                                 ` Bart Schaefer
2020-12-01  8:49                                   ` Felipe Contreras

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).