List of unresolved issues

List of unresolved issues

[Bart Schaefer]

Some of these may actually have been resolved and I just forgot/missed it.
Also, these are only issues raised in messages I sent myself.

10403		<(...) doesn't really create a FIFO
10346		rebuilding zshpaths.h
users/2979	assigning a string containing `:' to $path
10187		combining multiple zstyle commands into one
10169		%(l.t.f) and line wrapping in prompts
users/2973	clear to end of display
10154		$param:x and the FAQ
10080		move compctl support functions out of Misc/
10071		xtrace behavior of `.' and `source'
10005		bogus "no such job" / STAT_NOPRINT
9996		vared keymap
9982		comptest (lack of) verbosity
9962		renumbering comptest
9945		somebody check my work ...
9778		hanging on loops with lots of output
9762		nameddir assoc elements [*]
9702		configure switch for mymods.conf (cf. 9727)
9575		export arrays like ksh
9095		getopts is broken
9062		suspending `.' / `source'
9000		parameter formatting / ksh export (cf. 8674)
8861		posix module
8780		format of error messages
8758		locales and math (cf. 8792)
8698		bad node type in freenode (related to 9095?)
8619		locales and globbing
8378		configure --zsh-mem-debug

There are probably others from even longer ago, but my eyes are starting
to glaze over; I think I've reached the point of diminishing returns.

[*] as long as we're talking about fixing the lexer for AAs anyway ...

Also:

Document the kshautoload problem with the completion system and how one
works around it with zcompile.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

RE: List of unresolved issues

[Andrej Borsenkow]

> Some of these may actually have been resolved and I just 
> forgot/missed it.
> Also, these are only issues raised in messages I sent myself.
> 
> 10403		<(...) doesn't really create a FIFO

Yes, please ... :-)

-andrej

Re: List of unresolved issues

[Peter Stephenson]

"Bart Schaefer" wrote:
> Some of these may actually have been resolved and I just forgot/missed it.
> Also, these are only issues raised in messages I sent myself.

Oh, well, that's my weekend done for.

I looked at the discussion we had earlier about typeset -x in functions
(accidentally, by going to the top of the folder and starting to reply to
things 3000 messages ago... just as well I realised) and frankly it doesn't
increase my enthusiasm for doing anything about it.  It's all due to the
confusion of `typeset' and `local' in ksh.  With the documented fact that
`export' is equivalent to `typeset -gx', zsh is even now the only shell to
handle this in a consistent way.

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@CambridgeSiliconRadio.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: List of unresolved issues

[Bart Schaefer]

On Apr 12,  9:55pm, Peter Stephenson wrote:
> 
> I looked at the discussion we had earlier about typeset -x in functions
> (accidentally, by going to the top of the folder and starting to reply to
> things 3000 messages ago... just as well I realised) and frankly it doesn't
> increase my enthusiasm for doing anything about it.  It's all due to the
> confusion of `typeset' and `local' in ksh.  With the documented fact that
> `export' is equivalent to `typeset -gx', zsh is even now the only shell to
> handle this in a consistent way.

Are we both talking about the same things here?

There's already been a patch that makes `typeset -x' == `typeset -gx'.

The only remaining question (I think) is whether a variable that is already
a local can be put into the environment, and if so, whether such a variable
can be automatically removed from the environment again when the scope ends.

E.g.:

	local foo
	export foo	# in zsh this is a no-op;
			# in ksh it creates an environment string that is
			# removed from the environment at function exit

This doesn't have anything to do with confusion of `typeset' and `local';
it has to do with `export' NOT being confused with `typeset'.

How does that make zsh more consistent?

Re: List of unresolved issues

[Peter Stephenson]

> There's already been a patch that makes `typeset -x' == `typeset -gx'.

Oh, that passed me by.  In that case zsh definitely isn't consistent, either.

> 	local foo
> 	export foo	# in zsh this is a no-op;
> 			# in ksh it creates an environment string that is
> 			# removed from the environment at function exit
> 
> This doesn't have anything to do with confusion of `typeset' and `local';
> it has to do with `export' NOT being confused with `typeset'.
> 
> How does that make zsh more consistent?

Because it's documented that local variables don't become exported:  this is
the only thing I was proposing to change, I suspect it's only a propagated
limitation that's always been around.  It's also documented that
globals don't become visible if there's a local in the way and I am
absolutely and positively *not* proposing to change that, ever: every
change to the scoping rules doubles the number of bugs, and this would make
them exponentiate.  The only thing I could imagine which would work would
be a command to blow things away at one or all levels of scoping.

The only reason there is no confusion over what export does is because it
now explicitly has the flags -gx.  If the only proposals are to screw it up
in some other way, I'll leave it to someone else to do.

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

PATCH: local exports

[Peter Stephenson]

It seems to be agreed that not having local parameters exportable is a
bug.  This is an attempt to fix it.  It works, so far as I've tried, for
both normal and special parameters (with or without the -h option).  The
exported value should now be consistent (modulo any other problems with
exporting variables irrespective of their being local or not) with the
in-shell value, which seems to me the only sensible way of doing this.

I'm sure this needs more testing.  I removed some special cases that
prevented exports of locals in various places in the code, but it's
possible there are some others hanging around.

I made `typeset +g -x' respect the explicit `+g', and I allowed `local' to
take `-x' without forcing on `-g' --- I take it this is uncontroversial.
This was necessary to get around the -g-with--x kludge.  Ksh 88 doesn't
have that and its `typeset -x' behaves more or less like `typeset +g -x'.
However, apparently ksh 88 leaves the global value in the environment
together with the local; zsh now deletes it, which seems to correspond to
the user's intention at the expense of a fairly minor inefficiency.

I hope there's nothing to excite controversy in this patch.


On the other matter of handling the combination of localness and
exportedness: I haven't touched this, since I'd probably get it wrong.
Having confused myself over this, I've come to these conclusions:

1 - We don't actually need the -g-kludge for typeset -x any more; it
doesn't correspond to ksh 88 (see above), wasn't present or documented in
zsh until one of the current development series, and isn't consistent with
the behaviour of the remaining flags.

2 - `export' in any case behaves like it does in ksh, though in zsh's case
that means the effect of `typeset -xg', as currently documented.  The zsh
behaviour was always as now, I believe.  The ksh 88 behaviour makes
`export' and `typeset -x' behave differently, too, but it's undocumented.

3 - In fact, none of this interaction between exportedness and globalness
seems to be documented in the ksh 88 manual page.

4 - Therefore it's hard to see the use of an option or different emulation
settings, but it's possible I've missed some other issue.  We're agreed the
old behaviour was broken, so we don't want to emulate that.  If an option
for ksh compatibility is required, the base zsh behaviour should be as
clean as possible, i.e. no unexpected combinations (even if documented).

4 - Consequently I think we can remove the -g => -xg kludge in either case.
But I didn't quite follow why it was there.  If it was simply a workaround
for the -x being ignored otherwise, then maybe we can simply get rid of it
and put an end to the matter.


Index: Doc/Zsh/builtins.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/builtins.yo,v
retrieving revision 1.2
diff -u -r1.2 builtins.yo
--- Doc/Zsh/builtins.yo	2000/04/01 20:43:44	1.2
+++ Doc/Zsh/builtins.yo	2000/04/13 13:26:28
@@ -595,9 +595,10 @@
 endsitem()
 )
 findex(local)
-item(tt(local) [ {tt(PLUS())|tt(-)}tt(AEFLRUZahilrtu) [var(n)]] [ var(name)[tt(=)var(value)] ] ...)(
-Same as tt(typeset), except that the options tt(-g), tt(-x) and
-tt(-f) are not permitted.
+item(tt(local) [ {tt(PLUS())|tt(-)}tt(AEFLRUZahilrtux) [var(n)]] [ var(name)[tt(=)var(value)] ] ...)(
+Same as tt(typeset), except that the options tt(-g), and
+tt(-f) are not permitted.  In this case the tt(-x) option does not force
+the use of tt(-g), i.e. exported variables will be local to functions.
 )
 findex(log)
 vindex(watch, use of)
@@ -1003,9 +1004,7 @@
 function completes.  See
 ifzman(`Local Parameters' in zmanref(zshparam))\
 ifnzman(noderef(Local Parameters))\
-.  Local parameters are not exported unless tt(ALL_EXPORT) is set, in
-which case the parameter is exported em(only) when var(name) does not
-already exist.  The same rules apply to special shell parameters, which
+.  The same rules apply to special shell parameters, which
 retain their special attributes when made local.
 
 For each var(name)tt(=)var(value) assignment, the parameter
@@ -1161,7 +1160,9 @@
 )
 item(tt(-x))(
 Mark for automatic export to the environment of subsequently
-executed commands.
+executed commands.  Currently this implies the option tt(-g), unless tt(+g)
+is also explicitly given, in other words the parameter is not made local to
+the enclosing function.  This is for compatibility with other shells.
 )
 enditem()
 )
Index: Doc/Zsh/params.yo
===================================================================
RCS file: /cvsroot/zsh/zsh/Doc/Zsh/params.yo,v
retrieving revision 1.2
diff -u -r1.2 params.yo
--- Doc/Zsh/params.yo	2000/04/01 20:43:44	1.2
+++ Doc/Zsh/params.yo	2000/04/13 13:26:28
@@ -215,21 +215,19 @@
 
 Special parameters may also be made local; they retain their special
 attributes unless either the existing or the newly-created parameter
-has the tt(-h) (hide) attribute.  This may have unexpected effects.
-Firstly, there is no default value, so if there is no assigment at the
+has the tt(-h) (hide) attribute.  This may have unexpected effects:
+there is no default value, so if there is no assigment at the
 point the variable is made local, it will be set to an empty value (or zero
-in the case of integers).  Secondly, special parameters which are made
-local will not be exported (as with other parameters), so that the global
-value of the parameter remains present in the environment if it is already
-there.  This should be particularly noted in the case of tt(PATH): the
-shell will use the local version of tt(PATH) for finding programmes, but
-programmes using the shell's environment will inherit the global version.
+in the case of integers).  
 The following:
 
 example(typeset PATH=/new/directory:$PATH)
 
-is valid for temporarily allowing the shell to find the programs in
-tt(/new/directory) inside a function.
+is valid for temporarily allowing the shell or programmes called from it to
+find the programs in tt(/new/directory) inside a function.
+
+Note that the restriction in older versions of zsh that local parameters
+were never exported has been removed.
 
 texinode(Parameters Set By The Shell)(Parameters Used By The Shell)(Local Parameters)(Parameters)
 sect(Parameters Set By The Shell)
Index: Src/builtin.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
retrieving revision 1.6
diff -u -r1.6 builtin.c
--- Src/builtin.c	2000/04/13 08:43:39	1.6
+++ Src/builtin.c	2000/04/13 13:26:29
@@ -79,7 +79,7 @@
     BUILTIN("jobs", 0, bin_fg, 0, -1, BIN_JOBS, "dlpZrs", NULL),
     BUILTIN("kill", 0, bin_kill, 0, -1, 0, NULL, NULL),
     BUILTIN("let", 0, bin_let, 1, -1, 0, NULL, NULL),
-    BUILTIN("local", BINF_TYPEOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL, bin_typeset, 0, -1, 0, "AEFLRTUZahilrtu", NULL),
+    BUILTIN("local", BINF_TYPEOPTS | BINF_MAGICEQUALS | BINF_PSPECIAL, bin_typeset, 0, -1, 0, "AEFLRTUZahilrtux", NULL),
     BUILTIN("log", 0, bin_log, 0, 0, 0, NULL, NULL),
     BUILTIN("logout", 0, bin_break, 0, 1, BIN_LOGOUT, NULL, NULL),
 
@@ -644,14 +644,12 @@
     setsparam("OLDPWD", ztrdup(oldpwd));
 
     pm = (Param) paramtab->getnode(paramtab, "PWD");
-    if (!(pm->flags & PM_EXPORTED) &&
-	(!pm->level || (isset(ALLEXPORT) && !pm->old))) {
+    if (!(pm->flags & PM_EXPORTED)) {
 	pm->flags |= PM_EXPORTED;
 	pm->env = addenv("PWD", pwd, pm->flags);
     }
     pm = (Param) paramtab->getnode(paramtab, "OLDPWD");
-    if (!(pm->flags & PM_EXPORTED) &&
-	(!pm->level || (isset(ALLEXPORT) && !pm->old))) {
+    if (!(pm->flags & PM_EXPORTED)) {
 	pm->flags |= PM_EXPORTED;
 	pm->env = addenv("OLDPWD", oldpwd, pm->flags);
     }
@@ -1589,7 +1587,6 @@
     }
 
     /*
-     * According to the manual, local parameters don't get exported.
      * A parameter will be local if
      * 1. we are re-using an existing local parameter
      *    or
@@ -1598,10 +1595,6 @@
      *     or
      *   ii. we are creating a new local parameter
      */
-    if ((usepm && pm->level) ||
-	(!usepm && (pm || (locallevel && (on & PM_LOCAL)))))
-	on &= ~PM_EXPORTED;
-
     if (usepm) {
 	on &= ~PM_LOCAL;
 	if (!on && !roff && !value) {
@@ -1630,9 +1623,7 @@
 	    if (pm->flags & PM_EXPORTED) {
 		if (!(pm->flags & PM_UNSET) && !pm->env && !value)
 		    pm->env = addenv(pname, getsparam(pname), pm->flags);
-	    } else if (pm->env &&
-		       (!pm->level || (isset(ALLEXPORT) && !pm->old &&
-				       !(pm->flags & PM_HASHELEM)))) {
+	    } else if (pm->env && !(pm->flags & PM_HASHELEM)) {
 		delenv(pm->env);
 		zsfree(pm->env);
 		pm->env = NULL;
@@ -1708,7 +1699,11 @@
 	tpm->old = pm->old;
 	tpm->level = pm->level;
 	tpm->ct = pm->ct;
-	tpm->env = pm->env;
+	if (pm->env) {
+	    delenv(pm->env);
+	    zsfree(pm->env);
+	}
+	tpm->env = pm->env = NULL;
 
 	pm->old = tpm;
 	/*
@@ -1725,7 +1720,6 @@
 	    pm->ct = auxlen;
 	else
 	    pm->ct = 0;
-	pm->env = NULL;
     } else {
 	/*
 	 * Create a new node for a parameter with the flags in `on' minus the
@@ -1856,7 +1850,7 @@
 	return 0;
     }
 
-    if (!ops['g'] && !ops['x'])
+    if ((!ops['g'] && !ops['x']) || ops['g'] == 2 || *name == 'l')
 	on |= PM_LOCAL;
 
     if (on & PM_TIED) {
Index: Src/params.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/params.c,v
retrieving revision 1.3
diff -u -r1.3 params.c
--- Src/params.c	2000/04/04 12:02:04	1.3
+++ Src/params.c	2000/04/13 13:26:29
@@ -643,13 +643,21 @@
 	} else {
 	    pm = (Param) zcalloc(sizeof *pm);
 	    if ((pm->old = oldpm)) {
-		/* needed to avoid freeing oldpm */
+		/*
+		 * needed to avoid freeing oldpm, but we do take it
+		 * out of the environment when it's hidden.
+		 */
+		if (oldpm->env) {
+		    delenv(oldpm->env);
+		    zsfree(oldpm->env);
+		    oldpm->env = NULL;
+		}
 		paramtab->removenode(paramtab, name);
 	    }
 	    paramtab->addnode(paramtab, ztrdup(name), pm);
 	}
 
-	if (isset(ALLEXPORT) && !oldpm && !(flags & PM_HASHELEM))
+	if (isset(ALLEXPORT) && !(flags & PM_HASHELEM))
 	    flags |= PM_EXPORTED;
     } else {
 	pm = (Param) zhalloc(sizeof *pm);
@@ -1446,11 +1454,30 @@
 }
 
 /**/
+void
+export_param(Param pm)
+{
+    char buf[(sizeof(zlong) * 8) + 4], *val;
+
+    if (PM_TYPE(pm->flags) == PM_INTEGER)
+	convbase(val = buf, pm->gets.ifn(pm), pm->ct);
+    else if (pm->flags & (PM_EFLOAT|PM_FFLOAT))
+	val = convfloat(pm->gets.ffn(pm), pm->ct,
+			pm->flags, NULL);
+    else
+	val = pm->gets.cfn(pm);
+    if (pm->env)
+	pm->env = replenv(pm->env, val, pm->flags);
+    else {
+	pm->flags |= PM_EXPORTED;
+	pm->env = addenv(pm->nam, val, pm->flags);
+    }
+}
+
+/**/
 mod_export void
 setstrvalue(Value v, char *val)
 {
-    char buf[(sizeof(zlong) * 8) + 4];
-
     if (v->pm->flags & PM_READONLY) {
 	zerr("read-only variable: %s", v->pm->nam, 0);
 	zsfree(val);
@@ -1523,22 +1550,10 @@
 	break;
     }
     if ((!v->pm->env && !(v->pm->flags & PM_EXPORTED) &&
-	 !(isset(ALLEXPORT) && !v->pm->old && !(v->pm->flags & PM_HASHELEM))) ||
+	 !(isset(ALLEXPORT) && !(v->pm->flags & PM_HASHELEM))) ||
 	(v->pm->flags & PM_ARRAY) || v->pm->ename)
 	return;
-    if (PM_TYPE(v->pm->flags) == PM_INTEGER)
-	convbase(val = buf, v->pm->gets.ifn(v->pm), v->pm->ct);
-    else if (v->pm->flags & (PM_EFLOAT|PM_FFLOAT))
-	val = convfloat(v->pm->gets.ffn(v->pm), v->pm->ct,
-			v->pm->flags, NULL);
-    else
-	val = v->pm->gets.cfn(v->pm);
-    if (v->pm->env)
-	v->pm->env = replenv(v->pm->env, val, v->pm->flags);
-    else {
-	v->pm->flags |= PM_EXPORTED;
-	v->pm->env = addenv(v->pm->nam, val, v->pm->flags);
-    }
+    export_param(v->pm);
 }
 
 /**/
@@ -1969,6 +1984,15 @@
 	if ((PM_TYPE(oldpm->flags) == PM_SCALAR) &&
 	    oldpm->sets.cfn == strsetfn)
 	    adduserdir(oldpm->nam, oldpm->u.str, 0, 0);
+	if (oldpm->flags & PM_EXPORTED) {
+	    /*
+	     * Re-export the old value which we removed in typeset_single().
+	     * I don't think we need to test for ALL_EXPORT here, since if
+	     * it was used to export the parameter originally the parmeter
+	     * should still have the PM_EXPORTED flag.
+	     */
+	    export_param(oldpm);
+	}
     }
 
     paramtab->freenode((HashNode) pm); /* free parameter node */
@@ -2745,7 +2769,7 @@
      */
     if (t == path)
 	cmdnamtab->emptytable(cmdnamtab);
-    if ((pm->flags & PM_HASHELEM) || (isset(ALLEXPORT) ? !!pm->old : pm->level))
+    if (pm->flags & PM_HASHELEM)
 	return;
     u = t ? zjoin(t, ':', 1) : "";
     len_s = strlen(s);
@@ -3013,7 +3037,11 @@
 	    pm->flags = (tpm->flags & ~PM_NORESTORE);
 	    pm->level = tpm->level;
 	    pm->ct = tpm->ct;
-	    pm->env = tpm->env;
+	    if (pm->env) {
+		delenv(pm->env);
+		zsfree(pm->env);
+	    }
+	    pm->env = NULL;
 
 	    if (!(tpm->flags & PM_NORESTORE))
 		switch (PM_TYPE(pm->flags)) {
@@ -3035,6 +3063,9 @@
 		    break;
 		}
 	    zfree(tpm, sizeof(*tpm));
+
+	    if (pm->flags & PM_EXPORTED)
+		export_param(pm);
 	} else
 	    unsetparam_pm(pm, 0, 0);
     }

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Re: PATCH: local exports

[Bart Schaefer]

On Apr 13,  9:56am, Peter Stephenson wrote:
} Subject: Re: List of unresolved issues
}
} > There's already been a patch that makes `typeset -x' == `typeset -gx'.
} 
} Oh, that passed me by.  In that case zsh definitely isn't consistent, either.

The reason for that patch is that having `typeset -x' require `-g' when
used inside a shell function breaks compatibility with 3.0 and earlier
zsh.  I could be argued out of it (or simply overruled, I supposed) but
it caused some of my autoloaded functions to fail (e.g., a cvs wrapper
that I use to force CVSROOT=$(<CVS/Root) into the environment, which is
intended to have it remain in the environment after the function exits).

On Apr 13,  3:10pm, Peter Stephenson wrote:
} Subject: PATCH: local exports
}
} It seems to be agreed that not having local parameters exportable is a
} bug.  This is an attempt to fix it.

It's only a bug in ksh emulation.  It's not a bug in bare zsh.  Exporting
locals is a possibly-useful feature, but no more.

} I'm sure this needs more testing.  I removed some special cases that
} prevented exports of locals in various places in the code, but it's
} possible there are some others hanging around.

The following seems to be a pretty effective test:

  unenv() {
    local x
    x=($(typeset +x))
    local $x
    unset $x
    /usr/bin/printenv	# $path is (correctly?) nuked by unset PATH
  }

The output on my system differs if I run this function twice.  The first
time it is:

  TERM=xterm
  HOME=/home/schaefer
  _=/usr/bin/printenv
  COLUMNS=80
  LINES=24

and the second time:

  _=/usr/bin/printenv

which is what I would have expected both times.

Futher, if I do this:

  printenv | sort > /tmp/before
  unenv
  printenv | sort > /tmp/after
  diff /tmp/before /tmp/after

I get no differences in the before and after output, which is exactly as
it should be.
 
} I made `typeset +g -x' respect the explicit `+g', and I allowed `local' to
} take `-x' without forcing on `-g' --- I take it this is uncontroversial.

I think this is fine.

} This was necessary to get around the -g-with--x kludge.  Ksh 88 doesn't
} have that and its `typeset -x' behaves more or less like `typeset +g -x'.

This is why I keep talking about ksh emulation.  I want to be able to
keep exporting things from functions with `typeset -x' the way I have
for the entire time I've been using zsh, but it causes problems when
running scripts written for ksh.

} 1 - We don't actually need the -g-kludge for typeset -x any more; it
} doesn't correspond to ksh 88 (see above), wasn't present or documented in
} zsh until one of the current development series, and isn't consistent with
} the behaviour of the remaining flags.

I don't know what you mean about "wasn't present or documented ...".  It
has been documented all through the 2.xx and 3.x series that `export' is
the same as `typeset -x'.  Changing the behavior of `typeset -x' to imply
localness is a serious incompatibility.

} 2 - `export' in any case behaves like it does in ksh, though in zsh's case
} that means the effect of `typeset -xg', as currently documented.  The zsh
} behaviour was always as now, I believe.  The ksh 88 behaviour makes
} `export' and `typeset -x' behave differently, too, but it's undocumented.

I agree with all those statements.

} 3 - In fact, none of this interaction between exportedness and globalness
} seems to be documented in the ksh 88 manual page.

However, we should be using the ksh93 manual and/or the new ksh book, not
the ksh88 manual.  There are links to current doc on www.kornshell.com.

} 4 - Therefore it's hard to see the use of an option or different emulation
} settings, but it's possible I've missed some other issue.  We're agreed the
} old behaviour was broken, so we don't want to emulate that.

Apparently we're not agreed that the old behavior was broken, for some
definition of "the old behavior."

} 4 - Consequently I think we can remove the -g => -xg kludge in either case.
} But I didn't quite follow why it was there.  If it was simply a workaround
} for the -x being ignored otherwise, then maybe we can simply get rid of it
} and put an end to the matter.

It's not just that the -x was ignored, it's that its meaning changed in a
way not compatible with older zsh.

With the sole exception of the ksh emulation mode question, the behavior
after 10728 is precisely what I intended to be suggesting all along.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

PATCH: Re: PATCH: local exports

[Peter Stephenson]

Bart Schaefer wrote:
> The following seems to be a pretty effective test:
> 
>   unenv() {
>     local x
>     x=($(typeset +x))
>     local $x
>     unset $x
>     /usr/bin/printenv	# $path is (correctly?) nuked by unset PATH
>   }
> 
> The output on my system differs if I run this function twice.  The first
> time it is:
> 
>   TERM=xterm
>   HOME=/home/schaefer
>   _=/usr/bin/printenv
>   COLUMNS=80
>   LINES=24

Oddly, this wasn't actually my fault; it came from the import of special
parameters at startup.  The code set the variable, then tried to replace
the value in the environment with the value generated internlly, for
consistency.  Unfortunately it was trying to replace it into the old
environment, which failed.  This returned null, which was set to the env
element of the parameter struct.  Hence when the new code looked for this
to see if the value was in the environment already it failed, so never
deleted it.  (It was doing this on the `local', not the unset, since the
local parameter wasn't being exported.)  This usually didn't show up
because the values in the environment were OK anyway, as in this case, so
the fact that they didn't get replaced didn't matter.

> } This was necessary to get around the -g-with--x kludge.  Ksh 88 doesn't
> } have that and its `typeset -x' behaves more or less like `typeset +g -x'.
> 
> This is why I keep talking about ksh emulation.  I want to be able to
> keep exporting things from functions with `typeset -x' the way I have
> for the entire time I've been using zsh, but it causes problems when
> running scripts written for ksh.

OK, so it's a ksh compatibility vs. old zsh compatibility issue.  The
problem with using an option is that in this case the latter is the kludge,
but needs to be enabled by default.  There isn't a good way round that.
If somebody wants to write an option they can.  It could be called e.g.
GLOBAL_EXPORT and be on by default; at least that tells you that something's
up.

Index: Src/params.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/params.c,v
retrieving revision 1.4
diff -u -r1.4 params.c
--- Src/params.c	2000/04/13 14:22:47	1.4
+++ Src/params.c	2000/04/13 17:51:26
@@ -454,6 +454,7 @@
 {
     Param ip, pm;
     char **new_environ, **envp, **envp2, **sigptr, **t;
+    char **old_environ = environ;
     char buf[50], *str, *iname;
     int num_env, oae = opts[ALLEXPORT];
 #ifdef HAVE_UNAME
@@ -516,9 +517,12 @@
 		    pm->flags |= PM_EXPORTED;
 		    pm->env = *envp++ = ztrdup(*envp2);
 		    *envp = NULL;
-		    if (pm->flags & PM_SPECIAL)
+		    if (pm->flags & PM_SPECIAL) {
+			environ = new_environ;
 			pm->env = replenv(pm->env, getsparam(pm->nam),
 					  pm->flags);
+			environ = old_environ;
+		    }
 		}
 	    }
 	    *str = '=';

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Re: List of unresolved issues (update)

[Bart Schaefer]

What I believe to be the status on these issues.  New ones:

11036		read -q
11030		_message
11023		_complete_help and matcher-list
11022		print -s (during completion)
11018		_find

Yeah, I know, it's not fair to list things that came up over the weekend.
Too bad, I want to collect them together somewhere.  (I suppose I could
file them as bugs on sourceforge ... do people find that bug tracker to
be worthwhile?)

The following have been fixed or acted upon, or were improperly listed:

} 10187		combining multiple zstyle commands into one
} 10154		$param:x and the FAQ
} 8698		bad node type in freenode (related to 9095?)
} 8619		locales and globbing
} Document the kshautoload problem with the completion system and how one
} works around it with zcompile.

The following are going to be left as they are in 3.1.7-pre-2, so that
counts as a resolution:

} users/2973	clear to end of display
} 10403		<(...) doesn't really create a FIFO
	(This one got some documentation at least.  I still think it
	could be fixed, but probably not entirely within getproc(),
	due to the race condition PWS noted.)
} 10071		xtrace behavior of `.' and `source'
} 10005		bogus "no such job" / STAT_NOPRINT
	(I guess that's what Sven's "Sniff." meant.)
} 9996		vared keymap
} 9945		somebody check my work ...
} 9062		suspending `.' / `source'

The following will be dealt with for 4.0, along with other reorgs:

} 10080		move compctl support functions out of Misc/
} 9962		renumbering comptest

The following have been discussed a bit further, but no conclusion yet:

} 10346		rebuilding zshpaths.h
} 9778		hanging on loops with lots of output
} 8780		format of error messages

And last, these are still entirely un-addressed:

} 10169		%(l.t.f) and line wrapping in prompts
} users/2979	assigning a string containing `:' to $path
} 9982		comptest (lack of) verbosity
} 9762		nameddir assoc elements
} 9702		configure switch for mymods.conf (cf. 9727)
} 9575		export arrays like ksh
} 9095		getopts is broken
} 9000		parameter formatting / ksh export (cf. 8674)
} 8861		posix module
} 8758		locales and math (cf. 8792)
} 8378		configure --zsh-mem-debug

Of those, I think 9095 is the only really crucial one.  The others might
warrant mention on the BUGS list or some such.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: List of unresolved issues (update)

[Adam Spiers]

Bart Schaefer (schaefer@candle.brasslantern.com) wrote:
> Too bad, I want to collect them together somewhere.  (I suppose I could
> file them as bugs on sourceforge ... do people find that bug tracker to
> be worthwhile?)

Yes, I think it's worthwhile.  If bugs/bug lists are only posted to
the list, unless someone (you in this case) is good enough to
regularly keep track of and post outstanding bugs, they are very hard
to spot in the archive.  If they're all in the bug tracker it's easy
for someone with a bit of free time (hah! as if...) to pick something
to work on.

The sourceforge bug tracker seems perfectly adequate to me, but if
there's a problem with it I've missed, we could always stuff your list
in Etc/BUGS.  Or if we decide to use the bug tracker as the definitive
list, we should certainly mention it in Etc/BUGS.

Re: List of unresolved issues (update)

[Peter Stephenson]

> Of those, I think 9095 is the only really crucial one.  The others might
> warrant mention on the BUGS list or some such.

The bug was this:

function oops {
    while getopts :m: f
    do
        echo $OPTIND is $f with $OPTARG
        shift $((OPTIND - 1))
    done
    print -r - $*
}

% oops -m 664
2 is m with 664
2 is ? with k
funky

(Note that the apparently correct display of `funky' is a coincidence, it
comes from incorrectly shifting by 2-1 twice.)

The fix looks easy, which in this particular chunk of code makes me very
suspicious (see the comment).  However, I'll commit it anyway until we find
the drawback.  We need comprehensive tests for getopts, including nested
functions, asap.

I now get:

% oops -m 664 funky
3 is m with 664
funky
% oops -m664 funky
2 is m with 664
funky

which I think is right.

Index: Src/builtin.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/builtin.c,v
retrieving revision 1.10
diff -u -r1.10 builtin.c
--- Src/builtin.c	2000/04/30 14:48:49	1.10
+++ Src/builtin.c	2000/05/02 09:47:05
@@ -3077,7 +3077,15 @@
 	    p = ztrdup(args[zoptind++]);
 	} else
 	    p = metafy(str+optcind, lenstr-optcind, META_DUP);
-	optcind = ztrlen(args[zoptind - 1]);
+	/*
+	 * Careful:  I've just changed the following two lines from
+	 *   optcind = ztrlen(args[zoptind - 1]);
+	 * and it's a rigorous theorem that every change in getopts breaks
+	 * something.  See zsh-workers/9095 for the bug fixed here.
+	 *   PWS 2000/05/02
+	 */
+	optcind = 0;
+	zoptind++;
 	zsfree(zoptarg);
 	zoptarg = p;
     } else {

-- 
Peter Stephenson <pws@cambridgesiliconradio.com>
Cambridge Silicon Radio, Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070

Re: List of unresolved issues

[Bart Schaefer]

On Apr 14, 10:18am, Sven Wischnowsky wrote:
} Subject: Re: List of unresolved issues
}
} > } > 10187		combining multiple zstyle commands into one
} > 
} > Actually, I now think the shell function implementation is fine.
} 
} I don't understand why it uses case instead of if/else, though:

Easier to change later if any other options need special-casing.  When
I started writing it I thought some might.
 
} The `zMULTstyle' and the missing `builtin's: I would be against naming
} it just `zstyle' for two reasons: 1) speed in the completion functions
} which don't need this shortcut syntax because it's only for defining
} styles

No problem.  Name it something else, alias it back to zstyle, and make
sure the completion autoloads use -U.

} and 2) _complete_help needs to define a zstyle function itself

Too bad we can't declare `local functions[zstyle]'.  Pretend I didn't
just suggest that.
 
} So, I haven't added/committed this because there probably is a better
} name than zmultstyle. Yes?

Is "zstyle+" bad because it would mean a file with a '+' in it's name?

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: List of unresolved issues

[Sven Wischnowsky]

Bart Schaefer wrote:

> On Apr 13,  2:46pm, Sven Wischnowsky wrote:
> } Subject: Re: List of unresolved issues
> } 
> } Bart Schaefer wrote:
> } 
> } > Some of these may actually have been resolved and I just forgot/missed it.
> } > Also, these are only issues raised in messages I sent myself.
> } > 
> } > ...
> } > 10187		combining multiple zstyle commands into one
> } 
> } I offer to write it when you folks decide that we want it.
> 
> Actually, I now think the shell function implementation is fine.  Maybe
> just copy that function into Functions/Misc?
> 
> I don't care about the -L output part.

In that case I would have voted for the function, too.

I don't understand why it uses case instead of if/else, though:

  zmultstyle() {
    if [[ $1 = -* ]]; then
      zstyle "$@"
    else
      setopt localoptions noksharrays
      integer i
      local context="$1"
      1=''
      for ((i=2; $#; ++i)); do
        if [[ $i -gt $# || "$argv[i]" == '+' ]]; then
          zstyle "$context${(@)argv[1,i-1]}"
  	shift "i > $# ? $# : i"  # Stupid shift error on i > $#
  	i=1
        fi
      done
    fi
  }

The `zMULTstyle' and the missing `builtin's: I would be against naming
it just `zstyle' for two reasons: 1) speed in the completion functions
which don't need this shortcut syntax because it's only for defining
styles and 2) _complete_help needs to define a zstyle function itself
to collect the style information (if requested). And _complete_help is 
too lazy to restore it because it thinks there shouldn't be a function 
with the name `zstyle' because of 1).

So, I haven't added/committed this because there probably is a better
name than zmultstyle. Yes?

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: List of unresolved issues

[Bart Schaefer]

On Apr 13,  2:46pm, Sven Wischnowsky wrote:
} Subject: Re: List of unresolved issues
}
} 
} Bart Schaefer wrote:
} 
} > Some of these may actually have been resolved and I just forgot/missed it.
} > Also, these are only issues raised in messages I sent myself.
} > 
} > ...
} > 10187		combining multiple zstyle commands into one
} 
} I offer to write it when you folks decide that we want it.

Actually, I now think the shell function implementation is fine.  Maybe
just copy that function into Functions/Misc?

I don't care about the -L output part.

} > 10071		xtrace behavior of `.' and `source'
} 
} I still find our behaviour much more consistent and it gives us both
} possibilities. So if at all, I think we should use a option for it
} (yet another `emulate ksh' thing).

I, too, think it should stay as it is, but I didn't want the decision to
be made by simply being overlooked.

} > 9778		hanging on loops with lots of output
} 
} Oops. I had forgotten about this one. Does it still happen?

At last report it did, but then I've never been able to reproduce it
myself.

} > ...
} > 9062		suspending `.' / `source'
} 
} An option? Tested only after the init scripts have been read?

Again, I don't care, this could stay the way it is.  It's another bash/ksh
thing though, IIRC.

} > 8698		bad node type in freenode (related to 9095?)
} 
} I sent a patch for this in 8721 (at least it solved it for me).

OK, that's fine; as I said, I may have missed the solutions to some of
these.

} And there is also still your FPATH-in-the-environment-and-then-that-if
} thing...

Yes, I was intending to look at that myself.  However, I'm about to go
out of town for a week (starting tomorrow (i.e. Friday) morning) so if
somebody else has a chance to examine it, please go ahead.

I may answer some more mail tomorrow before heading out, but there's not
even a telephone where I'll be next week, so past tomorrow you won't hear
from me until after Easter.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Re: List of unresolved issues

[Sven Wischnowsky]

Bart Schaefer wrote:

> Some of these may actually have been resolved and I just forgot/missed it.
> Also, these are only issues raised in messages I sent myself.
> 
> ...
> 10187		combining multiple zstyle commands into one

I have the feeling I should have an opinion about this one but it
appears that I haven't. If my setup is anything to go by (58 calls to
zstyle ;-), it *would* simplify some of them, but I normally add new
ones with \M-w\C-y, so...
I offer to write it when you folks decide that we want it. I don't
think I'll offer to make `zstyle -L' print them in this form, though
(but I haven't really thought about how difficult that would be).

> ...
> 10080		move compctl support functions out of Misc/

Maybe together with the other function cleanup in the completion
system.

> 10071		xtrace behavior of `.' and `source'

I still find our behaviour much more consistent and it gives us both
possibilities. So if at all, I think we should use a option for it
(yet another `emulate ksh' thing).

> 10005		bogus "no such job" / STAT_NOPRINT

Sniff.

> 9996		vared keymap

A very simple way would be to just allow users to give a keymap via an 
option that would be used as a `local' keymap the way menu-selection
does it, i.e. all functions but undefined-key in that map override the
definitions in the normally used keymap.
This is the place where I wished Anthony would have found the time to
give us the code for the stuff he told us about in 6938.

> ...
> 9778		hanging on loops with lots of output

Oops. I had forgotten about this one. Does it still happen?

> ...
> 9062		suspending `.' / `source'

An option? Tested only after the init scripts have been read?

> ...
> 8698		bad node type in freenode (related to 9095?)

I sent a patch for this in 8721 (at least it solved it for me). And
Peter had a look at and a patch for it when he came back. And since
then the whole node-stuff has been removed (the nodes in question were
the ones used to store executable code).

> 8619		locales and globbing

The last reply to this was 8625 -- and I still think this should be
done with a glob flag or an option and a glob flag. Or something.


And there is also still your FPATH-in-the-environment-and-then-that-if
thing...

Bye
 Sven


--
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de