[PATCH] move zsh reserved words out of the way when invoked in sh/ksh emulation

[PATCH] move zsh reserved words out of the way when invoked in sh/ksh emulation

[Martijn Dekker]

[-- Attachment #1: Type: text/plain, Size: 2308 bytes --]

In zsh-workers 27648 and 27650 (commit 8ac97f33, back in 2010), the
"repeat" reserved word was disabled when zsh starts in any emulation
mode, to avoid interfering with sh scripts when zsh is invoked as sh.

(Note that this change disabled the (t)csh-style 'repeat' loop even if
zsh is invoked as csh. It seems unlikely that was intended.)

Meanwhile, I've identified five other reserved words that are unique to
zsh *and* are perfectly plausible function names, so could kill an sh or
ksh/bash script:

	end
	float
	foreach
	integer
	nocorrect

So I think they (and 'repeat') should be disabled if zsh is invoked in
sh or ksh emulation mode.

'float' and 'integer' are part of the typeset family, so the underlying
builtins would be exposed. It would then be inconsistent not to do the
same with the rest of the typeset family:

	declare
	export
	local
	readonly
	typeset

Thankfully, the KSH_TYPESET option still works. It should revert to
being automatically enabled for ksh emulation and not for sh emulation,
as in zsh up to 5.0.8.

Treating the whole typeset family as builtins subject to KSH_TYPESET
actually increases the accuracy of sh and ksh emulation modes.

One issue is that POSIX explicitly specifies 'export' and 'readonly' as
special builtins and not as reserved words. So POSIX scripts should be
able to override them at least with an alias.[*] This breaks if they are
reserved words.

For the incomplete csh emulation mode (does anyone actually use this?)
it's probably not worth bothering to disable anything, so instead of
testing for !EMULATION(EMULATE_ZSH) we should test for
EMULATION(EMULATE_SH) || EMULATION(EMULATE_KSH). This restores 'repeat'
when zsh is invoked as csh.

By the way, contrary to claims in the documentation, the KSH_TYPESET
option was never completely obsolete. Even with the post-5.0.8 reserved
word interface activated, the behaviour of MAGIC_EQUAL_SUBST still takes
KSH_TYPESET into account.

Thanks,

- Martijn

[*] This is useful if a sh script needs to parse the output of 'export
-p' (or 'readonly -p'). It is not possible to grep their output reliably
as entries may contain newlines. The only reliable POSIX way is to have
the shell parse it, which is done by temporarily aliasing 'export' to a
handler shell function and doing an 'eval "$(export -p)"'.

[-- Attachment #2: init_builtins.patch --]
[-- Type: text/plain, Size: 5775 bytes --]

diff --git a/Doc/Zsh/builtins.yo b/Doc/Zsh/builtins.yo
index f460e48..9763a6c 100644
--- a/Doc/Zsh/builtins.yo
+++ b/Doc/Zsh/builtins.yo
@@ -1866,8 +1866,9 @@ tt(integer), tt(local), tt(readonly) or tt(typeset) is matched when the
 line is parsed (N.B. not when it is executed).  In this case the arguments
 are parsed as assignments, except that the `tt(+=)' syntax and the
 tt(GLOB_ASSIGN) option are not supported, and scalar values after tt(=)
-are em(not) split further into words, even if expanded (regardless of the
-setting of the tt(KSH_TYPESET) option; this option is obsolete).
+are em(not) split further into words, even if expanded (unless tt(zsh)
+was invoked as tt(sh) or tt(ksh), in which case it depends on the
+setting of the tt(KSH_TYPESET) option).
 
 Examples of the differences between command and reserved word parsing:
 
diff --git a/Doc/Zsh/grammar.yo b/Doc/Zsh/grammar.yo
index d2c7cd2..bb720ce 100644
--- a/Doc/Zsh/grammar.yo
+++ b/Doc/Zsh/grammar.yo
@@ -155,6 +155,10 @@ item(tt(nocorrect))(
 Spelling correction is not done on any of the words.  This must appear
 before any other precommand modifier, as it is interpreted immediately,
 before any parsing is done.  It has no effect in non-interactive shells.
+
+The tt(nocorrect) reserved word is disabled by default when the
+shell is invoked as tt(sh) or tt(ksh).  It can be enabled
+with the command `tt(enable -r nocorrect)'.
 )
 findex(noglob)
 item(tt(noglob))(
@@ -229,8 +233,8 @@ which must evaluate to a number var(n).
 var(list) is then executed var(n) times.
 
 The tt(repeat) syntax is disabled by default when the
-shell starts in a mode emulating another shell.  It can be enabled
-with the command `tt(enable -r repeat)'
+shell is invoked as tt(sh) or tt(ksh).  It can be enabled
+with the command `tt(enable -r repeat)'.
 )
 findex(case)
 cindex(case selection)
@@ -442,6 +446,10 @@ A short form of the arithmetic tt(for) command.
 findex(foreach)
 item(tt(foreach) var(name) ... tt(LPAR()) var(word) ... tt(RPAR()) var(list) tt(end))(
 Another form of tt(for).
+
+The tt(foreach) syntax is disabled by default when the
+shell is invoked as tt(sh) or tt(ksh).  It can be enabled
+with the command `tt(enable -r foreach end)'.
 )
 item(tt(while) var(list) tt({) var(list) tt(}))(
 An alternative form of tt(while).  Note the limitations on the form of
diff --git a/Doc/Zsh/options.yo b/Doc/Zsh/options.yo
index 25b3d57..782ad34 100644
--- a/Doc/Zsh/options.yo
+++ b/Doc/Zsh/options.yo
@@ -2061,17 +2061,27 @@ pindex(NOKSHTYPESET)
 cindex(argument splitting, in typeset etc.)
 cindex(ksh, argument splitting in typeset)
 item(tt(KSH_TYPESET))(
-This option is now obsolete: a better appropximation to the behaviour of
-other shells is obtained with the reserved word interface to
-tt(declare), tt(export), tt(float), tt(integer), tt(local), tt(readonly)
-and tt(typeset).  Note that the option is only applied when the reserved
-word interface is em(not) in use.
-
-Alters the way arguments to the tt(typeset) family of commands, including
+If tt(zsh) is invoked as tt(sh) or tt(ksh), this option alters
+the way arguments to the tt(typeset) family of commands, including
 tt(declare), tt(export), tt(float), tt(integer), tt(local) and
 tt(readonly), are processed.  Without this option, zsh will perform normal
 word splitting after command and parameter expansion in arguments of an
 assignment; with it, word splitting does not take place in those cases.
+
+By default, this option is superseded by a more natural way of
+parsing assignment-arguments: the reserved word interface to
+tt(declare), tt(export), tt(float), tt(integer), tt(local), tt(readonly)
+and tt(typeset).  The tt(KSH_TYPESET) option is only applied to these when
+the reserved word interface is em(not) in use -- that is, either when the
+shell was invoked as tt(sh) or tt(ksh), or when the reserved words are
+explicitly disabled using tt(disable -r), exposing the underlying builtin
+commands directly.
+
+Regardless of the above, this option always modifies the behaviour of the
+tt(MAGIC_EQUAL_SUBST) option.
+
+For tt(ksh) emulation mode, this option is enabled by default; otherwise,
+it is disabled by default.
 )
 pindex(KSH_ZERO_SUBSCRIPT)
 pindex(NO_KSH_ZERO_SUBSCRIPT)
diff --git a/Src/builtin.c b/Src/builtin.c
index fb59738..f557d0c 100644
--- a/Src/builtin.c
+++ b/Src/builtin.c
@@ -210,10 +210,29 @@ freebuiltinnode(HashNode hn)
 void
 init_builtins(void)
 {
-    if (!EMULATION(EMULATE_ZSH)) {
-	HashNode hn = reswdtab->getnode2(reswdtab, "repeat");
-	if (hn)
-	    reswdtab->disablenode(hn, 0);
+    if (EMULATION(EMULATE_SH) || EMULATION(EMULATE_KSH)) {
+	static const char *disablereswd[] = {
+	/* reserved words that may kill (k)sh scripts */
+		"end",
+		"foreach",
+		"nocorrect",
+		"repeat",
+	/* the following revert to builtins subject to KSH_TYPESET */
+		"declare",
+		"export",
+		"float",
+		"integer",
+		"local",
+		"readonly",
+		"typeset",
+		0 };
+	const char **n;
+	HashNode hn;
+	for (n = disablereswd; *n; n++) {
+	    hn = reswdtab->getnode2(reswdtab, *n);
+	    if (hn)
+		reswdtab->disablenode(hn, 0);
+	}
     }
 }
 
diff --git a/Src/options.c b/Src/options.c
index 590652e..6145b77 100644
--- a/Src/options.c
+++ b/Src/options.c
@@ -176,7 +176,7 @@ static struct optname optns[] = {
 {{NULL, "kshautoload",	      OPT_EMULATE|OPT_BOURNE},	 KSHAUTOLOAD},
 {{NULL, "kshglob",	      OPT_EMULATE|OPT_KSH},	 KSHGLOB},
 {{NULL, "kshoptionprint",     OPT_EMULATE|OPT_KSH},	 KSHOPTIONPRINT},
-{{NULL, "kshtypeset",	      0},			 KSHTYPESET},
+{{NULL, "kshtypeset",	      OPT_EMULATE|OPT_KSH},	 KSHTYPESET},
 {{NULL, "kshzerosubscript",   0},			 KSHZEROSUBSCRIPT},
 {{NULL, "listambiguous",      OPT_ALL},			 LISTAMBIGUOUS},
 {{NULL, "listbeep",	      OPT_ALL},			 LISTBEEP},

Re: [PATCH] move zsh reserved words out of the way when invoked in sh/ksh emulation

[Martijn Dekker]

Any opinions on zsh-workers/42533, please?

Thanks,

- M.

Re: [PATCH] move zsh reserved words out of the way when invoked in sh/ksh emulation

[Peter Stephenson]

On Mon, 26 Mar 2018 00:23:07 +0200
Martijn Dekker <martijn@inlv.org> wrote:
> Meanwhile, I've identified five other reserved words that are unique
> to zsh *and* are perfectly plausible function names, so could kill an
> sh or ksh/bash script:
> 
> 	end
> 	float
> 	foreach
> 	integer
> 	nocorrect
> 
> So I think they (and 'repeat') should be disabled if zsh is invoked in
> sh or ksh emulation mode.

As far as I can see this isn't a particularly big deal as long a
properly documented seeing as "enable -r" works OK if people want
to mix code in emulation mode (in which case they can expect
to have the odd extra hoop to jump through).

> 'float' and 'integer' are part of the typeset family, so the
> underlying builtins would be exposed. It would then be inconsistent
> not to do the same with the rest of the typeset family:
> 
> 	declare
> 	export
> 	local
> 	readonly
> 	typeset
> 
> Thankfully, the KSH_TYPESET option still works. It should revert to
> being automatically enabled for ksh emulation and not for sh
> emulation, as in zsh up to 5.0.8.

This is when I got really confused when I first saw this patch.  This
breaks the syntax supporting inline parentheses present in POSIX mode
like any other and replaces it with what was always a kludge (though
does remain useful in certain cases), and furthermore doesn't have any
clear functional motivation.  So my eyes just glazed over.

pws

Re: [PATCH] move zsh reserved words out of the way when invoked in sh/ksh emulation

[Bart Schaefer]

On Apr 16,  9:32am, Peter Stephenson wrote:
} Subject: Re: [PATCH] move zsh reserved words out of the way when invoked i
}
} On Mon, 26 Mar 2018 00:23:07 +0200
} Martijn Dekker <martijn@inlv.org> wrote:
} > 'float' and 'integer' are part of the typeset family, so the
} > underlying builtins would be exposed. It would then be inconsistent
} > not to do the same with the rest of the typeset family:
} > 
} > 	declare
} > 	export
} > 	local
} > 	readonly
} > 	typeset
} 
} This is when I got really confused when I first saw this patch.  This
} breaks the syntax supporting inline parentheses present in POSIX mode

This is fairly bad.  We made those into reserved words particularly for
this reason.

I would not support breaking this.