more dependencies on emulation

more dependencies on emulation

[Zefram]

-----BEGIN PGP SIGNED MESSAGE-----

Continuing on from my FUNCTION_ARGZERO patch in article 1669, this
patch removes some direct dependencies on emulation from subst.c.
After this, the only use of emulation is in the emulate builtin itself,
and in initialisation (where it's difficult to avoid).

There are two distinct uses of emulation addressed here.  One affects
the position of filename expansion in the order of expansion; I add a
new option, SH_FILE_SUBST, to control this.  This option is affected
by changing emulation mode, so the current behaviour will be unchanged
except where the new option is explicitly used.

The second use of emulation is to control whether the $foo[2] subscript
placement will work; sh and ksh don't recognise it as a subscript.
The patch makes this be controlled by the KSH_ARRAYS option, as it should
have been all along.  The KSH_ARRAYS option is also changed to be set
by default when emulating sh, which was previously not essential.

Both of the above optional behaviours are also documented by this patch.
Previously they were totally undocumented.

My comments in article 1669 about the option patch in article 1275 apply
here also.  Applying this patch by hand if you don't have the option
patch applied is not difficult.  If anyone wants an updated copy of the
option patch, mail me.

 -zefram

      Index: Doc/zshexpn.man
      ===================================================================
      RCS file: /home/zefram/usr/cvsroot/zsh/Doc/zshexpn.man,v
      retrieving revision 1.9
      diff -c -r1.9 zshexpn.man
      *** zshexpn.man	1996/07/16 14:26:42	1.9
      --- zshexpn.man	1996/07/18 11:36:36
      ***************
      *** 25,30 ****
      --- 25,40 ----
        .BR ' ,
        and \fB"\fP are removed and the result is subjected to
        \fIfilename expansion\fP followed by \fIfilename generation\fP.
      + .PP
      + If the
      + .B SH_FILE_SUBST
      + option is set, the order of expansion is modified
      + for compatibility with sh and ksh.
      + .I Filename expansion
      + is performed immediately after
      + .IR "alias substitution" ,
      + preceding the set of five substitutions mentioned above.
      + is
        .SH FILENAME EXPANSION
        Each word is checked to see if it begins with an unquoted
        .BR ~ .
      Index: Doc/zshoptions.man
      ===================================================================
      RCS file: /home/zefram/usr/cvsroot/zsh/Doc/zshoptions.man,v
      retrieving revision 1.11
      diff -c -r1.11 zshoptions.man
      *** zshoptions.man	1996/07/16 16:35:47	1.11
      --- zshoptions.man	1996/07/18 11:32:19
      ***************
      *** 292,299 ****
        .TP
        \fBKSH_ARRAYS\fP
        Emulate \fIksh\fP array handling as closely as possible. If this option
      ! is set, array elements are numbered from zero and an array parameter
      ! without subscript refers to the first element instead of the whole array.
        .TP
        \fBKSH_OPTION_PRINT\fP
        Alters the way options settings are printed.
      --- 292,301 ----
        .TP
        \fBKSH_ARRAYS\fP
        Emulate \fIksh\fP array handling as closely as possible. If this option
      ! is set, array elements are numbered from zero, an array parameter
      ! without subscript refers to the first element instead of the whole array,
      ! and braces are required to delimit a subscript (``${path[2]}'' rather
      ! than just ``$path[2]'').
        .TP
        \fBKSH_OPTION_PRINT\fP
        Alters the way options settings are printed.
      ***************
      *** 448,453 ****
      --- 450,464 ----
        .TP
        \fBRM_STAR_SILENT\fP (\-\fBH\fP)
        Do not query the user before executing "rm *" or "rm path/*".
      + .TP
      + \fBSH_FILE_SUBST\fP
      + Perform filename expansion (e.g., ~ expansion)
      + .I before
      + parateter expansion, command substitution, arithmetic expansion
      + and brace expansion.
      + If this option is unset, it is performed
      + .I after
      + brace expansion, so things like ``~$USERNAME'' ``~{pfalstad,rc}'' will work.
        .TP
        \fBSH_GLOB\fP
        Disables the special meaning of `(', `|', `)' and '<' for globbing the
      Index: Doc/zshparam.man
      ===================================================================
      RCS file: /home/zefram/usr/cvsroot/zsh/Doc/zshparam.man,v
      retrieving revision 1.8
      diff -c -r1.8 zshparam.man
      *** zshparam.man	1996/07/16 14:26:43	1.8
      --- zshparam.man	1996/07/18 11:36:18
      ***************
      *** 33,43 ****
      --- 33,45 ----
        expansion as if it were surrounded by "$((...))".  
        The elements are numbered beginning with 1 unless the
        \fBKSH_ARRAYS\fP option is set when they are numbered from zero.
      + .PP
        A subscript of the form \fB[*]\fP or \fB[@]\fP evaluates to all
        elements of an array; there is no difference between the two
        except when they appear within double quotes.
        "$foo[*]" evaluates to "$foo[1] $foo[2] ...", while
        "$foo[@]" evaluates to "$foo[1]" "$foo[2]", etc.
      + .PP
        A subscript of the form \fB[\fIexp1\fP,\fIexp2\fB]\fR
        selects all elements in the range \fIexp1\fP to \fIexp2\fP,
        inclusive.
      ***************
      *** 51,56 ****
      --- 53,67 ----
        case the subscripts specify a substring to be extracted.
        For example, if \fBFOO\fP is set to \fBfoobar\fP, then
        \fBecho $FOO[2,5]\fP prints \fBooba\fP.
      + .PP
      + Subscripts may be used inside braces used to delimit a parameter name, thus
      + .B ${foo[2]}
      + is equivalent to
      + .BR $foo[2] .
      + If the
      + .B KSH_ARRAYS
      + option is set, the braced form is the only one that will
      + work, the subscript otherwise not being treated specially.
        .PP
        If a subscript is used on the left side of an assignment the selected
        range is replaced by the expression on the right side.
      Index: Src/globals.h
      ===================================================================
      RCS file: /home/zefram/usr/cvsroot/zsh/Src/globals.h,v
      retrieving revision 1.11
      diff -c -r1.11 globals.h
      *** globals.h	1996/07/16 16:35:55	1.11
      --- globals.h	1996/07/18 11:43:38
      ***************
      *** 717,723 ****
            {"ignoreeof", 		'7',  0,    0},
            {"interactive", 		'i',  'i',  OPT_SPECIAL},
            {"interactivecomments", 	'k',  0,    OPT_EMULATE|OPT_BOURNE},
      !     {"ksharrays", 		0,    0,    OPT_EMULATE|OPT_KSH},
            {"kshoptionprint", 		0,    0,    OPT_EMULATE|OPT_KSH},
            {"listambiguous", 		0,    0,    0},
            {"listbeep", 		0,    0,    OPT_ALL},
      --- 717,723 ----
            {"ignoreeof", 		'7',  0,    0},
            {"interactive", 		'i',  'i',  OPT_SPECIAL},
            {"interactivecomments", 	'k',  0,    OPT_EMULATE|OPT_BOURNE},
      !     {"ksharrays", 		0,    0,    OPT_EMULATE|OPT_BOURNE},
            {"kshoptionprint", 		0,    0,    OPT_EMULATE|OPT_KSH},
            {"listambiguous", 		0,    0,    0},
            {"listbeep", 		0,    0,    OPT_ALL},
      ***************
      *** 750,755 ****
      --- 750,756 ----
            {"rcs", 			x'f', 0,    OPT_ALL},
            {"recexact", 		'S',  0,    0},
            {"rmstarsilent", 		'H',  0,    OPT_BOURNE},
      +     {"shfilesubst",		0,    0,    OPT_EMULATE|OPT_BOURNE},
            {"shglob", 			0,    0,    OPT_EMULATE|OPT_BOURNE},
            {"shinstdin", 		's',  's',  OPT_SPECIAL},
            {"shortloops", 		0,    0,    OPT_ALL},
      Index: Src/subst.c
      ===================================================================
      RCS file: /home/zefram/usr/cvsroot/zsh/Src/subst.c,v
      retrieving revision 1.10
      diff -c -r1.10 subst.c
      *** subst.c	1996/07/16 14:26:58	1.10
      --- subst.c	1996/07/18 11:30:07
      ***************
      *** 66,74 ****
        		setdata(node, (void *) getoutputfile(str));	/* =(...) */
        	    if (!getdata(node))
        		return;
      ! 	}
      ! 	else {
      ! 	    if (emulation == EMULATE_SH || emulation == EMULATE_KSH)
        		filesub((char **)getaddrdata(node), flags & 3);
        	    if (!(node = stringsubst(list, node, flags & 4)))
        		return;
      --- 66,73 ----
        		setdata(node, (void *) getoutputfile(str));	/* =(...) */
        	    if (!getdata(node))
        		return;
      ! 	} else {
      ! 	    if (isset(SHFILESUBST))
        		filesub((char **)getaddrdata(node), flags & 3);
        	    if (!(node = stringsubst(list, node, flags & 4)))
        		return;
      ***************
      *** 80,86 ****
        	    if (unset(IGNOREBRACES) && !(flags & 4))
        		while (hasbraces(getdata(node)))
        		    xpandbraces(list, &node);
      ! 	    if (emulation != EMULATE_SH && emulation != EMULATE_KSH)
        		filesub((char **)getaddrdata(node), flags & 3);
        	} else if (!(flags & 4))
        	    uremnode(list, node);
      --- 79,85 ----
        	    if (unset(IGNOREBRACES) && !(flags & 4))
        		while (hasbraces(getdata(node)))
        		    xpandbraces(list, &node);
      ! 	    if (unset(SHFILESUBST))
        		filesub((char **)getaddrdata(node), flags & 3);
        	} else if (!(flags & 4))
        	    uremnode(list, node);
      ***************
      *** 970,978 ****
        	copied = 1;
        	*s = sav;
        	v = (Value) NULL;
      !     } else if (!(v = getvalue(&s, (emulation != EMULATE_KSH &&
      ! 				   emulation != EMULATE_SH) ||
      ! 				  inbrace ? 1 : -1))) {
        	vunset = 1;
            } else if ((isarr = v->isarr)) {
        	aval = getarrvalue(v);
      --- 969,975 ----
        	copied = 1;
        	*s = sav;
        	v = (Value) NULL;
      !     } else if (!(v = getvalue(&s, (unset(KSHARRAYS) || inbrace) ? 1 : -1))) {
        	vunset = 1;
            } else if ((isarr = v->isarr)) {
        	aval = getarrvalue(v);
      ***************
      *** 1209,1216 ****
        	}
        	if (colf) {
        	    s--;
      ! 	    if (((emulation != EMULATE_KSH &&
      ! 		  emulation != EMULATE_SH) || inbrace)) {
        		if (!isarr)
        		    modify(&val, &s);
        		else {
      --- 1206,1212 ----
        	}
        	if (colf) {
        	    s--;
      ! 	    if (unset(KSHARRAYS) || inbrace) {
        		if (!isarr)
        		    modify(&val, &s);
        		else {
      Index: Src/zsh.h
      ===================================================================
      RCS file: /home/zefram/usr/cvsroot/zsh/Src/zsh.h,v
      retrieving revision 1.14
      diff -c -r1.14 zsh.h
      *** zsh.h	1996/07/16 16:35:56	1.14
      --- zsh.h	1996/07/18 10:57:22
      ***************
      *** 1132,1137 ****
      --- 1132,1138 ----
            RCS,
            RECEXACT,
            RMSTARSILENT,
      +     SHFILESUBST,
            SHGLOB,
            SHINSTDIN,
            SHORTLOOPS,

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMe4oBXD/+HJTpU/hAQFajAP/SZugoTHBNnfyAnuG+7od5QVY7qUSfvy4
Ojv5UL0d0oTNePQzPgvu0m1JuJWO5QnJJBWnurwN70utjueWNEzxRAhVnOPO9Wb/
9jt6yy528wtGOZ17+ausyZvhoyvmkORNT/z+iuTOFhKwvEONTyrQspRwpO6xA1kd
91Bl91xrLes=
=Y2Nd
-----END PGP SIGNATURE-----

Re: more dependencies on emulation

[Bart Schaefer]

On Jul 18,  3:23pm, Zefram wrote:
> Subject: more dependencies on emulation
> -----BEGIN PGP SIGNED MESSAGE-----
> 
> There are two distinct uses of emulation addressed here.  One affects
> the position of filename expansion in the order of expansion; I add a
> new option, SH_FILE_SUBST, to control this.  [...]
> 
> The second use of emulation is to control whether the $foo[2] subscript
> placement will work; sh and ksh don't recognise it as a subscript.
> The patch makes this be controlled by the KSH_ARRAYS option, as it should
> have been all along.  The KSH_ARRAYS option is also changed to be set
> by default when emulating sh, which was previously not essential.

[....]

>       + \fBSH_FILE_SUBST\fP
>       + If this option is unset, it is performed
>       + .I after
>       + brace expansion, so things like ``~$USERNAME'' ``~{pfalstad,rc}'' will work.

Two minor nits:

If zsh were really emulating sh, arrays wouldn't work at all.  So it's a
bit dubious to force ksh syntax only, given that it affects the indices
as well as the syntax.

I don't have a ksh to try this, but ~{pfalstad,rc} works fine in Bash
(that is, the ~ is expanded), even though ~$USER doesn't expand tilde.
In zsh's ksh emulation mode with this patch applied, ~$USER works, but
~{pfalstad,rc} fails unless IGNORE_BRACES is also unset.  There doesn't
seem to be any way to get the Bash behavior (maybe not important), but
the manual should probably mention the IGNORE_BRACES requirement if it
is going to use ~{...} in this example.

Re: more dependencies on emulation

[Zoltan Hidvegi]

Zefram wrote:
> Continuing on from my FUNCTION_ARGZERO patch in article 1669, this
> patch removes some direct dependencies on emulation from subst.c.
> After this, the only use of emulation is in the emulate builtin itself,
> and in initialisation (where it's difficult to avoid).
> 
> There are two distinct uses of emulation addressed here.  One affects
> the position of filename expansion in the order of expansion; I add a
> new option, SH_FILE_SUBST, to control this.  This option is affected
> by changing emulation mode, so the current behaviour will be unchanged
> except where the new option is explicitly used.

I did not want to introduce new options like FUNCTION_ARGZERO and
SH_FILE_SUBST because I consider these options useful only in sh/ksh
compatibility mode.  Increasing the number of settable options makes it
more and more difficult to write zsh functions which are independent of the
current options setting.  But my solition was certainly not very good.

> The second use of emulation is to control whether the $foo[2] subscript
> placement will work; sh and ksh don't recognise it as a subscript.
> The patch makes this be controlled by the KSH_ARRAYS option, as it should
> have been all along.  The KSH_ARRAYS option is also changed to be set
> by default when emulating sh, which was previously not essential.

That's fine for me.

> Both of the above optional behaviours are also documented by this patch.
> Previously they were totally undocumented.

I wanted to write a compatibility section in zshmisc.man but that major
crash we had last weekend took all of my time.

> My comments in article 1669 about the option patch in article 1275 apply
> here also.  Applying this patch by hand if you don't have the option
> patch applied is not difficult.  If anyone wants an updated copy of the
> option patch, mail me.

I'd like one.  I wanted to include this in pre3 but again I had no time.

Zoltan

Re: more dependencies on emulation

[Zoltan Hidvegi]

Bart Schaefer wrote:
> If zsh were really emulating sh, arrays wouldn't work at all.  So it's a
> bit dubious to force ksh syntax only, given that it affects the indices
> as well as the syntax.

In sh ${foo[1]} is a syntax error so it does not introdueces any
incompatibility.

> I don't have a ksh to try this, but ~{pfalstad,rc} works fine in Bash

% ksh
$ echo ${.sh.version}
Version 12/28/93d
$ echo ~{hzoli,root}
~hzoli ~root

pdksh behaves similarily.

> (that is, the ~ is expanded), even though ~$USER doesn't expand tilde.
> In zsh's ksh emulation mode with this patch applied, ~$USER works, but

Do you mean that it gives the home directory of $USER?  It did not do that
before the patch.

> ~{pfalstad,rc} fails unless IGNORE_BRACES is also unset.  There doesn't

IGNORE_BRACES should probably be not set if zsh is invoked as ksh.  I'll
change that.

There is no standard for brace expansion and probably that's why shells are
differerent here.

Zoltan

Re: more dependencies on emulation

[Bart Schaefer]

On Jul 19,  2:12am, Zoltan Hidvegi wrote:
> Subject: Re: more dependencies on emulation
> Zefram wrote:
> > After this, the only use of emulation is in the emulate builtin itself,
> > and in initialisation (where it's difficult to avoid).
> 
> I did not want to introduce new options like FUNCTION_ARGZERO and
> SH_FILE_SUBST because I consider these options useful only in sh/ksh
> compatibility mode.

Does that mean you're not planning to include these patches in 3.0?
Or does this:

> But my solition was certainly not very good.

mean that you ARE planning to include them?

> Increasing the number of settable options makes it
> more and more difficult to write zsh functions which are independent of the
> current options setting.

Actually, the only options that make this difficult are those that (1)
affect non-interactive behavior and (2) are not tied to an emulation.
As long as the "emulate" command sets/unsets all the appropriately-
tagged options, you can always do:

	zshfoo() {
	    setopt localoptions
	    emulate zsh
	    # Now do stuff using default zsh settings
	}

Note, though, that for this to work "emulate" should never turn OFF
localoptions -- I note that it turns it ON for ksh, so right now it
probably also turns it off sometimes.

> > My comments in article 1669 about the option patch in article 1275 apply
> > here also.  Applying this patch by hand if you don't have the option
> > patch applied is not difficult.  If anyone wants an updated copy of the
> > option patch, mail me.
> 
> I'd like one.  I wanted to include this in pre3 but again I had no time.

Ooooh.  I think I'd rather it stayed out for 3.0.

Re: more dependencies on emulation

[Bart Schaefer]

On Jul 19,  3:34am, Zoltan Hidvegi wrote:
> Subject: Re: more dependencies on emulation
> > In zsh's ksh emulation mode with this patch applied, ~$USER works, but
> 
> Do you mean that it gives the home directory of $USER?  It did not do that
> before the patch.

Right; I meant the patch is correct in that case, but wrong in the other
case (unless IGNORE_BRACES is not set).

> > ~{pfalstad,rc} fails unless IGNORE_BRACES is also unset.
> 
> IGNORE_BRACES should probably be not set if zsh is invoked as ksh.  I'll
> change that.

That's an OK solution too.

> There is no standard for brace expansion and probably that's why shells are
> differerent here.

Yup.  The only remaining question is, do we care about being able to do
the Bash-equivalent case?  I suspect not.

Re: more dependencies on emulation

[Zefram]

>If zsh were really emulating sh, arrays wouldn't work at all.  So it's a
>bit dubious to force ksh syntax only, given that it affects the indices
>as well as the syntax.

In sh, $foo[1] is equivalent to ${foo}[1].  Consequently "emulate sh"
will have to affect this part of the syntax.  The other aspects of
arrays don't need to be affected by sh emulation, true, but it seems
rather stupid to make this distinction.

>I don't have a ksh to try this, but ~{pfalstad,rc} works fine in Bash
>(that is, the ~ is expanded), even though ~$USER doesn't expand tilde.
>In zsh's ksh emulation mode with this patch applied,

This patch doesn't affect that aspect of behaviour -- "emulate ksh" and
"emulate zsh" will affect this expansion behaviour exactly as they did
before.  It only affects behaviour when the new option is individually
changed.

>                                                     ~$USER works, but
>~{pfalstad,rc} fails unless IGNORE_BRACES is also unset.  There doesn't
>seem to be any way to get the Bash behavior (maybe not important), but
>the manual should probably mention the IGNORE_BRACES requirement if it
>is going to use ~{...} in this example.

Maybe.  The documentation phrases things in terms of the SH_FILE_SUBST
option, which is independent of IGNORE_BRACES.  A note about
IGNORE_BRACES would certainly be reasonable, but it's not essential.

-zefram

Re: more dependencies on emulation

[Zefram]

>% ksh
>$ echo ${.sh.version}

Now *there's* a good way to handle special variables.  I'd had exactly
this idea (start them all with .) before, but didn't realise anyone had
actually implemented that in a real shell.  Unfortunately it's far too
late to fix all our portability problems this way.

>IGNORE_BRACES should probably be not set if zsh is invoked as ksh.  I'll
>change that.

I was wondering about that.  My ksh at home, pdksh, does do brace expansion.

>There is no standard for brace expansion and probably that's why shells are
>differerent here.

Indeed.  I think the current behaviours are just fine.

-zefram

Re: more dependencies on emulation

[Zefram]

>Yup.  The only remaining question is, do we care about being able to do
>the Bash-equivalent case?  I suspect not.

"emulate bash"?  Of course, we'd have to conditionally do
"malloc(100000);sleep(5);" in loop().

Maybe not such a good idea.

-zefram

Re: more dependencies on emulation

[Zefram]

>Actually, the only options that make this difficult are those that (1)
>affect non-interactive behavior and (2) are not tied to an emulation.
>As long as the "emulate" command sets/unsets all the appropriately-
>tagged options, you can always do:
>
>	zshfoo() {
>	    setopt localoptions
>	    emulate zsh
>	    # Now do stuff using default zsh settings
>	}
>
>Note, though, that for this to work "emulate" should never turn OFF
>localoptions -- I note that it turns it ON for ksh, so right now it
>probably also turns it off sometimes.

This one's easy.  Execute emulate BEFORE setting localoptions.  The
setting of localoptions at the *end* of the function is what determines
which options are restored.

I hadn't considered that use of emulation before.  Do you think maybe
the emulation mode should be restored by localoptions in the way
options are?  Even after my patches, the emulation does have some
direct effect -- the single-letter option names.  I don't think that
should be changed.

-zefram

Re: more dependencies on emulation

[Bart Schaefer]

On Jul 19,  3:26pm, Zefram wrote:
> Subject: Re: more dependencies on emulation
> >
> >	zshfoo() {
> >	    emulate zsh
> >	    setopt localoptions
> >	    # Now do stuff using default zsh settings
> >	}
> 
> I hadn't considered that use of emulation before.  Do you think maybe
> the emulation mode should be restored by localoptions in the way
> options are?

That sounds like a good idea to me.