Another patch to compctl-examples CVS compctl, and a bug?

Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

It's now possible to avoid several external processes in generating the
list of CVS-controlled files in the `cvstargets' function.  The key is
this wonderful "parameter" (boy, is that misnamed) subsitution:

	"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"

This almost bears explaining in and of itself.  The {...} are needed
around the nullcmd input redirection, so that the 2> redirection won't
be improperly parsed as part of the file name (see my previous report).
The $(...) substitution grabs the contents of the CVS/Entries file to
get a list of all files in the current directory that CVS knows about.
This is wrapped in ${(f@)...} to force each line to be interpreted as
a separate word; this works because of the outer double quotes.  Next
is ${...#/} to strip a leading slash from every line of the CVS/Entries
file.  Finally, this is wrapped in ${...%%/*} to strip from each line
everything following the file name itself.

If there's a better way than `for f in ... do echo ... done' to prefix
every element of the resulting array with ${pref}, I'd love to see it.

Now for the bug:  If the ${pref}CVS/Entries file exists, all of this
works fine.  However, if no CVS directory exists at all, the above
subsitution prints a newline to the terminal -- which messes up zle.
The newline is not going into $reply, because using <redisplay> or
<clear-screen> and then continuing a menu completion cycles through
the other choices without further zle confusion.

Where is that newline coming from?

The other interesting (and the slow) part of $reply is:

	${pref}*/**/CVS(:h)

which returns names of any directories below the ${pref} directory that
in turn contain a CVS subdirectory.

Hmm, maybe all that should get added to the comments in the file. :-}

*** Misc/compctl-examples.0	Fri Jul  5 10:57:42 1996
--- Misc/compctl-examples	Sat Jul 13 16:50:40 1996
***************
*** 439,447 ****
  	pref=
      fi
      [[ -n "$pref" && "$pref" != */ ]] && pref=$pref/
!     reply=($(for f in $(cat ${pref}CVS/Entries 2>/dev/null | \
!     		sed 's/^\/\([^\/]*\).*/\1/'); do echo $pref$f; done)
! 	   $(echo ${pref}**/CVS(:h) | sed 's/CVS//'))
  }
  #------------------------------------------------------------------------------
  # RedHat Linux rpm utility
--- 439,448 ----
  	pref=
      fi
      [[ -n "$pref" && "$pref" != */ ]] && pref=$pref/
!     reply=($(for f in \
! 		"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}" \
!     		do echo $pref$f; done)
! 	   ${pref}*/**/CVS(:h))
  }
  #------------------------------------------------------------------------------
  # RedHat Linux rpm utility

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

On Jul 13,  5:34pm, Bart Schaefer wrote:
} Subject: Another patch to compctl-examples CVS compctl, and a bug?
}
} Now for the bug:  If the ${pref}CVS/Entries file exists, all of this
} works fine.  However, if no CVS directory exists at all, the above
} subsitution prints a newline to the terminal -- which messes up zle.
} The newline is not going into $reply, because using <redisplay> or
} <clear-screen> and then continuing a menu completion cycles through
} the other choices without further zle confusion.
} 
} Where is that newline coming from?
} 
} !     reply=($(for f in \
} ! 		"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}" \
} !     		do echo $pref$f; done)
} ! 	   ${pref}*/**/CVS(:h))

There's one too many backslashes in the above.  Take out the one just
before the "do".  With that backslash removed, everything works as
intended.  I still don't know why the extra newline is going straight
to the terminal, but this works:

    reply=($(for f in \
		"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"
		do echo $pref$f; done)
	   ${pref}*/**/CVS(:h))


-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

On Jul 13,  6:55pm, Bart Schaefer wrote:
} Subject: Re: Another patch to compctl-examples CVS compctl, and a bug?
}
} } !     reply=($(for f in \
} } ! 		"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}" \
} } !     		do echo $pref$f; done)
} } ! 	   ${pref}*/**/CVS(:h))
} 
} There's one too many backslashes in the above.  Take out the one just
} before the "do".  With that backslash removed, everything works as
} intended.

Sigh.  I take it back; it still generates the stray newline.  However, the
backslash does need to be removed for the completion to work properly in
directories that do have a CVS subdirectory.

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Zoltan Hidvegi]

> It's now possible to avoid several external processes in generating the
> list of CVS-controlled files in the `cvstargets' function.  The key is
> this wonderful "parameter" (boy, is that misnamed) subsitution:
> 
> 	"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"

There are two problems with that.  First I've modified the lexer so that
!, [[, { and } are reserved words now.  This means that these must be
delimited.  So {echo} no longer works, { echo } should be used instead.
Similarily [[-z $foo]] does not work, [[ -z $foo ]] should be used.

The other problem is that this depends on READNULLCMD being cat.
$(<...) always works since it is always recognized as a special command
substitution.  That's why the 2> redirection does not work in that case
(but a space between `(' and `<' fixes that problem).

> If there's a better way than `for f in ... do echo ... done' to prefix
> every element of the resulting array with ${pref}, I'd love to see it.

It's simply

 	"${pref}${^${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"

> works fine.  However, if no CVS directory exists at all, the above
> subsitution prints a newline to the terminal -- which messes up zle.

Are you sure that there is not empty element in the reply?  The above
`parameter expansion' always produces at least one element.

Zoltan

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

On Jul 15,  3:02am, Zoltan Hidvegi wrote:
} Subject: Re: Another patch to compctl-examples CVS compctl, and a bug?
}
} > 	"${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"
} 
} There are two problems with that.  First I've modified the lexer so that
} !, [[, { and } are reserved words now.  This means that these must be
} delimited.  So {echo} no longer works, { echo } should be used instead.
} Similarily [[-z $foo]] does not work, [[ -z $foo ]] should be used.

Hmm.  I don't think anything else I've done depends on that, but given
that we're already at 3.0-preN for N >1, it seems a rather bad time to
make the change.

} The other problem is that this depends on READNULLCMD being cat.

It depends on READNULLCMD being something that knows when its output
is not a terminal, at least; "less" and "more" both work just as well
as "cat".

} $(<...) always works since it is always recognized as a special command
} substitution.  That's why the 2> redirection does not work in that case

Still seems like a bug to me.

} > If there's a better way than `for f in ... do echo ... done' to prefix
} > every element of the resulting array with ${pref}, I'd love to see it.
} 
} It's simply
} 
}  	"${pref}${^${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"

I thought something like that should work, but I never tried the carat
in that particular spot.

} > works fine.  However, if no CVS directory exists at all, the above
} > subsitution prints a newline to the terminal -- which messes up zle.
} 
} Are you sure that there is not empty element in the reply?

Why should that make a difference?  It still shouldn't output a newline.

} The above `parameter expansion' always produces at least one element.

Here's the behavior I see in a directory that has no CVS subdirectory:

If I replace my `for ... echo' loop with your ${^...} solution above,
then *every* time I hit TAB I get a newline on the terminal, and I
never see any completions.  (Your solution works normally in a dir
that *does* have a CVS subdir.)

If I use my `for ... echo' loop, I get a newline the first time I hit
TAB.  The second time I hit TAB I get the leading hyphen from the -k
array of option letters, followed by a newline, followed by `A' (the
first option letter).  From that point on I get no stray newlines, and
the options cycle normally (though zle is confused if I don't do a
<redisplay> at some point.

If I use your solution with an explicit "cat", I don't get any newlines,
but I also don't get any completions -- not even the -k ones.

If I switch back to an explicit "cat" and use my original `for' loop,
everything works exactly as I would expect -- no stray newlines at all,
and all the expected -k completions appear.

So it has something to do with READNULLCMD, and there's something else
broken about your solution.  (Maybe that's where the empty element in
the array is involved?)

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Zoltan Hidvegi]

> Hmm.  I don't think anything else I've done depends on that, but given
> that we're already at 3.0-preN for N >1, it seems a rather bad time to
> make the change.

But I think that such a change would be even worse after 3.0 is out.

Zoltan

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

Here's a patch, to be applied on top of my last patch, that incorporates
some of the changes suggested by the discussion.  It avoids using $(<...)
construct on nonexistent files because of the stray newline that appears.
I'd still like to know where that's coming from.

*** Misc/compctl-examples.1	Sat Jul 13 16:50:40 1996
--- Misc/compctl-examples	Sun Jul 14 23:57:31 1996
***************
*** 439,447 ****
  	pref=
      fi
      [[ -n "$pref" && "$pref" != */ ]] && pref=$pref/
!     reply=($(for f in "${${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}" \
!     		do echo $pref$f; done)
! 	   ${pref}*/**/CVS(:h))
  }
  #------------------------------------------------------------------------------
  # RedHat Linux rpm utility
--- 439,451 ----
  	pref=
      fi
      [[ -n "$pref" && "$pref" != */ ]] && pref=$pref/
!     if [[ -f ${pref}CVS/Entries ]]
!     then
! 	reply=( "${pref}${^${${(f@)$(<${pref}CVS/Entries)}#/}%%/*}"
! 		${pref}*/**/CVS(:h) )
!     else
! 	reply=( ${pref}*/**/CVS(:h) )
!     fi
  }
  #------------------------------------------------------------------------------
  # RedHat Linux rpm utility

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

On Jul 15, 12:03am, Bart Schaefer wrote:
} Subject: Re: Another patch to compctl-examples CVS compctl, and a bug?
}
} Here's a patch, to be applied on top of my last patch, that incorporates
} some of the changes suggested by the discussion.  It avoids using $(<...)
} construct on nonexistent files because of the stray newline that appears.

Seems Zoltan and I had similar thoughts on this ... but you don't need the
2>/dev/null redirection if you're testing ahead of time for the existence
of the CVS/Entries file.  Here's my same patch except against 3.0-pre3.

*** Misc/compctl-examples.0	Mon Jul 15 00:07:16 1996
--- Misc/compctl-examples	Mon Jul 15 09:57:31 1996
***************
*** 439,449 ****
  	pref=
      fi
      [[ -n "$pref" && "$pref" != */ ]] && pref=$pref/
!     if [[ -f "${pref}CVS/Entries" ]] then
!     reply=( "${pref}${^${${(f@)$(cat ${pref}CVS/Entries 2> /dev/null)}#/}%%/*}"
! 	    ${pref}*/**/CVS(:h) )
      else
!     reply=( ${pref}*/**/CVS(:h) )
      fi
  }
  #------------------------------------------------------------------------------
--- 439,450 ----
  	pref=
      fi
      [[ -n "$pref" && "$pref" != */ ]] && pref=$pref/
!     if [[ -f ${pref}CVS/Entries ]]
!     then
! 	reply=( "${pref}${^${${(f@)$(<${pref}CVS/Entries)}#/}%%/*}"
! 		${pref}*/**/CVS(:h) )
      else
! 	reply=( ${pref}*/**/CVS(:h) )
      fi
  }
  #------------------------------------------------------------------------------

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Completion behavior change in 3.0-pre3

[Bart Schaefer]

Here's zsh 3.0-pre2:

zagzig[85] echo z*i<TAB>
(feep)
zagzig[85] echo z*i

Here's zsh 3.0-pre3:

zagzig<3> echo z*i<TAB>
(feep)
zagzig<3> echo z\*i

I really prefer the pre2 behavior.  I suspect this has something to do
with Zoltan's simplification of the code in docomplete(), but I'm not
sure what.  Probably replacement of the loop that walks chline with a
single call to inststr(chline)?

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Completion behavior change in 3.0-pre3

[Christoph von Stuckrad]

On Mon, 15 Jul 1996, Bart Schaefer wrote:

> Subject: Completion behavior change in 3.0-pre3 
--pre2--example deleted--
> Here's zsh 3.0-pre3:
> 
> zagzig<3> echo z*i<TAB>
> (feep)
> zagzig<3> echo z\*i

Isn't this a feature instead of a bug ?

THIS 'z\*i' is exactly what would be used on the input line if the
pattern doesn't glob (is it?, well, HERE it is, but i've set the option
'nonomatch' to get this behaviour).

So if this behaviour depends on 'nonomatch' I would appreciate it!
If not, well, I think one can get used to it.


By the way, I fear almoust all scripts I use break due to
the new '((' and '[[' 'blanks', but I'll have to try yet
I'm just compiling...

Fanatic zsh-user Stucki

Christoph von Stuckrad       *  *  | talk to  | <stucki@math.fu-berlin.de> \
Freie Universitaet Berlin    |/__* | nickname | ...!unido!fub!leibniz!stucki|
Fachbereich Mathematik, EDV  |\  * | 'stucki' | Tel:+49 30 838-7545{9|8}    |
Arnimallee 2-6/14195 Berlin  * *   |  on IRC  | Fax:+49 30 838-5913        /

Re: Completion behavior change in 3.0-pre3

[Bart Schaefer]

On Jul 17, 11:52am, Christoph von Stuckrad wrote:
> Subject: Re: Completion behavior change in 3.0-pre3
> On Mon, 15 Jul 1996, Bart Schaefer wrote:
> 
> > Here's zsh 3.0-pre3:
> > 
> > zagzig<3> echo z*i<TAB>
> > (feep)
> > zagzig<3> echo z\*i
> 
> Isn't this a feature instead of a bug ?

Given that Zoltan didn't mention this as an intentional change, I don't
think it's meant to be a feature.

> THIS 'z\*i' is exactly what would be used on the input line if the
> pattern doesn't glob

I *might* agree if I had GLOB_COMPLETE set; but I don't, so it
should be waiting for me to make the pattern unambiguous before it
decides that it has to quote anything.  Even with GLOB_COMPLETE,
I can't really go along with you; performing completion/expansion
shouldn't change the *semantics* of the command line:

	touch z*i	# Fails with "no match"
	touch z\*i	# Creates a new file

In effect, zsh should never rewrite the line when it (feep)s, only
when it successfully expands or completes something.

> So if this behaviour depends on 'nonomatch' I would appreciate it!

I don't have nonomatch set either.  (Gad, that sounds illiterate.
A triple negative?  Anyway, it doesn't depend on nonomatch.)

> If not, well, I think one can get used to it.

Get used to backing up to remove quoting from every glob pattern on the
line after every completion?  Or get used to not using completion?

Re: Completion behavior change in 3.0-pre3

[Bart Schaefer]

On Jul 15, 11:00pm, Bart Schaefer wrote:
> Subject: Completion behavior change in 3.0-pre3
> Here's zsh 3.0-pre2:
> 
> zagzig[85] echo z*i<TAB>
> (feep)
> zagzig[85] echo z*i
> 
> Here's zsh 3.0-pre3:
> 
> zagzig<3> echo z*i<TAB>
> (feep)
> zagzig<3> echo z\*i

I've traced the point where the extra backslash gets inserted into
makecomplist(), right here:

    /* ... */
    lpre = zalloc(lpl + 1);
    memcpy(lpre, s, lpl);
    lpre[lpl] = '\0';
    p = quotename(lpre, NULL, NULL, NULL);
    if (strcmp(p, lpre) && !strpfx(p, qword)) {
	/* ... */
	inststrlen(p, 1, l2 = strlen(p));	/* This adds the backslash */
	/* ... */
    }
    /* ... */

In 3.0-pre2 and earlier, the backslash would get removed again as shown
by the following patch (which backs out two hunks of Zoltan's changes).
I'm not sure why Zoltan thought the line should only be restored when
going around the xorrec: loop.  Should cs et.al. get restored along with
the line when *not* going around the loop?  (They weren't in pre2, this
just puts back the pre2 behavior.)

*** Src/zle_tricky.c.0	Mon Jul 15 22:46:49 1996
--- Src/zle_tricky.c	Wed Jul 17 14:45:34 1996
***************
*** 2957,2962 ****
--- 2957,2964 ----
      validlist = 1;
      if(nmatches && !errflag)
  	return 0;
+     /* No matches: restore the command line. */
+     strcpy((char *)line, (char *)ol);
  
      if ((isf || cc->xor) && !parampre) {
  	/* We found no matches, but there is a xor'ed completion: *
***************
*** 2967,2973 ****
  	wb = owb;
  	we = owe;
  	cs = ocs;
- 	strcpy((char *)line, (char *)ol);
  	offs = oloffs;
  	s = dupstring(os);
  	free(amatches);
--- 2969,2974 ----

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Bart Schaefer]

On Jul 14,  7:32pm, Bart Schaefer wrote:
>
> On Jul 15,  3:02am, Zoltan Hidvegi wrote:
> } 
> }  	"${pref}${^${${(f@)$({<${pref}CVS/Entries} 2>/dev/null)}#/}%%/*}"
> 
> Here's the behavior I see in a directory that has no CVS subdirectory:
> 
> If I replace my `for ... echo' loop with your ${^...} solution above,
> then *every* time I hit TAB I get a newline on the terminal, and I
> never see any completions.  (Your solution works normally in a dir
> that *does* have a CVS subdir.)

I think I finally tracked this down, crashing my Linux 1.3.15 kernel
three times in the process.  (Use GDB to put a breakpoint on a system
call such as write(), and then watch what happens when the process
forks and the child uses the system call.)

When you write this form of READNULLCMD:

	$( { <nonexistentfile } 2>/dev/null )

Zsh forks to execute the { ... } part so that it can redirect stderr.
READNULLCMD in the child process then discovers that nonexistentfile
doesn't exist, and calls zerr() to output the "no such file" message.

This is all fine most of the time, but when the above is used in a
completion, zleactive==1 and so zerr() calls trashzle().  The child
thus messes up the parent's terminal.

However, I don't know where to put a `zleactive = 0;' to prevent this.
Does it go in getoutput()?  That would solve this particular case, but
there might be others.  Does it go in entersubsh()?  Or is entersubsh()
sometimes called when we haven't really forked?

Insights appreciated.

Re: Bug in case stmt with '('

[Bart Schaefer]

On Jul 21, 11:31pm, Bart Schaefer wrote:
} Subject: Re: Bug in case stmt with '('
}
} I think we should leave it POSIX-style.
} 
} However, there is a remaining bug in Zoltan's patch.
} 
} zagzig<8> case foo in
} > (foo) echo yes;;
} zsh: parse error near `echo'
} 
} Apparently there still needs to be at least one token between the close
} paren and the first word of the actual command.

I thought Zoltan's "if" cascade looked a little too easy.  Here's the fix.

*** Src/parse.c.0	Fri Jul 19 11:17:12 1996
--- Src/parse.c	Sun Jul 21 23:45:10 1996
***************
*** 522,544 ****
  		str2[sl] = Bar;
  		str2[sl+1] = '\0';
  		str = str2;
! 	    } else if (tok == STRING) {
! 		char *str2;
  		int sl = strlen(str);
  
! 		if (str[sl - 1] != Bar)
! 		    YYERRORV;
! 		str2 = ncalloc(sl + strlen(tokstr) + 1);
! 		strcpy(str2, str);
! 		strcpy(str2 + sl, tokstr);
! 		str = str2;
! 	    } else {
! 		/* POSIX allows (foo*) patterns */
! 		char *s = str;
  
! 		if (skipparens(Inpar, Outpar, &s) || *s)
! 		    YYERRORV;
! 		break;
  	    }
  	}
  	addlinknode(pats, str);
--- 522,547 ----
  		str2[sl] = Bar;
  		str2[sl+1] = '\0';
  		str = str2;
! 	    } else {
  		int sl = strlen(str);
  
! 		if (str[sl - 1] != Bar) {
! 		    /* POSIX allows (foo*) patterns */
! 		    char *s = str;
  
! 		    if (skipparens(Inpar, Outpar, &s) || *s)
! 			YYERRORV;
! 		    break;
! 		} else {
! 		    char *str2;
! 
! 		    if (tok != STRING)
! 			YYERRORV;
! 		    str2 = ncalloc(sl + strlen(tokstr) + 1);
! 		    strcpy(str2, str);
! 		    strcpy(str2 + sl, tokstr);
! 		    str = str2;
! 		}
  	    }
  	}
  	addlinknode(pats, str);
***************

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Another patch to compctl-examples CVS compctl, and a bug?

[Peter Stephenson]

schaefer@candle.brasslantern.com wrote:
> However, I don't know where to put a `zleactive = 0;' to prevent this.
> Does it go in getoutput()?  That would solve this particular case, but
> there might be others.  Does it go in entersubsh()?  Or is entersubsh()
> sometimes called when we haven't really forked?
> 
> Insights appreciated.

I'm not sure if I'd go as far as `insight', but putting zleactive to
zero in entersubsh() sounds very plausible to me.  Entersubsh() is
sometimes called when we haven't really forked, but the intention is
always that the shell is about to exec a command in the manner it
would if it actually had forked.  In fact, turning off the interactive
capabilities is one of its major raisons d'etre, so this looks just
the place.

It's not clear to me, by the way, whether the subset of options that
gets turned off in entersubsh() --- currently MONITOR and USEZLE (aha!
that proves it --- if USEZLE is turned off, zleactive should certainly
go to zero) is correct.  If MONITOR, why not INTERACTIVE itself?

% for opt in monitor interactive; do
for> print $([[ -o $opt ]] && print $opt is set)
for> done

  interactive is set

*** Src/exec.c.bart	Mon Jul 22 09:27:52 1996
--- Src/exec.c	Mon Jul 22 09:28:14 1996
***************
*** 1857,1862 ****
--- 1857,1863 ----
      if (sigtrapped[SIGQUIT] != 2)
  	signal_default(SIGQUIT);
      opts[MONITOR] = opts[USEZLE] = 0;
+     zleactive = 0;
      if (cl)
  	clearjobtab();
      times(&shtms);


-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77330
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: Bug in case stmt with '('

[Zoltan Hidvegi]

> I confess to not understanding what it even *means* to "[disable] the
> special meaning of `(', `|', `)' and `<' for globbing the result of
> parameter and command substitutions."  And what *are* the "some other
> places where the shell accepts patterns" where this has effect?

In fact it affects every places where patterns can be used except explicit
glob patterns on the command line and case patterns.  This means -m builtin
options, completion glob patterns, ${...%...} substitutions, $foo[(r)...]
indexing etc.

It does not affect explicit glob patterns because in that case these
patterns result in syntax error in sh.  Also it would make the lexer a bit
more complicated since more special test sould be added and case patterns
should be treated specially.

Zoltan

Re: Bug in case stmt with '('

[Bart Schaefer]

On Jul 24,  9:29am, Zefram wrote:
} Subject: Re: Bug in case stmt with '('
}
} >I can't find any case where zsh doesn't permit '(' to be a glob metachar.
} >SH_GLOB relates to parameter and command substitutions, not filenames.
} 
} Which is a failing, in my opinion.  The glob syntax should be as
} consistent as possible (we seem to *need* some special handling in
} `case`).  SH_GLOB having an effect only in certain places seems very
} silly, and is rather confusing.  What's the rationale for it?

I confess to not understanding what it even *means* to "[disable] the
special meaning of `(', `|', `)' and `<' for globbing the result of
parameter and command substitutions."  And what *are* the "some other
places where the shell accepts patterns" where this has effect?

One of them seems to be any command that takes a -m option as with
`typeset -m x\*' -- which seems bizarre to me; another is parameter
assignments with globsubst set.  Others?

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Bug in case stmt with '('

[Peter Stephenson]

schaefer@candle.brasslantern.com wrote:
> Given the choice of:
> 
> 1.  The "current" behavior, i.e. spaces ARE significant only when the
>     optional leading paren is there;
> 2.  The POSIX behavior, i.e. spaces are not significant in case patterns;
> 3.  Spaces are never significant in glob patterns anywhere.
> 
> I'll take (2) before I'll take (3).  My solution is equivalent to (2),
> because it makes the spaces insignificant whether or not the opening
> paren is there.

This seems to me the best solution.  It seems to create the minimal
disruption to existing zsh users and to would-be POSIX users.
Changing glob semantics more widely worries me --- even if done
logically.  Shells just aren't logical anyway.

> I can't find any case where zsh doesn't permit '(' to be a glob metachar.

Actually, there's one place we had to handle syntactically specially:

[[ ($foo = (bar|rab)) ]]

The first and matching `(' do grouping, then following the `=' zsh has
to be told to expect glob patterns.  I've never seen a problem with
this, though, and the upshot is that `(' does pattern matching when
you want it to.


On something which is related, I've never been quiet happy with this:

[ ( foo = bar ) ]

(note no globbing metacharacters inside the parentheses).  As test is
an ordinary builtin, it simply gets called with the single argument `(
foo = bar )' --- the parentheses are not stripped, which has non-zero
length, so the test is true.  In fact, parentheses here are
effectively not treated as glob metacharacters.

I felt rather guilty about this: some time ago I allowed spaces inside
parentheses because I couldn't think of a good reason not to, and
because it seemed to work better with glob modifiers --- things like
*(:s/foo/bar/) --- which appeared in the same patch, rather than from
any fundamental syntactical reason.  Before, the shell would have
reported a syntax error on the above test and the user would have
known to quote the parentheses.  This is what ksh88 does.  Perhaps it
should be subject to nomatch testing (which it isn't at the moment),
perhaps to stripping of the parentheses, too.

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77330
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: Bug in case stmt with '('

[Zefram]

>I can't find any case where zsh doesn't permit '(' to be a glob metachar.
>SH_GLOB relates to parameter and command substitutions, not filenames.

Which is a failing, in my opinion.  The glob syntax should be as
consistent as possible (we seem to *need* some special handling in
`case`).  SH_GLOB having an effect only in certain places seems very
silly, and is rather confusing.  What's the rationale for it?

-zefram

Re: Bug in case stmt with '('

[Bart Schaefer]

On Jul 23,  4:01pm, Morris M. Siegel wrote:
} Subject: Re: Bug in case stmt with '('
}
} > } The best would be to handle it in glob.c but here it is a problem how can
} > } we distinguish ( foo\ | bar ) from ( foo | bar ).
} >
} > The `if (incasepat && ...)' patch that I sent handles this correctly.
} > The only drawback I've found so far to my patch is that if you *don't*
} > use the POSIX balanced-parens syntax for "case", then parenthesized
} > patterns can't have *meaningful* spaces in them.
} 
} Do you really think it user-friendly for the semantics of the case-prefix
} to depend on whether the optional leading '(' is present or not?

Given the choice of:

1.  The "current" behavior, i.e. spaces ARE significant only when the
    optional leading paren is there;
2.  The POSIX behavior, i.e. spaces are not significant in case patterns;
3.  Spaces are never significant in glob patterns anywhere.

I'll take (2) before I'll take (3).  My solution is equivalent to (2),
because it makes the spaces insignificant whether or not the opening
paren is there.

The "drawback" of my solution is that it makes PARENTHESES significant
only when the leading paren is there.  I don't have a problem with that
because other shells don't permit parentheses [other than the half-
optional outermost set] at all; thus the optional leading paren is
simply also introducing *additional* optional syntax.

} It seems to me, and I alluded to this in a previous posting, that it's
} most elegant for a whole case-prefix such as
} 	( a | b | c )
} to be in fact treated simply as a single zsh glob pattern (again, provided
} that zsh options allow '(' to be a glob metacharacter as it usually is).

I can't find any case where zsh doesn't permit '(' to be a glob metachar.
SH_GLOB relates to parameter and command substitutions, not filenames.
I have no objection to treating the whole case prefix as a single pattern;
as I've said, this is how it already works.

} That's why it's useful to ignore trivial blanks after '(', around '|',
} before ')', and maybe elsewhere, as Zoltan writes.

Except that blanks inside parenthesized glob patterns ARE NEVER TRIVIAL,
by the CURRENT definition of parenthesized glob patters.  I object to
changing this behavior.

} (Actually, it might
} perhaps be best for all blanks inside parentheses [unless quoted or
} backslashed] to be ignored unless their omission would cause distinct
} tokens to be erroneously fused.  E.g., '( a b | ^ d .c )' would be
} equivalent to '(a b|^d .c)'.  [...])

This is a big reason why I don't like the idea of stripping the spaces
everywhere.  Do YOU really think it user-friendly to have a set of
magic tokens around which spaces are or are not stripped?

} Is there some document containing the formal grammar of zsh?

Only comments in parse.c, unfortunately, and they aren't complete.

} On the contrary, zsh is improved by having glob patterns have
} a uniform meaning throughout the grammar.

I agree with the sentiment, but not with the realization.  IMHO it is
the insignificant spaces in case patterns that should be disallowed,
not spaces in glob patterns that should become insignificant.  POSIX
forbids us from making the better change; I don't believe we should
compound the error by making a worse one.  Two wrongs don't make a
right, and being consistently wrong is not better than being right
whenever possible.

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Bug in case stmt with '('

[Morris M. Siegel]

On Jul 23,  9:25am, Bart Schaefer wrote:
> Subject: Re: Bug in case stmt with '('
> On Jul 23,  4:08pm, Zoltan Hidvegi wrote:
> } Subject: Re: Bug in case stmt with '('
> }
> } The best would be to handle it in glob.c but here it is a problem how can
> } we distinguish ( foo\ | bar ) from ( foo | bar ).
>
> The `if (incasepat && ...)' patch that I sent handles this correctly.
> The only drawback I've found so far to my patch is that if you *don't*
> use the POSIX balanced-parens syntax for "case", then parenthesized
> patterns can't have *meaningful* spaces in them.

Do you really think it user-friendly for the semantics of the case-prefix
to depend on whether the optional leading '(' is present or not?

> I don't think this is a significant difficulty, since "case" already
> has the `|' syntax so there's not much reason to use glob-grouping
> parens in "case" patterns in the first place.

It seems to me, and I alluded to this in a previous posting, that it's
most elegant for a whole case-prefix such as
	( a | b | c )
to be in fact treated simply as a single zsh glob pattern (again, provided
that zsh options allow '(' to be a glob metacharacter as it usually is).
That's why it's useful to ignore trivial blanks after '(', around '|',
before ')', and maybe elsewhere, as Zoltan writes.  (Actually, it might
perhaps be best for all blanks inside parentheses [unless quoted or
backslashed] to be ignored unless their omission would cause distinct
tokens to be erroneously fused.  E.g., '( a b | ^ d .c )' would be
equivalent to '(a b|^d .c)'.  However, '~' might pose problems, since it
serves both as a directory symbol when unary and as a pattern complementation
operator when binary.  At any rate, the need for compatibility with standard
'case' syntax is satisfied if blanks are ignored minimally, as Zoltan
suggests.)

In order for me to participate more intelligently in this discussion,
it would be helpful for me to know where STRINGs occur in the zsh grammar.
Where do they in fact occur besides in glob patterns?
Is there some document containing the formal grammar of zsh?

> } The simplest solution is to convert every unquoted space
> } and TAB which is inside a globbing paren to a null-space and null-tab token
> } and later in glob.c a null-space or null-tab is either treated as space/tab
> } or discarded if it is adjacent to | or comes after a `(' of before a `)'
> } (or it may be better to disard these after `)' and before `(' as well).
> }
> } This seems to be a quite simple solution.
>
> Maybe, but not simpler than mine, and it's solving a problem that I don't
> think we ought to be solving -- we don't need to be able to ignore spaces
> in generalized glob patterns, only in "case" statements.  I don't like the
> suggestion to do it everywhere.

On the contrary, zsh is improved by having glob patterns have
a uniform meaning throughout the grammar.

-- Morrie Siegel

Re: Bug in case stmt with '('

[Bart Schaefer]

On Jul 23,  4:08pm, Zoltan Hidvegi wrote:
} Subject: Re: Bug in case stmt with '('
}
} The best would be to handle it in glob.c but here it is a problem how can
} we distinguish ( foo\ | bar ) from ( foo | bar ).

The `if (incasepat && ...)' patch that I sent handles this correctly.
The only drawback I've found so far to my patch is that if you *don't*
use the POSIX balanced-parens syntax for "case", then parenthesized
patterns can't have *meaningful* spaces in them.

I don't think this is a significant difficulty, since "case" already
has the `|' syntax so there's not much reason to use glob-grouping
parens in "case" patterns in the first place.

} The simplest solution is to convert every unquoted space
} and TAB which is inside a globbing paren to a null-space and null-tab token
} and later in glob.c a null-space or null-tab is either treated as space/tab
} or discarded if it is adjacent to | or comes after a `(' of before a `)'
} (or it may be better to disard these after `)' and before `(' as well).
} 
} This seems to be a quite simple solution.

Maybe, but not simpler than mine, and it's solving a problem that I don't
think we ought to be solving -- we don't need to be able to ignore spaces
in generalized glob patterns, only in "case" statements.  I don't like the
suggestion to do it everywhere.


-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

New male in /home/schaefer:
>N  2 Justin William Schaefer  Sat May 11 03:43  53/4040  "Happy Birthday"

Re: Bug in case stmt with '('

[Zoltan Hidvegi]

> >Unfortunately there is still an incompatibility in case:
> >
> >case foo in
> >( f* | b* ) echo yes
> >esac
> >
> >should print yes but in zsh it works as
> >
> >case foo in
> >\ f*\ |\ b*\ ) echo yes;;
> >esac
> 
> I was wondering whether that would be the case.  It's a serious problem.
> 
> >To fix this would be really difficult I think.
> 
> It could be easily fixed by modifying glob semantics such that unquoted
> whitespace embedded in a pattern is ignored.  I doubt that any real
> code relies on the current behaviour, which is in any case
> undocumented.

It is used in ${...%...} substitutions where a space stands for itself.
And it is not even a zsh feature.  So unquoted whitespace should not always
be ignored.  Ignoring unquoted whitespace before | and ) and after ( and |
seems to be a better solution but it is more difficult to implement.
Consistency is desirable so patterns should behave similarily in case
statements, ${...%...} substitutions (double quoted or not), in argumenents
to builtins after -m etc.

At a few places the code assumes that the lexer just tokenizes the input
but does not actually modfy it so omitting these spaces in the lexer is not
the best solution.

The best would be to handle it in glob.c but here it is a problem how can
we distinguish ( foo\ | bar ) from ( foo | bar ).  It would require a new
token for a null-space (the word `token' is amiguous in zsh, it can be
either a token returned by gettok or a character token for a character in
ztokens, here I use the second meaning).  A token for null-TAB may also be
necessary here.  The simplest solution is to convert every unquoted space
and TAB which is inside a globbing paren to a null-space and null-tab token
and later in glob.c a null-space or null-tab is either treated as space/tab
or discarded if it is adjacent to | or comes after a `(' of before a `)'
(or it may be better to disard these after `)' and before `(' as well).

This seems to be a quite simple solution.

On the other topic of empty patterns: it is true that POSIX does not allow
that.  Even previous zsh versions had problems with empty patterns in case
statement so it probably does not cause any problems if we do not allow
that in case statement.  That would simplyfy parsing since it means that
after a BAR a STRING token must come in a case pattern.  But in other
places empty patterns are usefull.  I ofter use (...|) for an optional
match and this should not be disallowed.

Zoltan

PS.  I'm now moving this quite technical discussion to zsh-workers.