Enhanced shell

Enhanced shell

[Peter Stephenson]

David Korn is asking for suggestions for features from the various
shells to be included in a standard, so I've been jotting possible
things from zsh down.  This is extremely rough at the moment, but before
I try and tidy it up, has anyone got any comments or suggestions
(preferably brief --- I'm looking for a list of features rather than an
essay on compatilibity)?

Some of these may already be in all three of zsh, ksh, and bash, I
haven't gone through the ksh manual yet (or even Single Unix for that
matter) --- some are certainly in both zsh and bash.  I'll try and do
some more homework tomorrow.

emulate builtin
coproc keyword (|& as per csh is already semi-standard and useful)
? ZDOTDIR? (or equivalent)
autoload -U
<<<'word'
precmd, chpwd, periodic
? [#16]?
${+foo}
{0..10}, {00..10}
? =cmd?
? glob qualifiers (perhaps in some standardised NO_BARE_GLOB_QUAL form)
** (both glob and math?)
CPUTYPE/HOSTYPE/OSTYPE, EGID/GID, EUID/UID,  ERRNO, HOST or HOSTNAME (bash)
`ESS_VERSION' (version no. of extended shell standard)
? disable/enable? [bash]
? typeset -T? (possible with discipline functions)
? typeset etc. -g?
command [bash]
suspend [bash]
? pushd?

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: Enhanced shell

[Oliver Kiddle]

Peter Stephenson wrote:
> 
> David Korn is asking for suggestions for features from the various
> shells to be included in a standard, so I've been jotting possible
> things from zsh down.  This is extremely rough at the moment, but before

I had also jotted down a list so I've merged it in with yours and added
a few comments.

> emulate builtin

whence builtin and variants
disown

typeset builtin (yes, it isn't in POSIX). Also, note that bash has
obsoleted `typeset' in favour of `declare' which I think is better.

> ? disable/enable? [bash]
not so useful. they are also common as commands to enable/disable
printers (on AIX and Solaris at least).

> coproc keyword (|& as per csh is already semi-standard and useful)
> ? ZDOTDIR? (or equivalent)
> autoload -U
and various other fpath/autoloading differences

> <<<'word'
multios option (and maybe other redirection extensions)

> precmd, chpwd, periodic
preexec
I'm not too sure about these because of the sort of problems we get
with, for example the prompt themes overwriting the existing functions.
I can think of a few more flexible ways of allowing the same.

> ? [#16]?
possible, not sure

> ${+foo}
yes and some other parameter expansion extensions

> {0..10}, {00..10}
and {a,b} brace expansion

> ? =cmd?
yes

> ? glob qualifiers (perhaps in some standardised NO_BARE_GLOB_QUAL form)
yup.
backreferences (ksh93's possibly being better)
? glob flags
? glob operators: x~y ^x x# x## <x-y>
> ** (both glob and math?)

> CPUTYPE/HOSTYPE/OSTYPE, EGID/GID, EUID/UID,  ERRNO, HOST or HOSTNAME (bash)
VENDOR/MACHTYPE, LOGNAME (USER/USERNAME), PWD, OLDPWD, COLUMNS, LINES,
pipestatus [PIPESTATUS in bash], ARGC/ARGV(?), RANDOM(?), SECONDS(?),
SHLVL, TTY

> `ESS_VERSION' (version no. of extended shell standard)
or possibly take that idea further to cover more details. I like
ksh93's use of a .sh namespace

> ? typeset -T? (possible with discipline functions)
that's a good point about the disciplines

> ? typeset etc. -g?
zsh/bash dynamically scoped local variables.

> command [bash]
-, builtin and noglob. I think command is already in the standard.

> suspend [bash
it's a default alias in ksh

> ? pushd?
not too sure about that because it can be done with a couple of scripts
and because autopushd with the ~ completion is more useful.

cd -
two argument form of cd
cd's handling of symbolic links
directory stack, and access with ~num

process substitution
C style for loop syntax
? repeat loop
prompt percent substitutions
[[ ... ]] conditions
? global aliases
some of the default key bindings (e.g tab for completion)
various history features (shared, duplicate removal etc, fc options, datestamps)
the common -L option to many builtins [similar to typeset -p in ksh]
-l option to the shell - I wish bash would accept -l in addition to --login
array subscripts with start and end
subscripts on a scalar variable

? select, let and function statements (all of which I'd leave out)
I'd also leave out nested substitutions and parameter expansion flags.

I've not gone through the list of options so some of them may be useful.
There may be other builtins too.

Oliver

Re: Enhanced shell

[Peter Stephenson]

Oliver Kiddle wrote:
> > David Korn is asking for suggestions for features from the various
> > shells to be included in a standard, so I've been jotting possible
> > things from zsh down.
> 
> I had also jotted down a list so I've merged it in with yours and added
> a few comments.
> 
> typeset builtin (yes, it isn't in POSIX). Also, note that bash has
> obsoleted `typeset' in favour of `declare' which I think is better.

It strikes me as a bit late for that now.

> > ? disable/enable? [bash]
> not so useful. they are also common as commands to enable/disable
> printers (on AIX and Solaris at least).

Yes, I can see it would be silly defining a standard which is known to have
this clash in it.

> multios option (and maybe other redirection extensions)

This is very useful, but it's maybe a bit too much of a hack to
standardise (e.g. the race condition on output multios in the current
implementation, although I hope that's fixable).

> > precmd, chpwd, periodic
> preexec
> I'm not too sure about these because of the sort of problems we get
> with, for example the prompt themes overwriting the existing functions.
> I can think of a few more flexible ways of allowing the same.

In which case, we should probably think about that for our own purposes
before attempting to standardise.

> backreferences (ksh93's possibly being better)

I haven't looked at those at all.

> > `ESS_VERSION' (version no. of extended shell standard)
> or possibly take that idea further to cover more details. I like
> ksh93's use of a .sh namespace

Since namespaces were on David Korn's list, that might be preferable.

> > ? typeset -T? (possible with discipline functions)
> that's a good point about the disciplines

> > ? typeset etc. -g?
> zsh/bash dynamically scoped local variables.

Hmm.  I don't know how this will pan out if ksh is going the way of
statically scoped parameters.

> > command [bash]
-, builtin and noglob. I think command is already in the standard.

I hadn't looked at the standard --- this was something I meant to check.
I'm pretty sure builtin is there hence the POSIX_BUILTINS option.  noglob
is really for saving typing interactively, so I'm not sure it's what we're
looking for at the moment.

> > suspend [bash
> it's a default alias in ksh

OK, may well be worth standardising

> cd -
> two argument form of cd
> directory stack, and access with ~num

Again, these are really interactive shortcuts; you can (and probably would)
do them in a more verbose way in a script.  Standardising handling of
symbolic links in cd might be sensible, though.

> process substitution

That's already in ksh, but it wasn't on David Korn's list, so we could
suggest it.

> C style for loop syntax

I would tend to avoid trying to standardise variant syntax altogether.

> prompt percent substitutions

Interactive shortcut again --- very useful, but less so for scripts.  Quite
a sizeable extension if you haven't already got it.  It's annoying bash
does this with backslashes.

> [[ ... ]] conditions

I suspect that's already implied.

> some of the default key bindings (e.g tab for completion)

Some of our key bindings have come from ksh, it might indeed be sensible to
keep track of which we want to list as standard.

> array subscripts with start and end
> subscripts on a scalar variable

Apparently this works a different way:  ${param:start:offset}, or
something, haven't looked at the details.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: Enhanced shell

[Nadav Har'El]

On Mon, Jul 23, 2001, Oliver Kiddle wrote about "Re: Enhanced shell":
> > ? disable/enable? [bash]
> not so useful. they are also common as commands to enable/disable
> printers (on AIX and Solaris at least).

These are useful for restricted-shell mode, when you sometimes want to
disable builtins (e.g., the "read" builtin allows the restricted user to
read files that they shouldn't).

-- 
Nadav Har'El                        |           Monday, Jul 23 2001, 3 Av 5761
nyh@math.technion.ac.il             |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |A good programmer is someone who looks
http://nadav.harel.org.il           |both ways before crossing a one-way street.

Enable/disable (was Re: Enhanced shell)

[Bart Schaefer]

On Jul 23,  3:00pm, Nadav Har'El wrote:
}
} On Mon, Jul 23, 2001, Oliver Kiddle wrote about "Re: Enhanced shell":
} > > ? disable/enable? [bash]
} > not so useful. they are also common as commands to enable/disable
} > printers (on AIX and Solaris at least).
} 
} These are useful for restricted-shell mode, when you sometimes want to
} disable builtins (e.g., the "read" builtin allows the restricted user to
} read files that they shouldn't).

As of 4.0.1, `builtin read' will run `read' even after `disable read'.
Maybe we need to rethink that.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Enhanced shell

[Bart Schaefer]

This is a bit choppy because I'm trying not to say myself anything that
Oliver already said.

On Jul 22, 11:47pm, Peter Stephenson wrote:
}
} ? ZDOTDIR? (or equivalent)

This is a slightly twitchy one.  It's only really useful if it's imported
from the environment, but it's dangerous for e.g. `su' to read other
people's dotfiles.  Maybe ZDOTDIR should be discarded unless ((EGID==GID)).
(Maybe that would also be a good test for whether to import FPATH, which
was another thread of discussion not long ago.)

} autoload -U

And `setopt NO_ALIASES' or the equivalent.

On Jul 23, 12:13pm, Oliver Kiddle wrote:
}
} > precmd, chpwd, periodic
} preexec
} I'm not too sure about these because of the sort of problems we get
} with, for example the prompt themes overwriting the existing functions.
} I can think of a few more flexible ways of allowing the same.

I think these should always have been arrays of function names to be
called, rather than magic function names in their own right.  Of course,
we could do both.

} > ${+foo}
} yes and some other parameter expansion extensions

I think where we'll run into problems here, aside from the fact that the
flags syntax used by zsh is a bit baroque, is with field splitting and
whether various extensions apply before or after it happens (I have the
impression that under traditional splitting rules, certain extensions
can't possibly be made to apply after splitting).  I see you've suggested
leaving out expansion flags, though.

} > ? glob qualifiers (perhaps in some standardised NO_BARE_GLOB_QUAL form)
} yup.
} backreferences (ksh93's possibly being better)

You didn't mention anything about that in workers/15348.  What are the
differences?

} > CPUTYPE/HOSTYPE/OSTYPE, EGID/GID, EUID/UID,  ERRNO, HOST or HOSTNAME (bash)
} VENDOR/MACHTYPE, LOGNAME (USER/USERNAME), PWD, OLDPWD, COLUMNS, LINES,
} pipestatus [PIPESTATUS in bash], ARGC/ARGV(?), RANDOM(?), SECONDS(?),
} SHLVL, TTY

ARGC/ARGV are going to be hard to standardize because of the 1-based v.
0-based array indexing issues.  In fact, I suspect array indexing is
going to be one of the bigger sticking points in the whole discussion.

} > command [bash]
} -, builtin and noglob. I think command is already in the standard.

Yes, `command' is in POSIX, but it works a bit differently than zsh's.

} > ? pushd?
} not too sure about that because it can be done with a couple of scripts
} and because autopushd with the ~ completion is more useful.

Bash has pushd/popd as builtins.  Also:

} directory stack, and access with ~num

You suggest having the directory stack as a special built-in but not the
commands to manipulate it?

} cd -
} two argument form of cd
} cd's handling of symbolic links

That last could stand some clarification even in current zsh.  It's been
on my to-do list for ages and I've never gotten to it.

The two-argument form of cd has stopped us doing some other interesting
things with options to the `cd' command.  It might not be the best form
to standardize.

} C style for loop syntax

You mean `for ((...))'?  I've always been mildly annoyed that it only
works with math expressions and not with more general shell commands.

} ? repeat loop

Yes.

} prompt percent substitutions

Maybe.

} [[ ... ]] conditions

Yes.

} ? global aliases

Not sure.

} the common -L option to many builtins [similar to typeset -p in ksh]

Option letter clashes are another place where we may have trouble.  I
think `-L' was a better choice than `-p', but ...

} array subscripts with start and end

More 0-vs-1 problems.

What about subscript flags and pattern subscripts?  array[(r)pat]
"Reverse indexing"?

On Jul 23, 12:38pm, Peter Stephenson wrote:
}
} > typeset builtin (yes, it isn't in POSIX). Also, note that bash has
} > obsoleted `typeset' in favour of `declare' which I think is better.
} 
} It strikes me as a bit late for that now.

But it could be added as another alias for typeset; ksh93 doesn't have
`local' either (does it?).

} > > ? disable/enable? [bash]
} > not so useful. they are also common as commands to enable/disable
} > printers (on AIX and Solaris at least).
} 
} Yes, I can see it would be silly defining a standard which is known to have
} this clash in it.

As has been pointed out, though, the functionality is useful.  Rather
than `disable -[afr] ...' maybe we could standardize on an option common
to other commands, e.g. `function -D name' would be `disable -f name',
`alias -D name' would be `disable -a name', etc.  The tricky one is
`builtin -D name' which would have to work on both reserved words and
builtin commands, or something.

} > multios option (and maybe other redirection extensions)
} 
} This is very useful, but it's maybe a bit too much of a hack to
} standardise (e.g. the race condition on output multios in the current
} implementation, although I hope that's fixable).

It's fixable; the shell just needs to keep track of the forked `tee'-job
and wait for it.  That just happens to be a little complicated right now.
 
} > > ? typeset etc. -g?
} > zsh/bash dynamically scoped local variables.
} 
} Hmm.  I don't know how this will pan out if ksh is going the way of
} statically scoped parameters.

Another typeset option?

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Enable/disable (was Re: Enhanced shell)

[Nadav Har'El]

On Mon, Jul 23, 2001, Bart Schaefer wrote about "Enable/disable (was Re: Enhanced shell)":
> } These are useful for restricted-shell mode, when you sometimes want to
> } disable builtins (e.g., the "read" builtin allows the restricted user to
> } read files that they shouldn't).
> 
> As of 4.0.1, `builtin read' will run `read' even after `disable read'.
> Maybe we need to rethink that.

Even if builtin's behavior remains, it's not so terrible, because somebody
who wants to disable a builtin like read will just need to also disable
"builtin", as well as (obviously) the "enable" command (or the restricted user
will be able to reenable the command).

But now that I think of it, does zsh have restricted-shell capabilities at
all? Zsh has been my favorite shell for the last 8 years or so, but when I
needed a restricted shell, I went with pdksh or bash 2. I don't know why...
Perhaps the words "Zsh" and "restricted" just didn't sound right together :)

-- 
Nadav Har'El                        |          Tuesday, Jul 24 2001, 4 Av 5761
nyh@math.technion.ac.il             |-----------------------------------------
Phone: +972-53-245868, ICQ 13349191 |I don't eat snails. I prefer fast food.
http://nadav.harel.org.il           |

Re: Enable/disable (was Re: Enhanced shell)

[Bart Schaefer]

On Jul 24,  1:00am, Nadav Har'El wrote:
> > 
> > As of 4.0.1, `builtin read' will run `read' even after `disable read'.
> > Maybe we need to rethink that.
> 
> Even if builtin's behavior remains, it's not so terrible, because somebody
> who wants to disable a builtin like read will just need to also disable
> "builtin"

That's unpleasant; it makes it impossible [*] to bypass a shell function
without unfunction-ing it.  It makes life more difficult for the person
setting up the restricted environment, as well as for the person using it.

It also potentially breaks the $functions parameter, in which the bodies
of undefined functions appear as `builtin autoload -X'.

> But now that I think of it, does zsh have restricted-shell capabilities at
> all?

It gained restricted shell mode somewhere fairly early in the 3.1 series,
so 4.0.x has it, but 3.0.x does not.

Try `setopt restricted'.  Just note that you can't unsetopt it again, so
you should be prepared to exit.

[*] OK, so it's not quite impossible:

function neither_alias_nor_function() {
  local +h aliases functions
  disable -a ${(k)aliases}
  disable -f ${(k)functions}
  "$@"
}
neither_alias_nor_function echo this is too clever

But of course being able to assign to the arrays from the $parameter module
also allows one to re-enable disabled commands, and one can always assign
to a variable by making it local, so it's impossible to disable enable
unless one disables zmodload, but then completion and lots of other things
don't work, and now you see what a hole we've dug ...

Here-strings and $functions

[Bart Schaefer]

On Jul 23,  6:12pm, Bart Schaefer wrote:
}
} function neither_alias_nor_function() {
}   local +h aliases functions
}   disable -a ${(k)aliases}
}   disable -f ${(k)functions}
}   "$@"
} }

This function revealed a problem with here-strings in the $functions
parameter:

schaefer[505] foo() {
function> <<<'This is a here-string'
function> }
schaefer[506] foo
This is a here-string
schaefer[507] functions[foo]=$functions[foo]
schaefer[508] foo
foo:1: command not found: is
schaefer[509] functions foo
foo () {
        is a here-string'' <<< '''This'
}

Uhh, yeah.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Here-strings and $functions

[Bart Schaefer]

On Jul 24,  2:45am, Bart Schaefer wrote:
}
} schaefer[505] foo() {
} function> <<<'This is a here-string'
} function> }
} schaefer[506] foo
} This is a here-string
} schaefer[507] functions[foo]=$functions[foo]
} schaefer[508] foo
} foo:1: command not found: is
} schaefer[509] functions foo
} foo () {
}         is a here-string'' <<< '''This'
} }
} 
} Uhh, yeah.

This bug is subtle, and I'm not sure how to fix it, and I can't spend
any more time on it tonight as I must get up very early tomorrow ...

The complication is that this:

foo() {
  <<-EOF
	This is a string
	EOF
}

gets converted internally into a REDIR_HERESTR, and so does the original
example with <<<.  But in the case of <<<, THE QUOTES ARE STILL PART OF
THE STRING, as \230 tokens.  For <<-, there are no quotes.  So when we
arrive in getredirs(), in the <<- case we need to insert quotes and then
call bslashquote(), but in the <<< case we only need to untokenize().

The only way to get this right is to know whether the original code was
really a here-string, or actually a full-fledged here-document; but
that information isn't passed down to getredirs() -- I'm not even sure
whether it's available in gettext2().

Otherwise we need to change the way <<< here-strings are parsed, so that
they're stored untokenized and with quotes stripped.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Here-strings and $functions

[Sven Wischnowsky]

Bart Schaefer wrote:

> ...
> 
> gets converted internally into a REDIR_HERESTR, and so does the original
> example with <<<.  But in the case of <<<, THE QUOTES ARE STILL PART OF
> THE STRING, as \230 tokens.  For <<-, there are no quotes.  So when we
> arrive in getredirs(), in the <<- case we need to insert quotes and then
> call bslashquote(), but in the <<< case we only need to untokenize().
> 
> The only way to get this right is to know whether the original code was
> really a here-string, or actually a full-fledged here-document; but
> that information isn't passed down to getredirs() -- I'm not even sure
> whether it's available in gettext2().
> 
> Otherwise we need to change the way <<< here-strings are parsed, so that
> they're stored untokenized and with quotes stripped.

Err... it would be enough to just remnulargs() for here strings,
wouldn't it?  At least it seems to work.  Just in case I missed
something I won't commit this just now (although I tried <<<, <<-, <<
and <<'EOF').


Bye
  Sven

Index: Src/parse.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/parse.c,v
retrieving revision 1.28
diff -u -r1.28 parse.c
--- Src/parse.c	2001/07/09 16:05:14	1.28
+++ Src/parse.c	2001/07/24 08:10:33
@@ -1685,6 +1685,9 @@
 	if ((tokstr[0] == Inang || tokstr[0] == Outang) && tokstr[1] == Inpar)
 	    type = tokstr[0] == Inang ? REDIR_INPIPE : REDIR_OUTPIPE;
 	break;
+    case REDIR_HERESTR:
+        remnulargs(name = dupstring(name));
+        break;
     }
     yylex();
 

-- 
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

Re: Enable/disable (was Re: Enhanced shell)

[Peter Stephenson]

"Nadav Har'El" wrote:
> On Mon, Jul 23, 2001, Bart Schaefer wrote about "Enable/disable (was Re: Enha
> nced shell)":
> > } These are useful for restricted-shell mode, when you sometimes want to
> > } disable builtins (e.g., the "read" builtin allows the restricted user to
> > } read files that they shouldn't).
> > 
> > As of 4.0.1, `builtin read' will run `read' even after `disable read'.
> > Maybe we need to rethink that.

I missed seeing the original of this.  Are you sure?  It doesn't in 4.1.0,
and I doubt whether anybody's added it to 4.0 retrospectively.  There was a
discussion about this, but since it wasn't entirely conclusive nothing has
changed.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

PATCH: Doc errors (Re: Enable/disable (was Re: Enhanced shell))

[Bart Schaefer]

On Jul 24, 10:23am, Peter Stephenson wrote:
} Subject: Re: Enable/disable (was Re: Enhanced shell)
}
} "Nadav Har'El" wrote:
} > On Mon, Jul 23, 2001, Bart Schaefer wrote about "Enable/disable (was Re: Enha
} > nced shell)":
} > > } These are useful for restricted-shell mode, when you sometimes want to
} > > } disable builtins (e.g., the "read" builtin allows the restricted user to
} > > } read files that they shouldn't).
} > > 
} > > As of 4.0.1, `builtin read' will run `read' even after `disable read'.
} > > Maybe we need to rethink that.
} 
} I missed seeing the original of this.  Are you sure?  It doesn't in 4.1.0

Urk.  I was sure I saw that patch committed.

The appended fixes one other typo introduced by my last doc commit.

diff -x CVS -ru zsh-forge/current/Doc/Zsh/builtins.yo zsh-4.0/Doc/Zsh/builtins.yo
--- zsh-forge/current/Doc/Zsh/builtins.yo	Tue Jul 10 01:52:02 2001
+++ zsh-4.0/Doc/Zsh/builtins.yo	Tue Jul 24 05:05:33 2001
@@ -130,8 +130,7 @@
 )
 findex(builtin)
 item(tt(builtin) var(name) [ var(args) ... ])(
-Executes the builtin var(name), with the given var(args), even if that
-builtin has been disabled (see tt(disable)).
+Executes the builtin var(name), with the given var(args).
 )
 alias(bye)(exit)
 module(cap)(zsh/cap)
diff -x CVS -ru zsh-forge/current/Doc/Zsh/grammar.yo zsh-4.0/Doc/Zsh/grammar.yo
--- zsh-forge/current/Doc/Zsh/grammar.yo	Tue Jul 10 01:52:02 2001
+++ zsh-4.0/Doc/Zsh/grammar.yo	Sat Jul 14 22:19:50 2001
@@ -398,7 +398,7 @@
 entirely quoted.  A literal `tt(')' character can be included in the
 string by using the `tt(\')' escape.
 
-@pindex(RC_QUOTES, use of)
+pindex(RC_QUOTES, use of)
 All characters enclosed between a pair of single quotes (tt('')) that
 is not preceded by a `tt($)' are quoted.  A single quote cannot appear
 within single quotes unless the option tt(RC_QUOTES) is set, in which case

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Here-strings and $functions

[Bart Schaefer]

On Jul 24, 10:12am, Sven Wischnowsky wrote:
}
} Bart Schaefer wrote:
} 
} > The only way to get this right is to know whether the original code was
} > really a here-string [...]
} > Otherwise we need to change the way <<< here-strings are parsed, so that
} > they're stored untokenized and with quotes stripped.
} 
} Err... it would be enough to just remnulargs() for here strings,
} wouldn't it?

Uhh, yeah, er, that's what I said, umm, sort of, sure, of course.

I think you can safely commit.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Enhanced shell

[Oliver Kiddle]

Bart Schaefer wrote:

| On Jul 23, 12:13pm, Oliver Kiddle wrote:
| }
| } > precmd, chpwd, periodic
| 
| I think these should always have been arrays of function names to be
| called, rather than magic function names in their own right.  Of course,
| we could do both.

Still, I think we should mention them with the point about the
problems. It could possibly also apply with the ERR trap.

| } > ${+foo}
| } yes and some other parameter expansion extensions
| 
| I think where we'll run into problems here, aside from the fact that the
| flags syntax used by zsh is a bit baroque, is with field splitting and
| whether various extensions apply before or after it happens (I have the
| impression that under traditional splitting rules, certain extensions
| can't possibly be made to apply after splitting).  I see you've suggested
| leaving out expansion flags, though.

That is roughly why I suggested leaving out expansion flags. I was
thinking of some of the simpler things like ${name/pattern/repl} and
${name:#pattern}.

| } backreferences (ksh93's possibly being better)
| 
| You didn't mention anything about that in workers/15348.  What are the
| differences?

Well, they are very different really. Basically instead of accessing
matched parts of the pattern later with variables, you can refer back
to earler parts of a pattern so, for example, [[ abba = @(?)bb\1 ]]
would return true because \1 will match whatever the first thing in
brackets matched.

workers/15348 was never going to be complete. I've found other things
since like [=c=] in a pattern matches "all characters with the same
primary collation weight as the character c".

| } directory stack, and access with ~num
| 
| You suggest having the directory stack as a special built-in but not the
| commands to manipulate it?

Well not really though with autopushd, that is how I use the directory
stack. The directory stack as a whole should probably be left out.

| } prompt percent substitutions
| Maybe.

I hadn't realised that bash used backslashes for this as Peter pointed
out so that'll probably be a problem

| } the common -L option to many builtins [similar to typeset -p in ksh]
| 
| Option letter clashes are another place where we may have trouble.  I
| think `-L' was a better choice than `-p', but ...

It doesn't help that -L is already used by typeset for left alignment.

| } array subscripts with start and end
| 
| More 0-vs-1 problems.

Yes, that's going to be a fairly big problem. We probably have to put
up with the standard following ksh and zsh using the ksharrays option.

| What about subscript flags and pattern subscripts?  array[(r)pat]
| "Reverse indexing"?

possibly.

| On Jul 23, 12:38pm, Peter Stephenson wrote:
| }
| } > typeset builtin (yes, it isn't in POSIX). Also, note that bash has
| } > obsoleted `typeset' in favour of `declare' which I think is better.
| }
| } It strikes me as a bit late for that now.
| 
| But it could be added as another alias for typeset; ksh93 doesn't have
| `local' either (does it?).

It is already there as another alias for typeset!

ksh93 doesn't have `local'.

| } > > ? disable/enable? [bash]
| } > not so useful. they are also common as commands to enable/disable
| } > printers (on AIX and Solaris at least).
| }
| } Yes, I can see it would be silly defining a standard which is known to have
| } this clash in it.
| 
| As has been pointed out, though, the functionality is useful.  Rather

Another possibility would be if the functionality was achieved by
modifying special arrays like those in the zsh/parameter module or
ksh93's .sh namespace. Anyway, I suggest we mention the functionality
in our list for David Korn and point out the issues when it comes up in
discussion with him.

| } > > ? typeset etc. -g?
| } > zsh/bash dynamically scoped local variables.
| } Hmm.  I don't know how this will pan out if ksh is going the way of
| } statically scoped parameters.
| Another typeset option?

Yes, as I think I mentioned in workers/15348, it is going to have to be
either no local variables or support both types (which is what Perl
does).

Oliver

_____________________________________________________________________
This message has been checked for all known viruses by the 
MessageLabs Virus Scanning Service. For further information visit
http://www.messagelabs.com/stats.asp

Re: Enhanced shell

[Zefram]

Peter Stephenson wrote:
>emulate builtin

We should only standardise "emulate posix_v2" as a way of specifying
"this is written for the enhanced shell standard".  Then future
revisions of the standard get to add new permitted arguments to emulate.
Without an emulate command in a script, the shell has to behave in a
manner compatible with the original POSIX.

The introduction of explicit specification of the language in which a
script is written makes it possible for us to make incompatible changes
to shell syntax (it's automatically compatibe because of the "emulate").
There's a noticeable problem otherwise, that the POSIX standard doesn't
leave many punctuation characters available for semantically interesting
uses (especially most of our extended glob characters).

Speaking of which, some kind of extended glob pattern syntax should
probably be part of the new standard (with emulate as described above,
the new syntax can be on by default in enhanced mode).

>coproc keyword (|& as per csh is already semi-standard and useful)

I was thinking about this.  The existing coproc semantics are rather
limiting ("<&p" only working with the one coprocess) and baroque (magic
fd closures on redirecting).  Having a coproc reserved word gives us a
place to put options on the coprocess -- we could have "coproc -i6 -o7"
to connect the coprocess's input to shell fd 6, etc., leaving the "p"
file descriptors alone.  We don't really need to standardise the "p"
file descriptors at all.

It's just a pity we can't make coproc a normal command -- it has to be
part of the shell grammar to pick up an entire pipeline, which seems to
be what we want.

>? ZDOTDIR? (or equivalent)

I'm very dubious about this kind of thing.  We should mostly ignore the
environment when deciding which dotfiles to execute.

Should we be standardising a larger set of dotfiles?  E.g., ".posix_env",
".posix_rc".

>? =cmd?

With cleaned-up semantics, yes.  It shouldn't be doing alias expansion.

>? glob qualifiers (perhaps in some standardised NO_BARE_GLOB_QUAL form)

The NO_BARE_GLOB_QUAL form should probably simply be an explicit way of
introducing the qualifier syntax we already have.  That was the idea
behind the option -- BARE_GLOB_QUAL makes glob qualifier syntax clash
with glob grouping syntax.

>CPUTYPE/HOSTYPE/OSTYPE, EGID/GID, EUID/UID,  ERRNO, HOST or HOSTNAME (bash)

Careful here.  My understanding of POSIX is that it has no special
variables (as we understand the term).  An assignable EGID, for example,
would add extra semantics to a name that looks like it's in the user's
namespace.  Adding special variables with the names we've traditionally
used would be a step backwards.

Instead, we should put all magic variables -- ones that affect the
shell or that the shell fiddles with, other than by explicit command
-- into a sub-namespace.  Things like PATH would have to remain where
they are, because they're needed in the environment.  But EGID should be
"egid.sh".  We can allow arbitrary special semantics of variables in the
".sh" namespace while leaving all names without "." as completely dumb
variables for the user to manage as he sees fit.

>`ESS_VERSION' (version no. of extended shell standard)

Perhaps this should be a magic array, "emulations.sh", listing the values
understood for "emulate".  Either way, it's not very useful in scripts.

Other stuff:

How about multios?  Pretty simple semantics, not difficult to implement,
though we ought to decide properly when an implicit tee gets waited for.
With the emulate command as described above, we can have multios be on
by default in "posix_v2" mode and (of course) off by default in POSIX
v1 mode.  We don't need to standardise "setopt multios", or most other
user-visible options.

<> redirection: is it already in the standard?

&> (and &>>): we should leave >& alone, not standardise the overloading
it gets in zsh.

prompt expansion: not sure if our % sequences should be standard.
However, we could quite nicely have the standard specify that prompts
get $-expanded (and no other special treatment), and then zsh users can
still do PS1='${(%)${:-%n@%m}}'.

modifiers on parameter expansion: the history modifiers could be
standardised for use in parameter expansion a la "${FOO:t}".  These seem
useful, and a clean syntax.

Process substitution, at least in the ">>(...)" and "<<(...)" form (where
the effect is merely to create pipes).  I'm dubious about standardising
the general >(...).

Brace expansion is already semi-standard.

-zefram

Re: Enhanced shell

[Peter Stephenson]

Zefram wrote:
> Peter Stephenson wrote:
> >emulate builtin
> 
> We should only standardise "emulate posix_v2" as a way of specifying
> "this is written for the enhanced shell standard".  Then future
> revisions of the standard get to add new permitted arguments to emulate.
> Without an emulate command in a script, the shell has to behave in a
> manner compatible with the original POSIX.

Hmm.  Zsh obviously doesn't, you have to trick it some other way.
 
> >? =cmd?
> 
> With cleaned-up semantics, yes.  It shouldn't be doing alias expansion.

Yes, I'm not sure how long that's been there, and I don't think it's all
that useful --- using the parameters module for this as for other
similar things strikes me as a better bet.  I find it hard to believe
anyone is relying on it, either, given that you can't tell if you're
going to end up with an external command, which are found in preference,
opposite to the normal order of lookup.  So I'm very tempted to remove
it or stick it in a deprecated option, off by default.

> >? glob qualifiers (perhaps in some standardised NO_BARE_GLOB_QUAL form)
> 
> The NO_BARE_GLOB_QUAL form should probably simply be an explicit way of
> introducing the qualifier syntax we already have.  That was the idea
> behind the option -- BARE_GLOB_QUAL makes glob qualifier syntax clash
> with glob grouping syntax.

I suppose you mean any enhanced-shell version should be a separate
option.  I can certainly see a point in that.

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: Enhanced shell

[Bart Schaefer]

On Jul 29, 11:07pm, Peter Stephenson wrote:
} Subject: Re: Enhanced shell
}
} Zefram wrote:
} > Peter Stephenson wrote:
} > >emulate builtin
} > 
} > We should only standardise "emulate posix_v2" as a way of specifying
} > "this is written for the enhanced shell standard".  Then future
} > revisions of the standard get to add new permitted arguments to emulate.
} > Without an emulate command in a script, the shell has to behave in a
} > manner compatible with the original POSIX.
} 
} Hmm.  Zsh obviously doesn't, you have to trick it some other way.

I should think that "has to behave in manner compatible with POSIX" would
apply only if the shell actually advertised itself under a standardized
name, e.g., "sh".

Should there be a standardized command-line option that is equivalent to
`emulate posix_v2'?  The only one available seems to be -z ... I don't
know if that already has a meaning in ksh; it doesn't yet in bash2.
 
} > >? =cmd?
} > 
} > With cleaned-up semantics, yes.  It shouldn't be doing alias expansion.
} 
} Yes, I'm not sure how long that's been there, and I don't think it's all
} that useful

It is, occasionally, extremely annoying, as one can't do [[ -x =cmd ]]
without getting syntax errors when cmd happens to be an alias.  On the
other hand, if cmd actually isn't found, that's an abortive error too.

Anyway, it appears to have gone in in late 2.4.xxx or sometime in 2.5, as
it is not in the version 2.4 sources I have and it's not mentioned in the
ChangeLog.3.0 file anywhere.  2.5 always was the mutant zsh from outer
space (no offense to those who worked on it; they meant well).

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Enhanced shell

[Bart Schaefer]

On Jul 29, 10:53am, Zefram wrote:
} Subject: Re: Enhanced shell
}
} >coproc keyword (|& as per csh is already semi-standard and useful)
} 
} It's just a pity we can't make coproc a normal command -- it has to be
} part of the shell grammar to pick up an entire pipeline, which seems to
} be what we want.

I think the syntax `coproc { LIST }' would be preferable to what we have
now.  Ksh has introduced `namespace NAME { LIST }' already.  If the LIST
associated with `coproc' were brace-delimited, we could use the regular
redirection operators to specify inputs and outputs, rather than using
options to the coproc pseudo-command.  Heck, you could even have coprocs
with multios!

} >? ZDOTDIR? (or equivalent)
} 
} I'm very dubious about this kind of thing.  We should mostly ignore the
} environment when deciding which dotfiles to execute.

That's why I suggested not importing ZDOTDIR unless geteuid() == getuid().
Seems to be the right balance.  (How does ksh deal with ENV ?)

} >CPUTYPE/HOSTYPE/OSTYPE, EGID/GID, EUID/UID,  ERRNO, HOST or HOSTNAME (bash)
} 
} Instead, we should put all magic variables -- ones that affect the
} shell or that the shell fiddles with, other than by explicit command
} -- into a sub-namespace.  Things like PATH would have to remain where
} they are, because they're needed in the environment.  But EGID should be
} "egid.sh".

Ksh namespaces are ${.namespace.parameter}, not ${parameter.namespace}.
I think we're probably going to be stuck with that.  However, I agree
that most of the "magic" variables should be in a namespace, and futher
I that the magic should be implemented with something like discipline
functions.  E.g., there should be a builtin command to change effective
UID, and assigning to .sh.euid should invoke that command.

} With the emulate command as described above, we can have multios be on
} by default in "posix_v2" mode and (of course) off by default in POSIX
} v1 mode.  We don't need to standardise "setopt multios", or most other
} user-visible options.

Hmm, perhaps that's true, but should we at least standardize a reserved
name (e.g., a .sh.options parameter) for changing the option settings?

Should we try to do anything at all about the plethora of single-letter
shell-startup command-line options?

} modifiers on parameter expansion: the history modifiers could be
} standardised for use in parameter expansion a la "${FOO:t}".  These seem
} useful, and a clean syntax.

The syntax isn't all that clean, really, once you get involved with using
more than one modifier per expansion ... and the semantics of :s/p/r and
:& etc. are rather baroque.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net