Proposal to standardize the shell

Proposal to standardize the shell

[David Korn]

Hi, I am the author of ksh.  One of my primary goals in writing ksh was
to provide a portable and universal scripting language for UNIX.

I worked with the POSIX group in the development of a shell standard
over ten years ago.  Since then there have been numerous developments
and various shells have emerged with many new and useful features.

I would like to see if it is possible to come up with an enhanced
shell standard that could be met by at least ksh, bash, and zsh.
I have already contacted the bash author Chet Ramey, and he is
interested.  I am especially concerned with programming issues,
but there might be some interactive features that could also
be standardized.

If the zsh workers are also interested in this effort let me know.
Also, if you know who else should be included, please let me know.
As a starting point we should develop the following lists:

1.	Extensions to POSIX that are in all three shells.
2.	Incompatibilities between shells.
3.	Extensions or features in each shell that should be in
	the standard.
4.	Things that should be obsoleted.
5.	How do handle compatibilty issues between shells and POSIX.

Once we have achieved some type of consensus, we should see if
we can get some standards organization, such as POSIX or X/Open
to work on standardization.

David Korn
research!dgk
dgk@research.att.com

Re: Proposal to standardize the shell

[Peter Stephenson]

David Korn wrote:
> I would like to see if it is possible to come up with an enhanced
> shell standard that could be met by at least ksh, bash, and zsh.

We're definitely interested in tracking standards.

A big question in our case is how to start up the shell in a way which
enforces compliance; zsh has been a bit wayward right from the start and
it's too late to change that now.  So I'd be interested in some common
command such as a version of our `emulate' command, e.g. with the argument
`emulate posix' or `emulate enhanced' if we can come up with an enhanced
version of it to ensure whatever shell you're in is likely to be as
receptive as possible.  No doubt this is less important for ksh and bash
which have been designed from the ground up with this sort of thing in
mind, but it would go a long way towards making scripts shareable in a
simple way.

Quite what degree of compatibility is possible between the more advanced
features I really don't know, but it's worth thinking about.  We've been
treating ksh93 as a sort of de facto standard for some of the advanced
features, though there's still quite a lot of that missing and (according
to my reading of Oliver Kiddle's post to zsh-workers just now) some of it
appears to be incompatible.

In interactive features, there might be something we do with simple things
like key binding syntax, for example.  I doubt if it'll go much deeper than
that; it depends too much on the implementation of the line editor, which
I'm sure is very different in each case.

So this could well be a big can of worms, but I'm sure there's plenty to
talk about.

-- 
Peter Stephenson <pws@pwstephenson.fsnet.co.uk>
Work: pws@csr.com
Web: http://www.pwstephenson.fsnet.co.uk

Re: Proposal to standardize the shell

[Oliver Kiddle]

David Korn wrote:

> I would like to see if it is possible to come up with an enhanced
> shell standard that could be met by at least ksh, bash, and zsh.
> 
> If the zsh workers are also interested in this effort let me know.

The effort does sound interesting to me. Your suggested starting points
seem sensible and I'd be keen for there to be input from us so that
current zsh features and incompatibilities can be accounted for in the
process.

> Also, if you know who else should be included, please let me know.

Maybe the pdksh developers but I'm not sure how alive that project is.

Peter Stephenson wrote:

> Quite what degree of compatibility is possible between the more advanced
> features I really don't know, but it's worth thinking about.  We've been
> treating ksh93 as a sort of de facto standard for some of the advanced
> features, though there's still quite a lot of that missing and (according
> to my reading of Oliver Kiddle's post to zsh-workers just now) some of it
> appears to be incompatible.

Certainly, there may be problems with some of the more advanced
features but I think there is a fair amount of scope for extensions to
POSIX which wouldn't cause many problems. If I'm not mistaken, there
were quite a few extensions to POSIX that were in ksh88 and these are
fairly well followed by bash and zsh. I think there are also a number of
newer features where we don't have incompatibilities. Process
substitution would be one example.

Can anyone please point me in the direction of the canonical web
location for the POSIX shell definition because my search wasn't very
successful?

> In interactive features, there might be something we do with simple things
> like key binding syntax, for example.  I doubt if it'll go much deeper than

I didn't mention ksh93 key binding in my recent post but it is very
different. In short, there is a KEYBD trap which you override to change
keypresses. I expect the zsh/bash bindkey builtin was inspired by tcsh.

There might be other areas of interactive features which can be
standardised on though. Parts of the history mechanism, fc options
maybe.

Oliver

_____________________________________________________________________
This message has been checked for all known viruses by the 
MessageLabs Virus Scanning Service. For further information visit
http://www.messagelabs.com/stats.asp

Re: Proposal to standardize the shell

[Bart Schaefer]

On Jul 9,  3:42pm, David Korn wrote:
} 
} Hi, I am the author of ksh.

(Did anyone here not know that?)

} I would like to see if it is possible to come up with an enhanced
} shell standard that could be met by at least ksh, bash, and zsh.
} 
} If the zsh workers are also interested in this effort let me know.

I'm certainly interested in participating.  I don't have anything to
add to what Oliver and PWS have said, at this point.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Proposal to standardize the shell

[Zefram]

David Korn wrote:
>If the zsh workers are also interested in this effort let me know.

I'm very interested in developing a new shell standard.

I have some thoughts about things that could go on your lists, but not
yet sufficiently organised to be particularly useful.  Are we going to
have a new forum to discuss these things?

-zefram

Re: Proposal to standardize the shell

[Felix Rosencrantz]

I'm more interested in zsh for it's interactive features than it's      
scripting capabilities.  I think it would be very useful if the         
interactive features became standard.                                   

Some of the things I think that would be useful to standardize would    
include CSH-style history, and zsh-style completion.                    

Though it would also be useful if there were some additional work       
generalizing and developing some additional interactive features        
that are already in some shells.  Some of these topics to me would      
be command line correction, command line validation, command line       
prediction, and command line building.                                  

Command line correction: There are already several shells that will     
suggest alternatives to a command that is not found.  It would be great 
if that could be made more general to also correct arguments and flags 
of the command line.  This seems related to completion, though even    
with zsh's completion it's not easy to do.                              

Command line validation:  There are already several shells that         
provide some sort of "rm *" protection.  It would be great if this      
functionality could somehow be made more general to allow the shell to  
verify other dangerous/questionable commands.                           

Command line prediction: It would be great if the shell could           
effectively use command history and user hints to predict what the 
user wants to do next.  There has been some academic research in this   
area that might help.  Prediction is already effectively used in the    
text fields of GUI tools (e.g. web browsers).  And zsh has a prediction-like
widget based on completion.

Command line building:  The VMS OS could check command arguments before 
running the command, and ask the user for missing arguments.  It might  
be useful if the shells could also do this.                             

Some of these features might be done by providing some hooks into the   
shell and providing some good mechanisms for parsing command lines.     
Parsing is not easy to do from a script, it's a little more involved     
than just a getopt function. First one has to deal with the shell syntax,    
and then with the command line syntax of individual commands.  Shells    
make it easy to work with files and the filesystems.  It would be great 
if it was just as easy to work with command lines.                      


Another thing that might be useful to standardize is the API for        
loadable modules.  Not sure if that can be done in a generic way across 
shells, but providing a mechanism for extensions that could be written  
in other languages would be very useful.  I would expect that it would  
be possible to use C/C++ and Java to write such modules/extensions.     

I would be interested in hearing discussions (e.g. mailing              
list/archive) of the enhanced shell standard.                          

-FR.


__________________________________________________
Do You Yahoo!?
Get personalized email addresses from Yahoo! Mail
http://personal.mail.yahoo.com/