POSIX compliance of shells - where to ask/talk about it?

POSIX compliance of shells - where to ask/talk about it?

[Ian Lynagh]

Hi all,

Where should one ask/talk about POSIX shell compliance (looking for a
mailing list probably)?

I am reading the standard at
http://www.opengroup.org/onlinepubs/007904975/utilities/xcu_chap02.html
and finding it very imprecise - disappointingly so (am I looking in the
wrong place?). I am therefore looking to implementations to clarify the
standard, but with things like this (all shells invoked as sh, bash
given --posix):

printf "%s\n" `echo '\\\\\\\\\\\\\\\\'`
printf "%s\n" "`echo '\\\\\\\\\\\\\\\\'`"

Using shells/ash:
\\\\
\\\\

Using shells/bash:
\\\\\\\\
\\\\\\\\

Using shells/zsh:
\\\\
\\\\\\\\

it is not really helping...


% zsh --version
zsh 4.0.4 (i686-pc-linux-gnu)
% bash --version
GNU bash, version 2.04.0(1)-release (i386-pc-linux-gnu)
Copyright 1999 Free Software Foundation, Inc.
% dpkg -l ash bash zsh
ii  ash            0.3.8-37       NetBSD /bin/sh
ii  bash           2.04-9         The GNU Bourne Again SHell
ii  zsh            4.0.4-31       A shell with lots of features.


Thanks
Ian

Re: POSIX compliance of shells - where to ask/talk about it?

[Andreas Schwab]

igloo@earth.li (Ian Lynagh) writes:

|> Hi all,
|> 
|> Where should one ask/talk about POSIX shell compliance (looking for a
|> mailing list probably)?
|> 
|> I am reading the standard at
|> http://www.opengroup.org/onlinepubs/007904975/utilities/xcu_chap02.html
|> and finding it very imprecise - disappointingly so (am I looking in the
|> wrong place?). I am therefore looking to implementations to clarify the
|> standard, but with things like this (all shells invoked as sh, bash
|> given --posix):
|> 
|> printf "%s\n" `echo '\\\\\\\\\\\\\\\\'`
|> printf "%s\n" "`echo '\\\\\\\\\\\\\\\\'`"
|> 
|> Using shells/ash:
|> \\\\
|> \\\\
|> 
|> Using shells/bash:
|> \\\\\\\\
|> \\\\\\\\
|> 
|> Using shells/zsh:
|> \\\\
|> \\\\\\\\

I think the key difference here is whether echo is interpreting
backslashes or not.  By default, bash's builtin echo does not interpret
backslashes, and if you use "echo -e" you get the same output as ash.  zsh
seems to be the odd case again.

Andreas.

-- 
Andreas Schwab, SuSE Labs, schwab@suse.de
SuSE Linux AG, Deutschherrnstr. 15-19, D-90429 Nürnberg
Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: POSIX compliance of shells - where to ask/talk about it?

[Peter Stephenson]

Sorry for the late reply... we've been implementing scatternets :-/ and
it's been too hot to stay at home and stare at a computer at the weekend.
(Yes, it's now raining again.  Oh, and the power's just gone down, so
everybody else has gone for lunch, so I might as well finish this...)
Here is Your Handy Cut-Out-'N'-Keep Guide To Backslashing In Zsh.

Ian Lynagh wrote:
> Hi all,
> 
> Where should one ask/talk about POSIX shell compliance (looking for a
> mailing list probably)?

You're probably not going to find anything better than one of the Open
Group mailing lists, if you want the people that define the spec.  The
Austin Group is the one responsible for tinkering with Posix and other
associated standards, including the shell specification.

> printf "%s\n" `echo '\\\\\\\\\\\\\\\\'`
> printf "%s\n" "`echo '\\\\\\\\\\\\\\\\'`"
>
> ...
>
> Using shells/zsh [invoked as sh]:
> \\\\
> \\\\\\\\

There are various things going on here; in particular, both of the
options BSD_ECHO and SH_GLOB are set for Bourne shell emulation.

Here is standard zsh as a baseline:

\\\\
\\\\

The number of backslashes has been halved twice, once by the original
parsing of the backquoted expression, once by `echo' (not by the printf
"%s", of course).  The former happens because inside backquotes you can
quote backquotes, so backslashes are active.  Compare with:

% printf "%s\n" $(echo '\\\\\\\\\\\\\\\\\') 
\\\\\\\\\

Since $(...) is terminated by a parenthesis, and the parenthesis is a
standard terminator character in the shell (compare subshells with
`(...)'), no backslash-stripping happens inside; the only layer of
stripping is due to the echo.  This distinction appears to be in bash,
too.

BSD_ECHO turns off the feature that the echo builtin (but not the print
or printf builtins) interpret backslashes.  This doubles the number you
get back from the original commands:

\\\\\\\\
\\\\\\\\

I think this is probably responsible for the other differences you saw,
between bash and ash; it's certainly the difference between bash and
the Solaris 8 system here's sh, since I've just tried
  echo '\\\\\\\\\\\\\\\\'
on both and only sh halves the backquotes.  This makes sense since the
sh in this case is presumably of AT&T SYSV ancestry (the Solaris people
can correct me if I'm wrong), hence doesn't have a BSD-like echo;
/bin/echo is similar.

The option SH_GLOB makes characters which are substituted by a parameter
or command substitution elegible for further interpretation, providing
the substitution was not quoted.  In zsh, that means the number of \'s
returned when the `...` is not quoted is halved, but inside double
quotes it isn't.  This gives what you observed, viz.

\\\\
\\\\\\\\

This is definitely all a bit of a mess.  I suspect the last feature can
be regarded as a bug.  The unfortunate fact that other shells think it's
a good idea for text from parameter and command substitutions to be
processed further as patterns etc. has given no end of trouble in zsh,
and the SH_GLOB and SH_WORD_SPLIT options still don't emulate that
behaviour (which, as far as I can tell, none of the current bunch of
developers really like) properly.

Maybe that :-/ should have been :-\.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: POSIX compliance of shells - where to ask/talk about it?

[Oliver Kiddle]

On 30 Jul, Peter wrote:

> The option SH_GLOB makes characters which are substituted by a parameter
> or command substitution elegible for further interpretation, providing

Except that GLOB_SUBST is the option from sh emulation which is doing that.

Oliver

This e-mail and any attachment is for authorised use by the intended recipient(s) only.  It may contain proprietary material, confidential information and/or be subject to legal privilege.  It should not be copied, disclosed to, retained or used by, any other party.  If you are not an intended recipient then please promptly delete this e-mail and any attachment and all copies and inform the sender.  Thank you.

Re: POSIX compliance of shells - where to ask/talk about it?

[Peter Stephenson]

Oliver Kiddle wrote:
> On 30 Jul, Peter wrote:
> 
> > The option SH_GLOB makes characters which are substituted by a parameter
> > or command substitution elegible for further interpretation, providing
> 
> Except that GLOB_SUBST is the option from sh emulation which is doing that.

Sorry, that's what I meant.  I keep getting them confused.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************