POSIX conformance in coreutils

POSIX conformance in coreutils

[Vin Shelton]

GNU coreutils (at least versions 5.2.1 and 5.3.0) enforce POSIX
conformance, so they don't like 'tail -1', instead they require
'tail -n 1'.

This causes problems in E01options.tst:

  # Count the number of directories on the stack.  Don't care what they are.
  dircount() { dirs -v | tail -1 | awk '{ print $1 + 1}'; }

tail: `-1' option is obsolete; use `-n 1'
Try `tail --help' for more information.


It's easy enough to change the E01options.tst script to use 'test -n
1', but that's probably not the right thing to do.

We could also add

_POSIX2_VERSION=199209

to the test.  This would probably not break older versions of tail,
and would force new versions of tail to accept the old syntax.  Patch
attached:

Index: ChangeLog
===================================================================
RCS file: /cvsroot/zsh/zsh/ChangeLog,v
retrieving revision 1.2463
diff -a -u -U0 -r1.2463 ChangeLog
--- ChangeLog	10 Jan 2005 18:43:19 -0000	1.2463
+++ ChangeLog	12 Jan 2005 00:58:28 -0000
@@ -0,0 +1,5 @@
+2005-01-11  Vin Shelton  <acs@xemacs.org>
+
+	* Test/E01options.ztst: Add _POSIX2_VERSION=199209 to silence new
+	coreutils' versions of tail.
+
Index: Test/E01options.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/E01options.ztst,v
retrieving revision 1.13
diff -a -u -r1.13 E01options.ztst
--- Test/E01options.ztst	9 Sep 2004 15:03:31 -0000	1.13
+++ Test/E01options.ztst	12 Jan 2005 00:58:32 -0000
@@ -116,7 +116,7 @@
 >scalar
   
   # Count the number of directories on the stack.  Don't care what they are.
-  dircount() { dirs -v | tail -1 | awk '{ print $1 + 1}'; }
+  dircount() { dirs -v | _POSIX2_VERSION=199209 tail -1 | awk '{ print $1 + 1}'; }
   unsetopt autopushd
   cd tmpcd
   dircount

HTH,
  Vin

Re: POSIX conformance in coreutils

[Stephane Chazelas]

On Tue, Jan 11, 2005 at 08:01:26PM -0500, Vin Shelton wrote:
> GNU coreutils (at least versions 5.2.1 and 5.3.0) enforce POSIX
> conformance, so they don't like 'tail -1', instead they require
> 'tail -n 1'.
> 
> This causes problems in E01options.tst:
> 
>   # Count the number of directories on the stack.  Don't care what they are.
>   dircount() { dirs -v | tail -1 | awk '{ print $1 + 1}'; }
> 
> tail: `-1' option is obsolete; use `-n 1'
> Try `tail --help' for more information.
[...]

YMMV.

~$ tail --version
tail (coreutils) 5.2.1
[...]
~$ tail -1 /dev/null
~$

tail -n 1, even if POSIX, is not portable. I think I've come
across a system where it was not supported recently (maybe
Solaris).


>   dircount() { dirs -v | tail -1 | awk '{ print $1 + 1}'; }

Maybe:

dircount() { print $((${${(z)${(f)"$(dirs -v)"}[-1]}[1]} + 1)); }

Or:

dircount() { dirs -v | awk '{n=$1}END{print n+1}'; }

-- 
Stéphane

Re: POSIX conformance in Solaris

[Danek Duvall]

On Wed, Jan 12, 2005 at 09:18:00AM +0000, Stephane Chazelas wrote:

> tail -n 1, even if POSIX, is not portable. I think I've come
> across a system where it was not supported recently (maybe
> Solaris).

Very possible.  When POSIX introduces an interface that would break
compatibility with already released Solaris commands, the functionality
gets stuck in /usr/xpg?.  So in this case /usr/bin/tail doesn't require (or
even support) -n, while /usr/xpg4/bin/tail supports (but doesn't require)
it.

Danek

Re: POSIX conformance in coreutils

[Peter Stephenson]

Vin Shelton wrote:
> GNU coreutils (at least versions 5.2.1 and 5.3.0) enforce POSIX
> conformance, so they don't like 'tail -1', instead they require
> 'tail -n 1'.

Might the environment variable POSIXLY_CORRECT be set?  That seems to be
the effect here.  We could unset it explicitly if that's the case.

Index: Test/ztst.zsh
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/ztst.zsh,v
retrieving revision 1.19
diff -u -r1.19 ztst.zsh
--- Test/ztst.zsh	26 Jul 2004 13:18:14 -0000	1.19
+++ Test/ztst.zsh	12 Jan 2005 10:26:43 -0000
@@ -28,6 +28,9 @@
 [[ -n $LC_COLLATE ]] && LC_COLLATE=C
 [[ -n $LANG ]] && LANG=C
 
+# POSIXLY_CORRECT can cause spurious error messages with "tail -<num>".
+unset POSIXLY_CORRECT
+
 # Set the module load path to correspond to this build of zsh.
 # This Modules directory should have been created by "make check".
 [[ -d Modules/zsh ]] && module_path=( $PWD/Modules )

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: POSIX conformance in coreutils

[Stephane Chazelas]

On Wed, Jan 12, 2005 at 10:27:01AM +0000, Peter Stephenson wrote:
> Vin Shelton wrote:
> > GNU coreutils (at least versions 5.2.1 and 5.3.0) enforce POSIX
> > conformance, so they don't like 'tail -1', instead they require
> > 'tail -n 1'.
> 
> Might the environment variable POSIXLY_CORRECT be set?  That seems to be
> the effect here.  We could unset it explicitly if that's the case.
[...]

As Vin said, it's more about the _POSIX2_VERSION env variable and
the default value (defined at compile time of it).

_POSIX2_VERSION=199209 tail -1

should be OK, whatever the value of POSIXLY_CORRECT for
coreutils 5.2.1 at least.

(unset POSIXLY_CORRECT; _POSIX2_VERSION=200112 tail -1)
tail: `-1' option is obsolete; use `-n 1'
Try `tail --help' for more information.


But I seem to remember there was also a warning message in some
other version, and it may not have been triggered by the same
envvar values... sigh.

-- 
Stéphane

Re: POSIX conformance in coreutils

[Vin Shelton]

Peter Stephenson <pws@csr.com> writes:

> Vin Shelton wrote:
>> GNU coreutils (at least versions 5.2.1 and 5.3.0) enforce POSIX
>> conformance, so they don't like 'tail -1', instead they require
>> 'tail -n 1'.
>
> Might the environment variable POSIXLY_CORRECT be set?  That seems to be
> the effect here.  We could unset it explicitly if that's the case.
>
> Index: Test/ztst.zsh
> ===================================================================
> RCS file: /cvsroot/zsh/zsh/Test/ztst.zsh,v
> retrieving revision 1.19
> diff -u -r1.19 ztst.zsh
> --- Test/ztst.zsh	26 Jul 2004 13:18:14 -0000	1.19
> +++ Test/ztst.zsh	12 Jan 2005 10:26:43 -0000
> @@ -28,6 +28,9 @@
>  [[ -n $LC_COLLATE ]] && LC_COLLATE=C
>  [[ -n $LANG ]] && LANG=C
>  
> +# POSIXLY_CORRECT can cause spurious error messages with "tail -<num>".
> +unset POSIXLY_CORRECT
> +
>  # Set the module load path to correspond to this build of zsh.
>  # This Modules directory should have been created by "make check".
>  [[ -d Modules/zsh ]] && module_path=( $PWD/Modules )
>

Hi Peter,

I checked and I do not have POSIXLY_CORRECT set.  Here's what the
'Standards conformance' section of the coreutils info says:

  In a few cases, the GNU utilities' default behavior is incompatible
  with the POSIX standard.  To suppress these incompatibilities, define
  the `POSIXLY_CORRECT' environment variable.  Unless you are checking
  for POSIX conformance, you probably do not need to define
  `POSIXLY_CORRECT'.

  Newer versions of POSIX are occasionally incompatible with older
  versions.  For example, older versions of POSIX required the command
  `sort +1' to sort based on the second and succeeding fields in each
  input line, but starting with POSIX 1003.1-2001 the same command is
  required to sort the file named `+1', and you must instead use the
  command `sort -k 2' to get the field-based sort.

  The GNU utilities normally conform to the version of POSIX that is
  standard for your system.  To cause them to conform to a different
  version of POSIX, define the `_POSIX2_VERSION' environment variable to
  a value of the form YYYYMM specifying the year and month the standard
  was adopted.  Two values are currently supported for `_POSIX2_VERSION':
  `199209' stands for POSIX 1003.2-1992, and `200112' stands for POSIX
  1003.1-2001.  For example, if you are running older software that
  assumes an older version of POSIX and uses `sort +1', `head -10', or
  `tail +10', you can work around the compatibility problems by setting
  `_POSIX2_VERSION=199209' in your environment.

This sentence: "The GNU utilities normally conform to the version of
POSIX that is standard for your system" seems to indicate that some
kind of inference is going on.

I don't have a strong opinion about how we fix this.  Either one of
Stephane's proposed solutions works just as well.

  - Vin

Re: POSIX conformance in coreutils

[Peter Stephenson]

Vin Shelton wrote:
> I checked and I do not have POSIXLY_CORRECT set.  Here's what the
> 'Standards conformance' section of the coreutils info says:

It doesn't look like we can expect much sense, then.

Here is a more heavyweight change.

Index: Test/ztst.zsh
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/ztst.zsh,v
retrieving revision 1.20
diff -u -r1.20 ztst.zsh
--- Test/ztst.zsh	12 Jan 2005 10:32:20 -0000	1.20
+++ Test/ztst.zsh	12 Jan 2005 15:56:50 -0000
@@ -28,9 +28,6 @@
 [[ -n $LC_COLLATE ]] && LC_COLLATE=C
 [[ -n $LANG ]] && LANG=C
 
-# POSIXLY_CORRECT can cause spurious error messages with "tail -<num>".
-unset POSIXLY_CORRECT
-
 # Set the module load path to correspond to this build of zsh.
 # This Modules directory should have been created by "make check".
 [[ -d Modules/zsh ]] && module_path=( $PWD/Modules )
@@ -60,6 +57,33 @@
 
 integer ZTST_testfailed
 
+# This is POSIX nonsense.  Because of the vague feeling someone, somewhere
+# may one day need to examine the arguments of "tail" using a standard
+# option parser, every Unix user in the world is expected to switch
+# to using "tail -n NUM" instead of "tail -NUM".  Older versions of
+# tail don't support this.
+tail() {
+  emulate -L zsh
+
+  if [[ -z $TAIL_SUPPORTS_MINUS_N ]]; then
+    local test
+    test=$(echo "foo\nbar" | command tail -n 1 2>/dev/null)
+    if [[ $test = bar ]]; then
+      TAIL_SUPPORTS_MINUS_N=1
+    else
+      TAIL_SUPPORTS_MINUS_N=0
+    fi
+  fi
+
+  integer argi=${argv[(i)-<->]}
+
+  if [[ $argi -le $# && $TAIL_SUPPORTS_MINUS_N = 1 ]]; then
+    argv[$argi]=(-n ${argv[$argi][2,-1]})
+  fi
+
+  command tail "$argv[@]"
+}
+
 # The source directory is not necessarily the current directory,
 # but if $0 doesn't contain a `/' assume it is.
 if [[ $0 = */* ]]; then


-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: POSIX conformance in coreutils

[Oliver Kiddle]

On 12 Jan, Peter wrote:
> Vin Shelton wrote:
> > I checked and I do not have POSIXLY_CORRECT set.  Here's what the
> > 'Standards conformance' section of the coreutils info says:
> 
> It doesn't look like we can expect much sense, then.
> 
> Here is a more heavyweight change.

> +  if [[ -z $TAIL_SUPPORTS_MINUS_N ]]; then

Would it perhaps be easier to just use something like this:

  tail() {
    local arr
    arr=( ${(f)"$(</dev/fd/0)"} )
    print -lr -- $arr[${1:--10},-1]
  }

It's shorter and potentially tests other zsh features as a side-effect
(such as emulating /dev/fd in redirections).

Oliver

Re: POSIX conformance in coreutils

[Peter Stephenson]

Oliver Kiddle wrote:
> Would it perhaps be easier to just use something like this:
> 
>   tail() {
>     local arr
>     arr=( ${(f)"$(</dev/fd/0)"} )
>     print -lr -- $arr[${1:--10},-1]
>   }
> 
> It's shorter and potentially tests other zsh features as a side-effect
> (such as emulating /dev/fd in redirections).

I don't think /dev/fd is currently faked to that extent.  There's
special handling for tests, and /dev/fd is sometimes used to *implement*
process substitution, but it won't handle it specially there.  You could
build up arr some other way, of course.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

**********************************************************************

Re: POSIX conformance in coreutils

[Bart Schaefer]

On Jan 12,  9:19am, Vin Shelton wrote:
} Subject: Re: POSIX conformance in coreutils
}
} 'Standards conformance' section of the coreutils info says:
} 
}   Newer versions of POSIX are occasionally incompatible with older
}   versions.  For example, older versions of POSIX required the command
}   `sort +1' to sort based on the second and succeeding fields in each
}   input line, but starting with POSIX 1003.1-2001 the same command is
}   required to sort the file named `+1', and you must instead use the
}   command `sort -k 2' to get the field-based sort.

This is, by the way, only the GNU coreutils maintainer's interpretation
of the POSIX requirements; there was quite a lengthy argument about this
on the austin-group mailing list back in October 2004 or so, though I
don't recall whether an "official" resolution was ever approved.  At the
least there was not an agreement that this behavior really is required,
so far as I could tell.