pws-18 is go

pws-18 is go

[Peter Stephenson]

http://www.ifh.de/~pws/computing/
-rw-r--r--  1 pws  quadrics  924653 May 11 17:03 zsh-3.1.5-pws-18.tar.gz
-rw-r--r--  1 pws  quadrics  374488 May 11 17:03 zsh-3.1.5-pws-18.doc.tar.gz
-rw-r--r--  1 pws  quadrics  750783 May 11 16:29 zsh-3.1.5-pws-18.tar.bz2
-rw-r--r--  1 pws  quadrics  252180 May 11 16:29 zsh-3.1.5-pws-18.doc.tar.bz2

This includes the large file support for SunOS, although Andrej's points
are all valid.  Note that setting CPPFLAGS by hand when runing configure
will disable it.  I hope the problems are small enough to allow upgrading
to full internal large file support gradually.  We should probably think
about handling integer shell parameters as 64 bit wherever this is
available by testing for long long or quad_t; the extra size is trivial
compared with the added convenience.

It doesn't include the patch to turn `incrementalappendhistory' into
`incappendhistory' to fit in the canonical size limits because I forgot,
sorry.  It's trivial and I'll do it now.

Changes:

Bart: 6188: compinit speedup

pws: 6193: [un]setopt shouldn't complain when setting an unsettable option
to the value it already has

Sven: 6194: complete assoc array arguments by default where necessary

Sven: 6195: _expand_word and _correct_word change.

Sven: 6197: off by one error parsing assignment in completion

pws: 6202: trivial _correct_filename change, ^Xc -> ^XC

pws: 6205: use FIONREAD wherever defined, read chars immediately into
buffer

Bart: 6213: race condition in $(...), use waitforpid() instead of
unblocking child (which shouldn't happen until later).

Tanaka Akira: 6219: initialize a variable in zle_tricky.c

Wayne: 6220: various compilation warnings

pws: 6224: alter 6205 to read chars only when necessary, but ensure
terminal is set appropriately.

pws: 6227: configuration for large file support (from bash aclocal.m4).

pws: 6235: unset -m shouldn't restore unset parameters; unsetting a global
should remove it from paramtab even inside a function.

Wayne: 6236: history changes to improve management of duplicate lines,
incremental history read/write, and sharing history

pws: 6237: window size code upgraded from 3.0.6-pre2, plus Bart's patch
4447.

pws: 6238: Wayne's share_history option set in ksh emulation

pws: 6239: need space after incrementalappendhistory for kshoptionprint

pws: 6240: a pipeline ending in a builtin didn't attach to the tty pgrp.

Wayne: 6241: history editing can use foreign history commands; history
appended in hend() instead of hbegin()

Sven: 6046: nested parameter expansions can return either arrays or
scalars.

pws: 6246: doc changes for 6046, plus subscripts done properly

Sven: 6249: fix for 6046 (problem showed up with $(...))

Wayne: 6255: more history: zle toggle between local/global history;  `zle
widget' can now take a direct numeric argument; small tweaks

pws: 6257: rewrite 6240 for any old builtin structure after the pipeline

pws: 6258: yet another attempt at the same problem

pws: 6259: second version of compinstall

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

LFS notes RE: pws-18 is go

[Andrej Borsenkow]

>
> This includes the large file support for SunOS, although Andrej's points
> are all valid.

Unfotunately, after more close look at it, this is broken.

1. It sets both LFS_CLFAGS *and* LFS64_CFLAGS. The two are for *very*
different modes of compilation; you may select one or other - but you cannot
assume, that mixing them is safe (even, if it helds true for any particular
OS). LFS_CFLAGS & Co is for the case, when you use normal, usual interfaces
(open/lseek ...) and makes off_t/ino_t 64 bit long. LFS64_CFLAGS & Co is for
the case, when you use explicit 64 bit interfaces (open64/lseek64/...) and
introduces new types off64_t, ino64_t and some others.

2. Zsh is using fseek/ftell. These are using long and are incompatible with
LFS; to be on safe side, one must use fseeko/ftello that use off_t.

3. Setting LFS[64]_CFLAGS & Co is not enough!!! One must set either
_LARGEFILE_SOURCE *or* _LARGEFILE64_SOURCE to indicate, which mode is used
(a side note, that _LARGEFILE64_SOURCE has some problems on our system).

For reference, look at
http://ftp.sas.com/standards/large.file/x_open.20Mar96.html I think, this
should be in Single Unix as well.

4. (minor) summary at the end of configure does not include LFS flags.

I urge you once again to really think about adopting LP64 model. If system
supports LFS it should (most probably) support LP64 as well. It does not
have as nice configure tests, must I admit ... but it does not require
changes in ZSH sources.

regards

/andrej

PATCH: Re: LFS notes RE: pws-18 is go

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> > This includes the large file support for SunOS, although Andrej's points
> > are all valid.
> 
> Unfotunately, after more close look at it, this is broken.
> 
> 1. It sets both LFS_CLFAGS *and* LFS64_CFLAGS.

Well, this is what I got from bash.  I changed it to use just LFS_CFLAGS,
where just fseek/ftell are explicitly affected.

> 2. Zsh is using fseek/ftell. These are using long and are incompatible with
> LFS; to be on safe side, one must use fseeko/ftello that use off_t.

This is now handled by #define's in system.h: in fact, on Solaris ftello is
defined to ftello64 anyway.  I've tested it and nothing seems immediately
wrong.  I fixed up one file pointer in input.c to use off_t (this is safe
as configure handles the check), other references to seek/tell were OK.
Strictly that means zalloc() should be using off_t, but I shudder to think
at this actually being necessary.

> 3. Setting LFS[64]_CFLAGS & Co is not enough!!! One must set either
> _LARGEFILE_SOURCE *or* _LARGEFILE64_SOURCE to indicate, which mode is used
> (a side note, that _LARGEFILE64_SOURCE has some problems on our system).

Eh?  The LFS_CFLAGS getconf stuff just probes for the right flags to set.
On Solaris 5.6 I get CPPFLAGS set to
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
Is the getconf test working where you are?  If not, none of this stuff
applies anyway.

> 4. (minor) summary at the end of configure does not include LFS flags.

I've added CPPFLAGS to the things it prints out, which is appropriate for
Solaris, at least.

> I urge you once again to really think about adopting LP64 model. If system
> supports LFS it should (most probably) support LP64 as well. It does not
> have as nice configure tests, must I admit ... but it does not require
> changes in ZSH sources.

As I said, I can't because I don't have it, or at least can't find it.  If
someone else wants to send me the stuff I'll apply it.  I don't even use
Solaris much.

> Additional note: I think, until all issues are resolved, the configure
> parameter (--enable-lfs would do) is needed. Currently I simply cannot
> disable LFS even if I don't need it at all (I don't want to set CPPFLAGS to
> some silly value; and in my case they are CFLAGS and not CPPFLAGS in the
> first place).

I've added this, but all you're supposed to do is set CPPFLAGS to anything,
even empty, and it will suppress modification of all the flags by the large
file check.

Somebody should read through what I've added to INSTALL.

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

RE: PATCH: Re: LFS notes RE: pws-18 is go

[Andrej Borsenkow]

>
> > 3. Setting LFS[64]_CFLAGS & Co is not enough!!! One must set either
> > _LARGEFILE_SOURCE *or* _LARGEFILE64_SOURCE to indicate, which
> mode is used
> > (a side note, that _LARGEFILE64_SOURCE has some problems on our system).
>
> Eh?  The LFS_CFLAGS getconf stuff just probes for the right flags to set.
> On Solaris 5.6 I get CPPFLAGS set to
> -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64
> Is the getconf test working where you are?  If not, none of this stuff
> applies anyway.
>

>From the page I quoted:

3.3.1 Compilation Environment - Visibility of Additions to the API
Applications which define the macro _LARGEFILE_SOURCE to be 1 before
inclusion of any header will enable at least the functionality described in
2.0 Changes to the Single UNIX Specification on implementations that support
these features. Implementations that support these features will define
_LFS_LARGEFILE to be 1 in <unistd.h>, as described in 3.1.2.12 <unistd.h>.

3.3.4 Utilities: Optional Method for Specifying the Size of an off_t
For programs to take advantage of different environments, it is necessary to
compile them for each particular environment. For programs to make use of
the features described in this section they must be compiled with new
compiler and linker options. The getconf utility called with the new
arguments can be used to generate compiler and linker options.
Example 1:

An example of compiling a program with a "large" off_t and that uses
fseeko() and ftello() and uses yacc:

   c89 -D_LARGEFILE_SOURCE     -o foo      \
        $(getconf LFS_CFLAGS)  y.tab.c b.o \
        $(getconf LFS_LDFLAGS)             \
        -ly $(getconf LFS_LIBS)
Example 2:
An example of compiling a program with a "large" off_t and that does not use
fseeko() and ftello() and has no application specific libraries:

   c89  $(getconf LFS_CFLAGS)  a.c         \
        $(getconf LFS_LDFLAGS)             \
        $(getconf LFS_LIBS)

A.3.2.1 Compilation Environment - Visibility of Additions to the API
Applications which use the fseeko() and ftello() interfaces should define
_LARGEFILE_SOURCE to be 1, then include <unistd.h> and then test that
_LFS_LARGEFILE is 1 to determine if the additional functionality is indeed
available. This additional functionality may be available even when
_LARGEFILE_SOURCE is not defined, but it will not be available to strictly
conforming X/Open programs.
This macro does not affect the size of off_t (see 3.3.3 Mixed API and
Compile Environments Within a Single Process).


In other words:

to use 64 bit off_t, one has to use LFS_CFLAGS
to use ftello/fseeko one must set _LARGEFILE_SOURCE
Solaris is non-conformant in automagically setting _LARGEFILE_SOURCE in
LFS_CFLAGS

>
> I've added this, but all you're supposed to do is set CPPFLAGS to
> anything,
> even empty, and it will suppress modification of all the flags by
> the large
> file check.
>

Hmm... I've tried it, but still got all LFS flags. I'll recheck it when I
have a bit more time.

/andrej

PATCH: Re: LFS notes RE: pws-18 is go

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> to use ftello/fseeko one must set _LARGEFILE_SOURCE
> Solaris is non-conformant in automagically setting _LARGEFILE_SOURCE in
> LFS_CFLAGS

How about this?

--- aczsh.m4.lfs2	Wed May 12 09:49:40 1999
+++ aczsh.m4	Wed May 12 12:05:22 1999
@@ -59,7 +59,14 @@
 case $ac_result in
 yes)
   for ac_shellvar in $ac_shellvars; do
-    eval $ac_shellvar=\$ac_test_$ac_shellvar
+    case "`eval echo $ac_shellvar-\\\$ac_test_$ac_shellvar`" in
+      CPPFLAGS*-D_LARGEFILE_SOURCE*) eval $ac_shellvar=\$ac_test_$ac_shellvar
+	;;
+      CPPFLAGS*) 
+        eval $ac_shellvar="\"-D_LARGEFILE_SOURCE \$ac_test_$ac_shellvar\""
+	;;
+      *) eval $ac_shellvar=\$ac_test_$ac_shellvar
+    esac
   done ;;
 esac
 ])

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

RE: PATCH: Re: LFS notes RE: pws-18 is go

[Andrej Borsenkow]

>
> I've added this, but all you're supposed to do is set CPPFLAGS to
> anything,
> even empty, and it will suppress modification of all the flags by
> the large
> file check.
>

I just realized, that this message is titled with "PATCH:", but the patch
itself is missing (at least, I have not got it).

And about CPPFLAGS - I once more tried it. Setting CPPFLAGS=" " does not
help.

/andrej

PATCH: try again: Re: LFS notes

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> I just realized, that this message is titled with "PATCH:", but the patch
> itself is missing (at least, I have not got it).

That's strange.

--- INSTALL.lfs	Tue Mar  2 09:42:37 1999
+++ INSTALL	Wed May 12 10:16:33 1999
@@ -244,3 +244,12 @@
      zprofile=pathname   # the full pathname of the global zprofile script
      zlogout=pathname    # the full pathname of the global zlogout script
      dynamic             # allow dynamically loaded binary modules
+     lfs                 # allow configure check for large files
+
+The option --enable-lfs turns on the configure check for support for large
+files; some 32-bit systems use this as a way of getting across the 2GB
+barrier.  It is a configure option because it is not yet completely
+supported internally:  zsh still mostly uses long integers internally,
+which are 32-bit on such systems.  However, the system part of this is
+expected to work, e.g. it should be possible to use redirection to write to
+files larger than 2GB.
--- Src/system.h.lfs	Mon Mar  1 10:46:51 1999
+++ Src/system.h	Wed May 12 09:54:57 1999
@@ -609,3 +609,12 @@
 #ifndef O_NOCTTY
 # define O_NOCTTY 0
 #endif
+
+#ifdef _LARGEFILE_SOURCE
+#ifdef HAVE_FSEEKO
+#define fseek fseeko
+#endif
+#ifdef HAVE_FTELLO
+#define ftell ftello
+#endif
+#endif
--- aczsh.m4.lfs	Thu May  6 16:58:50 1999
+++ aczsh.m4	Wed May 12 09:49:40 1999
@@ -38,15 +38,14 @@
 ac_shellvars='CPPFLAGS LDFLAGS LIBS'
 for ac_shellvar in $ac_shellvars; do
   case $ac_shellvar in
-  CPPFLAGS) ac_lfsvar=LFS_CFLAGS ac_lfs64var=LFS64_CFLAGS ;;
-  *) ac_lfsvar=LFS_$ac_shellvar ac_lfs64var=LFS64_$ac_shellvar ;;
+  CPPFLAGS) ac_lfsvar=LFS_CFLAGS ;;
+  *) ac_lfsvar=LFS_$ac_shellvar ;;
   esac
   eval test '"${'$ac_shellvar'+set}"' = set && ac_set=$ac_shellvar
   (getconf $ac_lfsvar) >/dev/null 2>&1 || { ac_result=no; break; }
   ac_getconf=`getconf $ac_lfsvar`
-  ac_getconf64=`getconf $ac_lfs64var`
-  ac_getconfs=$ac_getconfs$ac_getconf\ $ac_getconf64
-  eval ac_test_$ac_shellvar="\$ac_getconf\ \$ac_getconf64"
+  ac_getconfs=$ac_getconfs$ac_getconf
+  eval ac_test_$ac_shellvar="\$ac_getconf"
 done
 case "$ac_result$ac_getconfs" in
 yes) ac_result=no ;;
--- configure.in.lfs	Wed May 12 09:39:53 1999
+++ configure.in	Wed May 12 10:13:41 1999
@@ -97,6 +97,12 @@
   AC_DEFINE(ZSH_HASH_DEBUG)
 fi])
 
+dnl Do you want large file support, if available (mostly Solaris)?
+dnl Currently this is only partially implemented.
+undefine([lfs])dnl
+AC_ARG_ENABLE(lfs,
+[  --enable-lfs               turn on support for large files])
+
 dnl Pathnames for global zsh scripts
 undefine([zshenv])dnl
 AC_ARG_ENABLE(etcdir,
@@ -206,7 +212,9 @@
 
 dnl Check for large file support (Solaris).
 dnl This needs to be done early to get the stuff into the flags.
+if test x$enable_lfs != x; then
 zsh_LARGE_FILE_SUPPORT
+fi
 
 dnl if the user hasn't specified CFLAGS, then
 dnl   if compiler is gcc, then use -O2 and some warning flags
@@ -641,7 +649,8 @@
               sigblock sigsetmask sigrelse sighold killpg sigaction getrlimit \
               sigprocmask setuid seteuid setreuid setresuid setsid strerror \
               nis_list initgroups fchdir cap_get_proc readlink nice \
-	      getgrgid getgrnam getpwent getpwnam getpwuid setpgrp)
+	      getgrgid getgrnam getpwent getpwnam getpwuid setpgrp \
+	      fseeko ftello)
 
 dnl ---------------
 dnl CHECK FUNCTIONS
@@ -1253,6 +1262,7 @@
 host operating system     : ${host_cpu}-${host_vendor}-${host_os}
 source code location      : ${srcdir}
 compiler                  : ${CC}
+preprocessor flags        : ${CPPFLAGS}
 executable compiler flags : ${CFLAGS}"
 if test "$dynamic" = yes; then
   echo "\

-- 
Peter Stephenson <pws@ibmth.df.unipi.it>       Tel: +39 050 844536
WWW:  http://www.ifh.de/~pws/
Dipartimento di Fisica, Via Buonarroti 2, 56127 Pisa, Italy

RE: LFS notes RE: pws-18 is go

[Andrej Borsenkow]

Additional note: I think, until all issues are resolved, the configure
parameter (--enable-lfs would do) is needed. Currently I simply cannot
disable LFS even if I don't need it at all (I don't want to set CPPFLAGS to
some silly value; and in my case they are CFLAGS and not CPPFLAGS in the
first place).