Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Peter Stephenson]

On Thu, 13 Dec 2007 16:09:15 +0100
Pea <zsh@raveland.org> wrote:
> The new module zsh/curses fails on OpenBSD (i386 -current).
> 
> gcc -c -I.  -DHAVE_CONFIG_H -DMODULE -Wall -Wmissing-prototypes -O2
> -fPIC -o curses..o curses.c 
> In file included from /usr/include/curses.h:14, 
> 	from curses.c:46:
> /usr/include/ncurses.h:251: error: conflicting types for `wchar_t'
> /usr/include/stddef.h:54: error: previous declaration of `wchar_t'
...
> Some hits here: http://www.openbsd.org/porting.html#Other

Thanks for letting us know.

This suggests the right library to use on OpenBSD is curses, right?
In that case I hope the following patch works.  We won't be able to test
this...  it would be great if you were able to ASAP.  I can send the
corresponding configure if you need it.

You should also find it compiles with --with-term-lib=curses.

Thanks

Index: configure.ac
===================================================================
RCS file: /cvsroot/zsh/zsh/configure.ac,v
retrieving revision 1.86
diff -u -r1.86 configure.ac
--- configure.ac	7 Dec 2007 11:34:48 -0000	1.86
+++ configure.ac	13 Dec 2007 18:06:15 -0000
@@ -663,6 +663,9 @@
   termcap_curses_order="$ncursesw_test tinfo termcap $ncurses_test curses"
 fi],
 [case "$host_os" in
+  *openbsd*)
+   # Apparently curses is the one to use here.
+   termcap_curses_order="curses $ncursesw_test tinfo termcap $ncurses_test" ;;
   hpux10.*|hpux11.*|solaris*)
    termcap_curses_order="Hcurses $ncursesw_test $ncurses_test curses termcap" ;;
   *)



-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Pea]

Le Thu, 13 Dec 2007 18:10:50 +0000,
Peter Stephenson <pws@csr.com> a écrit :

> This suggests the right library to use on OpenBSD is curses, right?
> In that case I hope the following patch works.  We won't be able to
> test this...  it would be great if you were able to ASAP.  I can send
> the corresponding configure if you need it.
> 
> You should also find it compiles with --with-term-lib=curses.
> 
[...]
I tried your patch, dont it doesn't work.

Case 1 give me: 
./configure --enable-zsh-mem --enable-zsh-secure-free
--enable-maildir-support --with-tcsetpgrp --enable-multibyte

zsh configuration
-----------------
zsh version               : 4.3.4-dev-4
host operating system     : i386-unknown-openbsd4.2
source code location      : .
compiler                  : gcc
preprocessor flags        : 
executable compiler flags :  -Wall -Wmissing-prototypes -O2
module compiler flags     :  -Wall -Wmissing-prototypes -O2 -fPIC
executable linker flags   :   -s -Wl,-E
module linker flags       :   -s -shared -fPIC
library flags             : -ltermcap -lm  -lc
installation basename     : zsh
binary install path       : /usr/local/bin
man page install path     : ${prefix}/share/man
info install path         : ${prefix}/share/info
functions install path    : ${prefix}/share/zsh/4.3.4-dev-4/functions

Next i do : gmake

[...]

rm -f clone.so
gcc  -s -shared -fPIC -o clone.so   clone..o    -ltermcap -lm  -lc 
nawk -f ./curses_keys.awk /usr/include/ncurses.h /dev/null
>curses_keys.h gcc -c -I.  -DHAVE_CONFIG_H -DMODULE -Wall
>-Wmissing-prototypes -O2 -fPIC -o curses..o curses.c
In file included from /usr/include/curses.h:14,
                 from curses.c:46:
/usr/include/ncurses.h:251: error: conflicting types for `wchar_t'
/usr/include/stddef.h:54: error: previous declaration of `wchar_t'
/usr/include/ncurses.h:254: error: conflicting types for `wint_t'
/usr/include/stddef.h:59: error: previous declaration of `wint_t'
gmake[3]: *** [curses..o] Error 1
gmake[3]: Leaving directory
`/usr/ports/shells/zsh_pea/w-zsh-4.3.4-dev-4/zsh-4.3.4-dev-4/Src/Modules'
gmake[2]: *** [modules] Error 1 gmake[2]: Leaving directory
`/usr/ports/shells/zsh_pea/w-zsh-4.3.4-dev-4/zsh-4.3.4-dev-4/Src'
gmake[1]: *** [modules] Error 2 gmake[1]: Leaving directory
`/usr/ports/shells/zsh_pea/w-zsh-4.3.4-dev-4/zsh-4.3.4-dev-4/Src'
gmake: *** [all] Error 1




Case 2 give me:
./configure --enable-zsh-mem --enable-zsh-secure-free
--enable-maildir-support --with-tcsetpgrp --enable-multibyte
--with-term-lib=curses

zsh configuration
zsh version               : 4.3.4-dev-4
host operating system     : i386-unknown-openbsd4.2
source code location      : .
compiler                  : gcc
preprocessor flags        : 
executable compiler flags :  -Wall -Wmissing-prototypes -O2
module compiler flags     :  -Wall -Wmissing-prototypes -O2 -fPIC
executable linker flags   :   -s -Wl,-E
module linker flags       :   -s -shared -fPIC
library flags             : -lcurses -lm  -lc
installation basename     : zsh
binary install path       : /usr/local/bin
man page install path     : ${prefix}/share/man
info install path         : ${prefix}/share/info
functions install path    : ${prefix}/share/zsh/4.3.4-dev-4/functions

gmake

[...]

rm -f clone.so
gcc  -s -shared -fPIC -o clone.so   clone..o    -lcurses -lm  -lc 
gcc -c -I.  -DHAVE_CONFIG_H -DMODULE -Wall -Wmissing-prototypes -O2
-fPIC -o curses..o curses.c In file included
from /usr/include/curses.h:14, from curses.c:46:
/usr/include/ncurses.h:251: error: conflicting types for `wchar_t'
/usr/include/stddef.h:54: error: previous declaration of `wchar_t'
/usr/include/ncurses.h:254: error: conflicting types for `wint_t'
/usr/include/stddef.h:59: error: previous declaration of `wint_t'
gmake[3]: *** [curses..o] Error 1
gmake[3]: Leaving directory
`/usr/ports/shells/zsh_pea/w-zsh-4.3.4-dev-4/zsh-4.3.4-dev-4/Src/Modules'
gmake[2]: *** [modules] Error 1 gmake[2]: Leaving directory
`/usr/ports/shells/zsh_pea/w-zsh-4.3.4-dev-4/zsh-4.3.4-dev-4/Src'
gmake[1]: *** [modules] Error 2 gmake[1]: Leaving directory
`/usr/ports/shells/zsh_pea/w-zsh-4.3.4-dev-4/zsh-4.3.4-dev-4/Src'
gmake: *** [all] Error 1


Just for informations, /usr/include/curses.h contains:

/*      $OpenBSD: curses.h,v 1.3 2003/06/17 21:33:53 millert Exp $
*/

/*
 * Placed in the public domain by Todd C. Miller
<Todd.Miller@courtesan.com>
 * on June 17, 2003.
 */

#ifndef _CURSES_H_
#define _CURSES_H_

#ifdef _USE_OLD_CURSES_
#include <ocurses.h>
#else
#include <ncurses.h>
#endif

#endif /* _CURSES_H_ */

Hope it can help..

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Peter Stephenson]

Pea wrote:
> 
> /*      $OpenBSD: curses.h,v 1.3 2003/06/17 21:33:53 millert Exp $
> */
> 
> /*
>  * Placed in the public domain by Todd C. Miller
> <Todd.Miller@courtesan.com>
>  * on June 17, 2003.
>  */
> 
> #ifndef _CURSES_H_
> #define _CURSES_H_
> 
> #ifdef _USE_OLD_CURSES_
> #include <ocurses.h>
> #else
> #include <ncurses.h>
> #endif
> 
> #endif /* _CURSES_H_ */
> 
> Hope it can help..

Thanks for the additional info...

Gah.

I'm now pretty much stuck.  We seem to be forced into including
ncurses.h to get curses functionality, since curses.h includes it, but
that's got a clashing definition with a system header.  I don't really
understand how this can ever have worked.

The only alternative seems to be to define _USE_OLD_CURSES_, but why
would we want to do that?

Is there any indication of why the headers are allowing wchar_t to be
defined twice?  Is there are anything around lines 54/59 of stddef.h
that would indicate what's going on?

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Peter Stephenson]

On Fri, 14 Dec 2007 13:30:35 +0100
Pea <zsh@raveland.org> wrote:
> > Is there any indication of why the headers are allowing wchar_t to be
> > defined twice?  Is there are anything around lines 54/59 of stddef.h
> > that would indicate what's going on?
> > 
> 
> I think I found the problem.
> In ncurses.h (line 94):
> 
> #ifdef _XOPEN_SOURCE_EXTENDED
> #include <stddef.h>     /* we want wchar_t */
> #endif /* _XOPEN_SOURCE_EXTENDED */
> 
> and in Src/Modules/src.c we have #define _XOPEN_SOURCE_EXTENDED 1
> If i remove this line, compilation doesn't fail.

Thanks for investigating.  OK, so that has to come out for openbsd.  I hope
the following patch does the trick:  it's a bit complicated in order to
ensure we get the definition before any system header files.

Clint, can you remember why we needed _XOPEN_SOURCE_EXTENDED for curses.c?
We should at least document it.  This seems to have been there since the
start.

> I added --with-term-lib=curses to my configure and i have not applied
> your patch.

It should now be OK not to use --with-term-lib (and to ignore the previous
patch): it looks like this is ultimately picking up the same files anyway,
and I'd like to be sure it compiles without special options.

> zmodload zsh/curses works. Let me play with this and i will say you if
> i have some pbs.

Thank you.

Index: configure.ac
===================================================================
RCS file: /cvsroot/zsh/zsh/configure.ac,v
retrieving revision 1.86
diff -u -r1.86 configure.ac
--- configure.ac	7 Dec 2007 11:34:48 -0000	1.86
+++ configure.ac	14 Dec 2007 12:53:37 -0000
@@ -669,6 +669,22 @@
    termcap_curses_order="$ncursesw_test tinfo termcap $ncurses_test curses" ;;
 esac])dnl
 
+AH_TEMPLATE([ZSH_CURSES_NEEDS_XOPEN],
+[Define if the curses libraries need _XOPEN_SOURCE_EXTENDED defined])
+AC_CACHE_CHECK(if the curses library needs _XOPEN_SOURCE_EXTENDED,
+zsh_cv_curses_needs_xopen,
+[case "$host_os" in
+  *openbsd*)
+  zsh_cv_curses_needs_xopen=no
+  ;;
+  *)
+  zsh_cv_curses_needs_xopen=yes
+  ;;
+esac])
+if test x$zsh_cv_curses_needs_xopen = xyes; then
+  AC_DEFINE(ZSH_CURSES_NEEDS_XOPEN)
+fi
+
 AH_TEMPLATE([HAVE_BOOLCODES],
 [Define if you have the termcap boolcodes symbol.])
 AH_TEMPLATE([HAVE_NUMCODES],
Index: Src/system.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/system.h,v
retrieving revision 1.48
diff -u -r1.48 system.h
--- Src/system.h	1 Nov 2007 17:57:57 -0000	1.48
+++ Src/system.h	14 Dec 2007 12:53:37 -0000
@@ -52,6 +52,10 @@
 # undef HAVE_SYS_UTSNAME_H
 #endif
 
+#if defined(ZSH_CURSES_SOURCE) && defined(ZSH_CURSES_NEEDS_XOPEN)
+#define _XOPEN_SOURCE_EXTENDED 1
+#endif
+
 /*
  * Solaris by default zeroes all elements of the tm structure in
  * strptime().  Unfortunately that gives us no way of telling whether
Index: Src/Modules/curses.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/curses.c,v
retrieving revision 1.42
diff -u -r1.42 curses.c
--- Src/Modules/curses.c	21 Nov 2007 09:53:49 -0000	1.42
+++ Src/Modules/curses.c	14 Dec 2007 12:53:38 -0000
@@ -27,7 +27,7 @@
  *
  */
 
-#define _XOPEN_SOURCE_EXTENDED 1
+#define ZSH_CURSES_SOURCE 1
 
 #include "curses.mdh"
 #include "curses.pro"

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Peter Stephenson]

Pea wrote:
> > Thanks for investigating.  OK, so that has to come out for openbsd.
> > I hope the following patch does the trick:  it's a bit complicated in
> > order to ensure we get the definition before any system header files.
> 
> Thank you for your patch. It works like a charm !
> But if you don't to complicate your code, i can do a patch only for
> OpenBSD (just to remove the line #define _XOPEN_SOURCE_EXTENDED 1) (i
> am the maintainer of zsh port). Just tell me..

I'm perfectly happy to have it in the main shell; by our standards this
isn't even particularly complicated.

> I test on my two machines (@i386 and @amd64): compilation OK. But now,
> one regression test fail:
> 
> ./A03quoting.ztst: starting.
> Test ./A03quoting.ztst failed: bad status 1, expected 0 from:
>   print '<\u0041>'
>   printf '%s\n' $'<\u0042>'
>   print '<\u0043>'
>   printf '%s\n' $'<\u0044>'
> Error output:
> (eval):1: cannot do charset conversion
> Was testing: \u in both print and printf
> ./A03quoting.ztst: test failed.

Urk...  that means it hasn't found a suitable library for character set
conversion.  This isn't necessarily wrong if you actually don't have
those facilities.  The two main possibilities are

- environment supports ISO 10646 (plus wchar_t plus wctomb()).
- iconv is present (plus NLS).

See the code around line 4851 of Src/utils.c; you should be able to work
out which preprocessor tests are failing.

It's possible HAVE_WCHAR_H isn't defined if you don't have <wchar.h>
(I'm suggesting this because of the previous difficulties over finding
wchar_t).  It's possible then that wctomb() is actually somewhere else
and we don't need wchar.h at all.  Otherwise, we may have to make do
with what we've got, which compiles and is basically functioning.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR PLC, Churchill House, Cambridge Business Park, Cowley Road
Cambridge, CB4 0WZ, UK                          Tel: +44 (0)1223 692070

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Clint Adams]

On Fri, Dec 14, 2007 at 01:04:02PM +0000, Peter Stephenson wrote:
> Clint, can you remember why we needed _XOPEN_SOURCE_EXTENDED for curses.c?
> We should at least document it.  This seems to have been there since the
> start.

I believe it was to get wide character support out of ncurses.h .
There may have been something about attributes too, but my memory
is a bit fuzzy right now.

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Peter Stephenson]

On Tue, 11 Dec 2007 18:44:41 +0200
Ismail Dönmez <ismail@pardus.org.tr> wrote:
> Tuesday 11 December 2007 18:39:12 tarihinde şunları yazmıştınız:

(That can't be the Turkish for "Bart said", can it?  Just wondered...)

> > It's possible to mount an ext3 filesystem in a mode that does not do
> > an access time updates.  This is supposed to make the filesystem
> > respond much more quickly.
> 
> Mounted with noatime indeed! Thats the problem then.

For now, we'd better indicate the problem.

Index: Test/C02cond.ztst
===================================================================
RCS file: /cvsroot/zsh/zsh/Test/C02cond.ztst,v
retrieving revision 1.18
diff -u -r1.18 C02cond.ztst
--- Test/C02cond.ztst	11 Dec 2007 15:24:47 -0000	1.18
+++ Test/C02cond.ztst	13 Dec 2007 21:09:06 -0000
@@ -142,7 +142,9 @@
 F:This test can fail on NFS-mounted filesystems as the access and
 F:modification times are not updated separately.  The test will fail
 F:on HFS+ (Apple Mac OS X default) filesystems because access times
-F:are not recorded. This does not indicate a problem in the shell.
+F:are not recorded.  Also, Linux ext3 filesystems may be mounted
+F:with the noatime option which does not update access times.
+F:Failures in these cases do not indicate a problem in the shell.
 
   [[ newnewnew -nt zerolength && ! (unmodified -nt zerolength) ]]
 0:-nt cond


-- 
Peter Stephenson <p.w.stephenson@ntlworld.com>
Web page now at http://homepage.ntlworld.com/p.w.stephenson/

Re: 4.3.4-dev-4 and 4.2.6-dev-2 available

[Ismail Dönmez]

Thursday 13 December 2007 23:08:12 tarihinde şunları yazmıştınız:
> On Tue, 11 Dec 2007 18:44:41 +0200
>
> Ismail Dönmez <ismail@pardus.org.tr> wrote:
> > Tuesday 11 December 2007 18:39:12 tarihinde şunları yazmıştınız:
>
> (That can't be the Turkish for "Bart said", can it?  Just wondered...)

That means "On Tuesday 11 December 2007 18:39:12 you wrote this".

Regards,
ismail

-- 
Never learn by your mistakes, if you do you may never dare to try again.

History file locking (was Re: 4.3.4-dev-4 and 4.2.6-dev-2 available)

[Peter Stephenson]

(Move this to zsh-workers since this is now development only.)

Vincent Lefevre wrote:
> I've undone Wayne's patch zsh-users/8494 that didn't seem to work
> anyway and retried my patch, and while I got history corruption
> with the unpatched version a few minutes before (as usual), I can't
> reproduce the corruption with my patched version.

We can probably add your stuff as an option; I'm a bit worried about
forcing fcntl() locking on everybody willy nilly.  This probably needs
to wait till after 4.3.5 otherwise that will never get out at this rate.

> However, I get the following error when I quit the shell:
> 
> zsh: failed to write history file /home/vlefevre/.zhistory: bad file descript
> or
> 
> but nothing is missing in the history!

It's possible there's an overagressive test for a write failure; I added
some more after reports of problems.  You might be able to track down
where this is coming from fairly easily.

pws

Re: History file locking (was Re: 4.3.4-dev-4 and 4.2.6-dev-2 available)

[Vincent Lefevre]

On 2007-12-14 17:45:47 +0000, Peter Stephenson wrote:
> We can probably add your stuff as an option; I'm a bit worried about
> forcing fcntl() locking on everybody willy nilly.

A runtime option would be the best solution. On the NFS server here,
fcntl locking sometimes no longer works (probably because the lockd
daemon has crashed), though in general, the NFS server completely
crashes a few hours later. A runtime option would allow ones to work
in a degraded mode.

>  This probably needs to wait till after 4.3.5 otherwise that will
> never get out at this rate.

OK.

> It's possible there's an overagressive test for a write failure; I added
> some more after reports of problems.  You might be able to track down
> where this is coming from fairly easily.

I'll try to look at it.

-- 
Vincent Lefèvre <vincent@vinc17.org> - Web: <http://www.vinc17.org/>
100% accessible validated (X)HTML - Blog: <http://www.vinc17.org/blog/>
Work: CR INRIA - computer arithmetic / Arenaire project (LIP, ENS-Lyon)