Re: 4.2.0-pre-3; HP-UX

Re: 4.2.0-pre-3; HP-UX

[Paul Ackersviller]

On Fri, Mar 05, 2004 at 02:20:03PM +0000, Peter Stephenson wrote:
> I've uploaded 4.2.0-pre-3 to ftp.zsh.org.
> 
> Used autoconf 2.59 to try to improve matters on HPUX.

Indeed it has -- builds out of the box now, at least with HP's compiler.
I'll try gcc at some point after I actually get it installed.

The bad news is that some of the tests require tweaking of the modules
setup.  I'd swear I managed to dynamic load some pieces of pre-1 last
week, but I can't seem to do that on pre-3.  I'll try to figure out
what changed or if that was just my imagination.

Anyway, my current experience is that --disable-dynamic causes zsh/zpts
to get turned off, but after I add that as static all the tests pass up
to Y.  All three of the Y tests merely hang -- should I dig into this
more?  It sounds the same as what someone else reported recently.

Re: 4.2.0-pre-3; HP-UX

[Peter Stephenson]

Paul Ackersviller wrote:
> The bad news is that some of the tests require tweaking of the modules
> setup.  I'd swear I managed to dynamic load some pieces of pre-1 last
> week, but I can't seem to do that on pre-3.  I'll try to figure out
> what changed or if that was just my imagination.

It's possible it's broken; the HP-UX loading stuff always used to be
a bit different.  Nothing much we can do without an HP-UX system,
unfortunately.  I thought they'd made it more like standard dlopen.

> Anyway, my current experience is that --disable-dynamic causes zsh/zpts
> to get turned off, but after I add that as static all the tests pass up
> to Y.  All three of the Y tests merely hang -- should I dig into this
> more?  It sounds the same as what someone else reported recently.

This is almost certainly a problem with the pty module.  Again, only
digging around with that on HP-UX will make a difference.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: 4.2.0-pre-3; HP-UX

[Bart Schaefer]

On Mar 9, 12:17pm, Peter Stephenson wrote:
}
} It's possible it's broken; the HP-UX loading stuff always used to be
} a bit different.  Nothing much we can do without an HP-UX system,
} unfortunately.  I thought they'd made it more like standard dlopen.

By a strange coincidence, a publisher just sent me an unsolicited copy of
a HP-UX 11 internals book in the hope that I'd read it and post a review
on Amazon.  However, there are no index entries for anything resembling
"dlopen", "dynamic library", "shared library" or "shared object".

If you can think of something for me to look up, let me know.

Re: 4.2.0-pre-3; HP-UX

[Clint Adams]

> a HP-UX 11 internals book in the hope that I'd read it and post a review
> on Amazon.  However, there are no index entries for anything resembling
> "dlopen", "dynamic library", "shared library" or "shared object".
> 
> If you can think of something for me to look up, let me know.

shl_open, maybe.

Re: 4.2.0-pre-3; HP-UX

[Bart Schaefer]

On Mar 9, 11:00am, Clint Adams wrote:
}
} > a HP-UX 11 internals book in the hope that I'd read it and post a review
} > 
} > If you can think of something for me to look up, let me know.
} 
} shl_open, maybe.

Nope, the index goes straight from "Shift" to "SHMEM".

Re: 4.2.0-pre-3; HP-UX

[Clint Adams]

> Nope, the index goes straight from "Shift" to "SHMEM".

Hmm.  I'd log into one of the HP Test Drive HP-UX systems, except that I
seem to have forgotten my password.

They've got 2 PA-RISC's and 3 Itanium2's running HP-UX 11.

Re: 4.2.0-pre-3; HP-UX

[Paul Ackersviller]

On Tue, Mar 09, 2004 at 12:17:58PM +0000, Peter Stephenson wrote:
> Paul Ackersviller wrote:
> > Anyway, my current experience is that --disable-dynamic causes zsh/zpts
> > to get turned off, but after I add that as static all the tests pass up
> > to Y.  All three of the Y tests merely hang -- should I dig into this
> > more?  It sounds the same as what someone else reported recently.
> 
> This is almost certainly a problem with the pty module.  Again, only
> digging around with that on HP-UX will make a difference.

Yes, I understand that dynamic loading on HP needs some work, and I'd
be willing to help out on that as time allows.  But in the interests
of getting the current code configuring better out-of-the-box, might
there be an easy way to disable tests (automatically by default)
which depend on modules that aren't built?

Re: 4.2.0-pre-3; HP-UX

[Peter Stephenson]

Paul Ackersviller wrote:
> Yes, I understand that dynamic loading on HP needs some work, and I'd
> be willing to help out on that as time allows.  But in the interests
> of getting the current code configuring better out-of-the-box, might
> there be an easy way to disable tests (automatically by default)
> which depend on modules that aren't built?

Yes, that ought to be possible... test for the ability to zmodload -i
zsh/zpty and if it fails, skips the tests.  I've a vague feeling there's
already some mechanism for skipping tests, but I won't have time to look
at this for now.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: 4.2.0-pre-3; HP-UX

[Oliver Kiddle]

Clint Adams wrote:
> 
> Hmm.  I'd log into one of the HP Test Drive HP-UX systems, except that I
> seem to have forgotten my password.

I think they expire your account if you don't use it for a while. My old
one from when it was still Compaq has gone. Anyway, I signed up again.

On HP/UX, dlopen etc are there. This is HP/UX 11.23. The old shl_*
routines are also there. Of particular note is this passage at the end
of shl_load(3X):

      Future HP-UX environments may not support these routines and
      flags or may only support a subset of them.  Instead, they will
      use the SVR4 dynamic loading API.  Users are encouraged to
      migrate to the dl* family of dynamic linking routines.  See
      dlclose(3C), ...

HPUXDYNAMIC does not get defined so we are following that.

It builds okay but can't load all the modules - some such as zle and
files load, others like complete and math won't. There are also plenty
of warnings:

Warning 942: "module.c", line 429 # Types 'void *' and 'int' are not
    assignment-compatible.
            ret = dlopen(unmeta(buf), RTLD_LAZY | RTLD_GLOBAL);
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Warning 361: "init.c", line 1204 # Value-returning function might end without
    executing a return statement.
    zsh_main(int argc, char **argv)
    ^^^^^^^^

A few of these:
Warning 1042: "jobs.c", line 421 # Subscript out of range.
        if (sigtrapped[SIGCHLD] && job != thisjob)
                       ^^^^^^^
That also occurs for SIGWINCH, SIGPIPE and SIGALRM.

Then there are many many warnings like this:

This warning comes up a lot for many times in different places:
Warning 952: "compctl.c", line 608 # Conversion of 'const char *' to 'char *'
    loses 'const'.
                        *argv = "" - 1;
                        ^^^^^^

What is that code trying to do? Isn't that taking the address of a
constant string. Seems dodgy to me.

Then, this warning:
Warning 942: "zle_refresh.c", line 527 # Types 'unsigned char *' and 'char *'
    are not assignment-compatible.
            s = nbuf[tosln - 1];
            ^^^^^^^^^^^^^^
And, the same for zle_refresh.c:536,542,551 and zle_thingy.c:656

I also tried compiling on the Tru64 5.1 machine. It warns about many
unresolved things after linking each shared library. These look like the
things which will be resolved when they are loaded at runtime. We could
turn off the warnings with the option `-expect_unresolved *'

It stops at an error though:

cc -c -I.  -DHAVE_CONFIG_H -DMODULE -O  -o terminfo..o terminfo.c
cc: Error: /usr/include/curses.h, line 125: In this declaration, "DO" has no linkage and has a prior declaration in this scope at line number 212 in file ../../Src/zsh.h. (nolinkage)
                *DM, *DO, *ED, *EI, *K0, *K1, *K2, *K3, *K4, *K5, *K6,
---------------------^

This is an extern char declaration in curses.h.

Does anyone know of any similar places to HP's testdrive where I could
get a log on for IRIX 6 or AIX and where they include compilers and a
way to get the source code uploaded.

I don't have so much time at the moment either but if we're not in a
rush for 4.2, we should be able to get zsh working on the different
operating systems. Would it perhaps be worth putting out a 4.1.2 now to
get the code a bit more testing and allow it to get into Linux
distributions?

Oliver

Re: 4.2.0-pre-3; HP-UX

[Peter Stephenson]

Oliver Kiddle wrote:
> I think they expire your account if you don't use it for a while. My old
> one from when it was still Compaq has gone. Anyway, I signed up again.

Excellent.

> It builds okay but can't load all the modules - some such as zle and
> files load, others like complete and math won't.

But they compile OK?  Error messages?  Might simply be missing library
functions.

> There are also plenty of warnings:
> 
> Warning 942: "module.c", line 429 # Types 'void *' and 'int' are not
>     assignment-compatible.
>             ret = dlopen(unmeta(buf), RTLD_LAZY | RTLD_GLOBAL);
>             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Looks like a missing header.

> 
> Warning 361: "init.c", line 1204 # Value-returning function might end without
>     executing a return statement.
>     zsh_main(int argc, char **argv)
>     ^^^^^^^^

Ignorable.

> A few of these:
> Warning 1042: "jobs.c", line 421 # Subscript out of range.
>         if (sigtrapped[SIGCHLD] && job != thisjob)
>                        ^^^^^^^
> That also occurs for SIGWINCH, SIGPIPE and SIGALRM.

Suggests the awk script for signals screwed up.

> Then there are many many warnings like this:
> 
> This warning comes up a lot for many times in different places:
> Warning 952: "compctl.c", line 608 # Conversion of 'const char *' to 'char *'
>     loses 'const'.
>                         *argv = "" - 1;
>                         ^^^^^^
> 
> What is that code trying to do? Isn't that taking the address of a
> constant string. Seems dodgy to me.

Harmless.  We're not strict about `const'.  It would be nice to be,
but when you start going down that road you find you have to change
*everything*.  It's a hack so that argv[1] contains a NULL --- not a
particularly nice one.

> Then, this warning:
> Warning 942: "zle_refresh.c", line 527 # Types 'unsigned char *' and 'char *'
>     are not assignment-compatible.
>             s = nbuf[tosln - 1];
>             ^^^^^^^^^^^^^^
> And, the same for zle_refresh.c:536,542,551 and zle_thingy.c:656

This is mostly fixed up by using STOUC where necessary, but more
consistency of pointers would be better.

> I also tried compiling on the Tru64 5.1 machine. It warns about many
> unresolved things after linking each shared library. These look like the
> things which will be resolved when they are loaded at runtime. We could
> turn off the warnings with the option `-expect_unresolved *'
> 
> It stops at an error though:
> 
> cc -c -I.  -DHAVE_CONFIG_H -DMODULE -O  -o terminfo..o terminfo.c
> cc: Error: /usr/include/curses.h, line 125: In this declaration, "DO" has no 
> linkage and has a prior declaration in this scope at line number 212 in file 
> ../../Src/zsh.h. (nolinkage)
>                 *DM, *DO, *ED, *EI, *K0, *K1, *K2, *K3, *K4, *K5, *K6,
> ---------------------^

It's probably defined somewhere else.  We could rename those arguments.

> I don't have so much time at the moment either but if we're not in a
> rush for 4.2, we should be able to get zsh working on the different
> operating systems. Would it perhaps be worth putting out a 4.1.2 now to
> get the code a bit more testing and allow it to get into Linux
> distributions?

Frankly, I'm hanged if I personally am going to go round changing all
the version numbers again, but I don't mind releasing 4.2.0 and warning
users of HP-UX and AIX they may want to wait.  Nobody expects versions
ending in `0' to work anyway.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: 4.2.0-pre-3; HP-UX

[Oliver Kiddle]

Peter wrote:
> 
> But they compile OK?  Error messages?  Might simply be missing library
> functions.

Yes, compiles Ok with no error messages. make check just hangs and
without being able to load the completion library, it doesn't seem
particularly usable. When I get some time, I'll investigate further.

> > This warning comes up a lot for many times in different places:
> > Warning 952: "compctl.c", line 608 # Conversion of 'const char *' to 'char *'
> >     loses 'const'.
> >                         *argv = "" - 1;
> >                         ^^^^^^
> > 
> > What is that code trying to do? Isn't that taking the address of a
> > constant string. Seems dodgy to me.
> 
> Harmless.  We're not strict about `const'.  It would be nice to be,
> but when you start going down that road you find you have to change
> *everything*.  It's a hack so that argv[1] contains a NULL --- not a
> particularly nice one.

The warnings are spewed out for this on Tru64 too. Is there not
something we could change it to so that it is both more readable and
doesn't create warnings. Perhaps using a macro to wrap around any nasty
hacks or by having a single null-string variable. I'm happy to go
through all the instances changing them.

> >                 *DM, *DO, *ED, *EI, *K0, *K1, *K2, *K3, *K4, *K5, *K6,
> > ---------------------^
> 
> It's probably defined somewhere else.  We could rename those arguments.

Line 212 in zsh.h. Patch below renames our DO to DOLOOP which solves
the problem. It then builds and all tests pass (on Tru64 5.1). They
have a Tru64 4.0 machine too so I can also try that. And they have gcc
in /usr/local so that can also be tried.

> Frankly, I'm hanged if I personally am going to go round changing all
> the version numbers again, but I don't mind releasing 4.2.0 and warning
> users of HP-UX and AIX they may want to wait.  Nobody expects versions
> ending in `0' to work anyway.

Sounds like a fair enough plan.

Oliver

Index: Src/hashtable.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/hashtable.c,v
retrieving revision 1.16
diff -u -r1.16 hashtable.c
--- Src/hashtable.c	29 Oct 2003 19:17:30 -0000	1.16
+++ Src/hashtable.c	10 Mar 2004 17:04:45 -0000
@@ -925,7 +925,7 @@
     {NULL, "}", 0, OUTBRACE},
     {NULL, "case", 0, CASE},
     {NULL, "coproc", 0, COPROC},
-    {NULL, "do", 0, DO},
+    {NULL, "do", 0, DOLOOP},
     {NULL, "done", 0, DONE},
     {NULL, "elif", 0, ELIF},
     {NULL, "else", 0, ELSE},
Index: Src/lex.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/lex.c,v
retrieving revision 1.27
diff -u -r1.27 lex.c
--- Src/lex.c	3 Sep 2003 10:15:36 -0000	1.27
+++ Src/lex.c	10 Mar 2004 17:04:45 -0000
@@ -393,7 +393,7 @@
     case BAR:
     case BARAMP:
     case INOUTPAR:
-    case DO:
+    case DOLOOP:
     case THEN:
     case ELIF:
     case ELSE:
Index: Src/parse.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/parse.c,v
retrieving revision 1.41
diff -u -r1.41 parse.c
--- Src/parse.c	29 Oct 2003 19:17:30 -0000	1.41
+++ Src/parse.c	10 Mar 2004 17:04:45 -0000
@@ -978,7 +978,7 @@
     incmdpos = 1;
     while (tok == SEPER)
 	yylex();
-    if (tok == DO) {
+    if (tok == DOLOOP) {
 	yylex();
 	par_save_list(complex);
 	if (tok != DONE)
@@ -1260,7 +1260,7 @@
     incmdpos = 1;
     while (tok == SEPER)
 	yylex();
-    if (tok == DO) {
+    if (tok == DOLOOP) {
 	yylex();
 	par_save_list(complex);
 	if (tok != DONE)
@@ -1304,7 +1304,7 @@
     yylex();
     while (tok == SEPER)
 	yylex();
-    if (tok == DO) {
+    if (tok == DOLOOP) {
 	yylex();
 	par_save_list(complex);
 	if (tok != DONE)
Index: Src/zsh.h
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/zsh.h,v
retrieving revision 1.53
diff -u -r1.53 zsh.h
--- Src/zsh.h	6 Mar 2004 00:23:03 -0000	1.53
+++ Src/zsh.h	10 Mar 2004 17:04:45 -0000
@@ -209,7 +209,7 @@
     OUTBRACE,   /* }         */
     CASE,	/* case      */
     COPROC,	/* coproc    */
-    DO,		/* do        */
+    DOLOOP,	/* do        */
     DONE,	/* done      */ /* 45 */
     ELIF,	/* elif      */
     ELSE,	/* else      */
Index: Src/Zle/zle_tricky.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Zle/zle_tricky.c,v
retrieving revision 1.42
diff -u -r1.42 zle_tricky.c
--- Src/Zle/zle_tricky.c	8 Mar 2004 11:44:14 -0000	1.42
+++ Src/Zle/zle_tricky.c	10 Mar 2004 17:04:45 -0000
@@ -1105,7 +1105,7 @@
 	/* We reached the end. */
 	if (tok == ENDINPUT)
 	    break;
-	if ((ins && (tok == DO || tok == SEPER)) ||
+	if ((ins && (tok == DOLOOP || tok == SEPER)) ||
 	    (ins == 2 && i == 2) || (ins == 3 && i == 3) ||
 	    tok == BAR    || tok == AMPER     ||
 	    tok == BARAMP || tok == AMPERBANG ||

Re: 4.2.0-pre-3; HP-UX

[Oliver Kiddle]

Peter wrote:
> > Warning 942: "module.c", line 429 # Types 'void *' and 'int' are not
> >     assignment-compatible.
> >             ret = dlopen(unmeta(buf), RTLD_LAZY | RTLD_GLOBAL);
> >             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> 
> Looks like a missing header.

This led me to the problem. It was including dl.h (which contains all
the shl_* functions) instead of dlfcn.h.

With the following patch, the modules now all load and seem to work.

It still isn't 100% though judging by make check.
./A01grammar.ztst: starting.
sh[2]: 14893 Bus error(coredump)
and A04 hangs.

but I'm off home now so that (and the signals stuff) will have to wait.

Oliver

Index: module.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/module.c,v
retrieving revision 1.12
diff -u -r1.12 module.c
--- module.c    29 Oct 2003 19:17:30 -0000      1.12
+++ module.c    10 Mar 2004 17:55:12 -0000
@@ -349,7 +349,7 @@
 #else

 #ifdef HAVE_DLFCN_H
-# if defined(HAVE_DL_H) && defined(__hpux)
+# if defined(HAVE_DL_H) && defined(HPUXDYNAMIC)
 #  include <dl.h>
 # else
 #  include <dlfcn.h>