zpty on HP/UX

zpty on HP/UX

[Oliver Kiddle]

I've got zpty to work on HP/UX. I looked at pts(7) and tried the example
in there.

Basically it needs the ioctl streams stuff but not the ttcompat bit. See
the patch below to see what needed removing. Note that this isn't a patch
I intend to apply but merely in indication of the changes. We need to
sort out the #if stuff.

We can put the ttcompat ioctl inside a #ifndef __hpux but the __SVR4
part is going to cause us problems. What systems was that added to cover?

Oliver

Index: zpty.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/zpty.c,v
retrieving revision 1.30
diff -u -r1.30 zpty.c
--- zpty.c	26 Feb 2004 20:48:09 -0000	1.30
+++ zpty.c	15 Mar 2004 16:39:04 -0000
@@ -165,9 +165,7 @@
 {
     static char *name;
     static int mfd, sfd;
-#if defined(I_FIND) && defined(I_PUSH) && defined(__SVR4)
     int ret;
-#endif
 
     if (master) {
 	if ((mfd = open("/dev/ptmx", O_RDWR|O_NOCTTY)) < 0)
@@ -190,7 +188,6 @@
 	close(mfd);
 	return 1;
     }
-#if defined(I_FIND) && defined(I_PUSH) && defined(__SVR4)
     /*
      * Use if STREAMS is available.  The test is probably OK,
      * but we could use e.g. the sys/stropts.h test.
@@ -207,13 +204,6 @@
 	   close(sfd);
 	   return 1;
        }
-    if ((ret = ioctl(sfd, I_FIND, "ttcompat")) != 1)
-       if (ret == -1 || ioctl(sfd, I_PUSH, "ttcompat") == -1) {
-	   close(mfd);
-	   close(sfd);
-	   return 1;
-       }
-#endif
 
     *retfd = sfd;
 

Re: zpty on HP/UX

[Peter Stephenson]

Oliver Kiddle wrote:
> I've got zpty to work on HP/UX. I looked at pts(7) and tried the example
> in there.
> 
> Basically it needs the ioctl streams stuff but not the ttcompat bit. See
> the patch below to see what needed removing. Note that this isn't a patch
> I intend to apply but merely in indication of the changes. We need to
> sort out the #if stuff.
> 
> We can put the ttcompat ioctl inside a #ifndef __hpux but the __SVR4
> part is going to cause us problems. What systems was that added to cover?

The bits marked #if ... defined(__SVR4) are really there for Solaris.  I
asked if anyone knew other systems which needed the STREAMS stuff but
apparently no one did.  This was my best guess as to systems which might
need it.  The additional test turned out to be necessary because
allowing those ioctl's to run on some Linux systems (where STREAMS was
present) screwed things up.  Linux doesn't define __SVR4.

Unless someone turns out to be a secret STREAMS expert we are stuck with
doing it machine by machine, i.e. adding __hpux where necessary.
I don't see a good reason for changing from __SVR4 for the other case.

(Does STREAMS still have a use apart from screwing things up if it isn't
handled as a special case?)

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

PATCH: zpty on HP/UX

[Peter Stephenson]

> Unless someone turns out to be a secret STREAMS expert we are stuck with
> doing it machine by machine, i.e. adding __hpux where necessary.
> I don't see a good reason for changing from __SVR4 for the other case.

Here's a patch which doesn't screw up Solaris, at least.

Index: Src/Modules/zpty.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/zpty.c,v
retrieving revision 1.30
diff -u -r1.30 zpty.c
--- Src/Modules/zpty.c	26 Feb 2004 20:48:09 -0000	1.30
+++ Src/Modules/zpty.c	16 Mar 2004 14:47:05 -0000
@@ -160,12 +160,31 @@
 #include <sys/stropts.h>
 #endif
 
+#if defined(I_FIND) && defined(I_PUSH)
+/*
+ * These tests are ad hoc.  Unfortunately if you get the wrong ioctl,
+ * STREAMS simply hangs up, so there's no obvious way of doing this
+ * more systematically.
+ *
+ * Apparently Solaris needs all three ioctls, but HP-UX doesn't need
+ * ttcompat.  The Solaris definition has been extended to all __SVR4
+ * as a guess; I have no idea if this is right.
+ */
+#ifdef __SVR4
+#define USE_STREAMS_IOCTLS
+#define USE_STREAMS_TTCOMPAT
+#endif
+#ifdef __hpux
+#define USE_STREAMS_IOCTLS
+#endif
+#endif
+
 static int
 get_pty(int master, int *retfd)
 {
     static char *name;
     static int mfd, sfd;
-#if defined(I_FIND) && defined(I_PUSH) && defined(__SVR4)
+#ifdef USE_STREAMS_IOCTLS
     int ret;
 #endif
 
@@ -190,11 +209,7 @@
 	close(mfd);
 	return 1;
     }
-#if defined(I_FIND) && defined(I_PUSH) && defined(__SVR4)
-    /*
-     * Use if STREAMS is available.  The test is probably OK,
-     * but we could use e.g. the sys/stropts.h test.
-     */
+#ifdef USE_STREAMS_IOCTLS
     if ((ret = ioctl(sfd, I_FIND, "ptem")) != 1)
        if (ret == -1 || ioctl(sfd, I_PUSH, "ptem") == -1) {
 	   close(mfd);
@@ -207,6 +222,7 @@
 	   close(sfd);
 	   return 1;
        }
+#ifdef USE_STREAMS_TTCOMPAT
     if ((ret = ioctl(sfd, I_FIND, "ttcompat")) != 1)
        if (ret == -1 || ioctl(sfd, I_PUSH, "ttcompat") == -1) {
 	   close(mfd);
@@ -214,6 +230,7 @@
 	   return 1;
        }
 #endif
+#endif
 
     *retfd = sfd;
 

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: PATCH: zpty on HP/UX

[Oliver Kiddle]

Peter wrote:
> > Unless someone turns out to be a secret STREAMS expert we are stuck with
> > doing it machine by machine, i.e. adding __hpux where necessary.
> > I don't see a good reason for changing from __SVR4 for the other case.
> 
> Here's a patch which doesn't screw up Solaris, at least.

I've tested that on HP/UX and it works fine.

That leaves Mac OS X and IRIX as the known operating systems where zpty
doesn't work. I may try to have a look at Mac OS later this week. IRIX
has its own _getpty interface.

I'm away next week so if you're aiming for a Friday release, that would
suit me.

Oliver

Re: PATCH: zpty on HP/UX

[mneptok]

On Mar 16, 2004, at 11:47, Oliver Kiddle wrote:

> I've tested that on HP/UX and it works fine.
>
> That leaves Mac OS X and IRIX as the known operating systems where zpty
> doesn't work. I may try to have a look at Mac OS later this week. IRIX
> has its own _getpty interface.

Oli,

The version of zsh included with MacOS X.3.x seems to have a functional 
zpty.

[mneptok@anubis] mneptok :: zpty -L
zpty ls -l
zpty date date
[mneptok@anubis] mneptok ::

If you would like, I can try to track down who at Apple built the 
included zsh binary, and if they would be willing to hand their changes 
to you.

./k

kurt von finck
-- 
http://www.mneptok.com
mneptok@mneptok.com
--

I'm not concerned that all Hell will break loose, but rather that a 
PART of Hell will break loose. It'll be much harder to detect.

Re: PATCH: zpty on HP/UX

[Oliver Kiddle]

mneptok wrote:
> 
> The version of zsh included with MacOS X.3.x seems to have a functional 
> zpty.
> 
> [mneptok@anubis] mneptok :: zpty -L
> zpty ls -l
> zpty date date
> [mneptok@anubis] mneptok ::

But can you read from the commands. i.e. if you do:
  zpty date date
  zpty -r date
will the second command print the date or will it exit with a status of
1. In my tests on the Sourceforge machines, I can get as far as adding
a few commands but I can't read from them.

If reading does work, what version of zsh is it? It is just possible we
had it working before. If we didn't, tracking down the person at Apple
and getting the fixes would definitely be useful, thanks.

If you build zsh on MacOS X.3, does it build the dynamic modules
without your having to install the dlcompat library? The zshs included
with 10.1 and 10.2 have dynamic modules disabled so there is no zpty.
It'd be interesting to know if Apple have added a dlopen interface for
compatibility.

Oliver

Re: PATCH: zpty on HP/UX (and Mac OS X 10.3)

[James Devenish]

In message <24420.1079518350@trentino.logica.co.uk>
on Wed, Mar 17, 2004 at 11:12:30AM +0100, Oliver Kiddle wrote:
> But can you read from the commands. i.e. if you do:
>   zpty date date
>   zpty -r date
> will the second command print the date or will it exit with a status of
> 1.

% uname -a
Darwin ... 7.3.0 Darwin Kernel Version 7.3.0: Fri Mar 5 14:22:55 PST 2004; root:xnu/xnu-517.3.15.obj~4/RELEASE_PPC  Power Macintosh powerpc
% sw_vers
ProductName:    Mac OS X
ProductVersion: 10.3.3
BuildVersion:   7F44
% which zsh
/bin/zsh
% zpty -L
zsh: command not found: zpty
% zmodload zsh/zpty
% zpty date date
% zpty -r date
Wed Mar 17 18:17:21 WST 2004
%

> If reading does work, what version of zsh is it?

% echo $ZSH_VERSION
4.1.1

I wonder if this is the version that is available as tarball/CVS from
Apple's website?

> If you build zsh on MacOS X.3, does it build the dynamic modules
> without your having to install the dlcompat library?

% locate dlcompat | fgrep -v /sw/fink
%
% otool -L =zsh
/bin/zsh:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 71.0.0)
        /usr/lib/libncurses.5.dylib (compatibility version 5.0.0, current version 5.0.0)
% locate zpty | fgrep -v devenish
/usr/lib/zsh/4.1.1/zsh/zpty.so
/usr/share/zsh/4.1.1/functions/_zpty
%

Re: PATCH: zpty on HP/UX (and Mac OS X 10.3)

[mneptok]

On Mar 17, 2004, at 02:23, James Devenish wrote:

> In message <24420.1079518350@trentino.logica.co.uk>
> on Wed, Mar 17, 2004 at 11:12:30AM +0100, Oliver Kiddle wrote:
>> But can you read from the commands. i.e. if you do:
>>   zpty date date
>>   zpty -r date
>> will the second command print the date or will it exit with a status 
>> of
>> 1.

<snip>

My experience is the same as James'.

[mneptok@anubis] mneptok :: zpty date date
[mneptok@anubis] mneptok :: zpty -r date
Wed Mar 17 03:07:17 PST 2004
[mneptok@anubis] mneptok :: zsh --version
zsh 4.1.1 (powerpc-apple-darwin7.0)

Apple is shipping a pre-installed 4.1.1 binary, it's nothing I built. 
So someone at Apple has gotten this working. :)

./k

kurt von finck
-- 
http://www.mneptok.com
mneptok@mneptok.com
--

Success is more a function of consistent common sense than it is of 
genius. - An Wang

Re: Mac OS X 10.3 again [was: PATCH: zpty on HP/UX]

[James Devenish]

It seems that Darwin 7 has included dlcompat for at least six months (as
shown by /usr/include/dlfcn.h and the dl* man apges). So, this may
partly explain the success of zsh in Mac OS X 10.3. However, a few weeks
ago we saw on one of the zsh lists that zpty does not work in 10.2
(Client version) when dlcompat is available. I guess Oliver's recent
experiences with 10.2 (Server version) confirms that. Whether this means
10.2 is a lost cause, I don't know, but I'm guessing that Darwin 7 is
sufficiently "fixed" that zsh works out of the box. (I have not had
trouble compiling zsh or testing zpty under 10.3.x.) If Apple did not
provide zpty in 10.2 then perhaps there are no patches for zsh per se.
[Now reading Oliver's latest post.] I will try compiling zsh from CVS
tonight, if no one else posts about it.

Re: Mac OS X 10.3 again

[James Devenish]

For those wondering about the relationship between Darwin and Mac OS X
(Darwin can be obtained separately from Mac OS X, but Mac OS X is built
on top of Darwin), here is Apple's list:
<http://www.opensource.apple.com/darwinsource/index.html>

Also, note that the SourceForge compilation farm has the "Server"
versions of Mac OS X (as found in Xserves) while most users will
have the "Client" versions on their desktops. (However, hopefully,
the differences are immaterial as far as porting zsh goes.)

The boring news is that zsh HEAD basically works fine under Mac OS
10.3.3 ;-) The -N test still fails because HFS (the default filesystem)
does not record 'atime' -- we went through this on one of the zsh lists
earlier (but I notice the error message has not been updated to include
HFS). The minor issues that I saw during compilation were the following
warnings:

init.c:411: warning: implicit declaration of function `ioctl'
utils.c:985: warning: implicit declaration of function `ioctl'
utils.c:3349: warning: no previous prototype for `ucs4toutf8'
clone.c:81: warning: implicit declaration of function `ioctl'
zpty.c:324: warning: implicit declaration of function `ioctl'
zle_main.c:179: warning: implicit declaration of function `ioctl'
zle_utils.c:305: warning: implicit declaration of function `ioctl'

(ioctl is in sys/ioctl.h, and config.log shows:
    zsh_cv_header_sys_ioctl_h_ioctl_proto=yes
    #define HAVE_IOCTL_PROTO 1
but I guess there is still an actual #include that is lacking.)

Also saw this:

utils.c:3371: warning: suggest parentheses around arithmetic in operand of |

(Note: I have not gone through and enabled or disabled ./configurable
options, so I don't /really/ know if it all works. Nevertheless, `make
test` works, and there are no obvious show-stoppers like with Mac OS
10.2 and earlier. I don't recall any problems during my own usage.)

There is still a prominent number of people using 10.2, who might be
appreciative of patches, but it increasingly appears (at least to me --
and perhaps I am just repeating conclusions from earlier discussions)
that any 10.2-specific patches would be superfluous under 10.3.

Re: PATCH: zpty on HP/UX (and Mac OS X 10.3)

[Oliver Kiddle]

James Devenish wrote:
> % zpty -r date
> Wed Mar 17 18:17:21 WST 2004

Okay, good. Thanks.

> % echo $ZSH_VERSION
> 4.1.1

I tried building raw 4.1.1 and zpty has the same problems as for the
latest sources.

> I wonder if this is the version that is available as tarball/CVS from
> Apple's website?

Is this the tarball in http://darwinsource.opendarwin.org/tarballs/other/
Their zpty.c is unchanged from zsh 4.1.1.

Perhaps something was changed or fixed in Mac OS X 10.3 itself. If so,
it perhaps isn't worth expending too much effort trying to fix things for
older versions. Any chance someone with version 10.3 could try building
4.2.0-pre-4 (or latest CVS)?

> > If you build zsh on MacOS X.3, does it build the dynamic modules
> > without your having to install the dlcompat library?
> 
> % locate dlcompat | fgrep -v /sw/fink
> %
> % otool -L =zsh

Okay thanks. So that tells us that it isn't dynamically linked against
dlcompat but it is still quite possible that they built dlcompat and
linked zsh against the .a file.

If you try man dlopen, does the manpage exist (outside of your fink
installation)?

Thanks

Oliver

zpty on Mac OS X 10.2

[Peter Stephenson]

I tried the Macs at Sourceforge.  They claim to be be `Mac OS X 10.1'
and `Mac OS X 10.2'.  On the second uname -a gives `Darwin
ppc-osx2.cf.sourceforge.net 6.8 Darwin Kernel Version 6.8: Wed Sep 10
15:20:55 PDT 2003; root:xnu/xnu-344.49.obj~2/RELEASE_PPC Power Macintosh
powerpc'.  I don't know the relationship between Darwin and Mac OS X.

There appears to be no /dev/ptmx, so we are back in the ad-hoc stuff.
I couldn't see any evidence of STREAMS having been around, but the
documentation is a little thin.

There's special handling for Free BSD in zpty.c.  Here, however, cpp has
the following predefines (from cpp -dM):

#define __MACH__ 1
#define __POWERPC__ 1
#define __NATURAL_ALIGNMENT__ 1
#define __STDC_HOSTED__ 1
#define __NO_INLINE__ 1
#define __APPLE__ 1
#define __ppc__ 1
#define __GNUC__ 1
#define __DYNAMIC__ 1
#define __BIG_ENDIAN__ 1

Does it help to turn the #ifdef __FreeBSD__ in zpty.c into
#if defiend(__FreeBSD__) || defined(__APPLE__)

?

(The other Mac gives a similar list but this time I had to use `cc -E
-dM /dev/null' to get it.  That doesn't work in the previous case.)

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Science Park, Milton Road,
Cambridge, CB4 0WH, UK                          Tel: +44 (0)1223 692070


**********************************************************************
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. If you have received this email in error please notify
the system manager.

This footnote also confirms that this email message has been swept by
MIMEsweeper for the presence of computer viruses.

www.mimesweeper.com
**********************************************************************

Re: zpty on Mac OS X 10.2

[Oliver Kiddle]

Peter wrote:
> I tried the Macs at Sourceforge.  They claim to be be `Mac OS X 10.1'
> and `Mac OS X 10.2'.  On the second uname -a gives `Darwin
> ppc-osx2.cf.sourceforge.net 6.8 Darwin Kernel Version 6.8: Wed Sep 10

For what it's worth, the other is darwin 5.5.

> I don't know the relationship between Darwin and Mac OS X.

Darwin is the base system which is available as open source. It lacks
all the GUI stuff. You can install darwin alone (even on an x86 box
apparently), but you'd have little more than BSD Unix.

> Does it help to turn the #ifdef __FreeBSD__ in zpty.c into
> #if defiend(__FreeBSD__) || defined(__APPLE__)

It doesn't help. It's possibly a valid change though: pty(4) lists
     /dev/pty[p-sP-S][a-z0-9]   master pseudo terminals
     /dev/tty[p-sP-S][a-z0-9]   slave pseudo terminals
but if you go by the contents of /dev, it might be /dev/pty[p-w][a-f0-9]

Judging from what I read on an Apple web page, I think both __APPLE__
and __MACH__ are supposed to be checked. Presumably because __APPLE__
is defined on A/UX (though I can't imagine anyone running zsh on that
anymore).

Oliver