PATCH: zftp/tcp abstraction

PATCH: zftp/tcp abstraction

[Clint Adams]

I meant to get farther with this, but have had little time during the past
several days.  The idea is to move the more generic TCP functionality out
into a separate module so it can be reused for other functions.  I suspect
I may have broken some of the OS-kludges in the process.

Index: Src/Modules/tcp.c
===================================================================
RCS file: tcp.c
diff -N tcp.c
--- /dev/null	Thu May 24 22:33:05 2001
+++ tcp.c	Wed Jun  6 09:17:46 2001
@@ -0,0 +1,271 @@
+/*
+ * tcp.c - builtin FTP client
+ *
+ * This file is part of zsh, the Z shell.
+ *
+ * Copyright (c) 1998-2001 Peter Stephenson
+ * All rights reserved.
+ *
+ * Permission is hereby granted, without written agreement and without
+ * license or royalty fees, to use, copy, modify, and distribute this
+ * software and to distribute modified versions of this software for any
+ * purpose, provided that the above copyright notice and the following
+ * two paragraphs appear in all copies of this software.
+ *
+ * In no event shall Peter Stephenson or the Zsh Development
+ * Group be liable to any party for direct, indirect, special, incidental,
+ * or consequential damages arising out of the use of this software and
+ * its documentation, even if Peter Stephenson, and the Zsh
+ * Development Group have been advised of the possibility of such damage.
+ *
+ * Peter Stephenson and the Zsh Development Group specifically
+ * disclaim any warranties, including, but not limited to, the implied
+ * warranties of merchantability and fitness for a particular purpose.  The
+ * software provided hereunder is on an "as is" basis, and Peter Stephenson
+ * and the Zsh Development Group have no obligation to provide maintenance,
+ * support, updates, enhancements, or modifications.
+ *
+ */
+
+/*
+ * We need to include the zsh headers later to avoid clashes with
+ * the definitions on some systems, however we need the configuration
+ * file to decide whether we can include netinet/in_systm.h, which
+ * doesn't exist on cygwin.
+ */
+#include "tcp.h"
+
+/*
+ * For some reason, configure doesn't always detect netinet/in_systm.h.
+ * On some systems, including linux, this seems to be because gcc is
+ * throwing up a warning message about the redefinition of
+ * __USE_LARGEFILE.  This means the problem is somewhere in the
+ * header files where we can't get at it.  For now, revert to
+ * not including this file only on systems where we know it's missing.
+ * Currently this is just cygwin.
+ */
+#ifndef __CYGWIN__
+# include <netinet/in_systm.h>
+#endif
+#include <netinet/in.h>
+#include <netinet/ip.h>
+#include <arpa/inet.h>
+
+/* it's a TELNET based protocol, but don't think I like doing this */
+#include <arpa/telnet.h>
+
+/*
+ * We use poll() in preference to select because some subset of manuals says
+ * that's the thing to do, plus it's a bit less fiddly.  I don't actually
+ * have access to a system with poll but not select, however, though
+ * both bits of the code have been tested on a machine with both.
+ */
+#ifdef HAVE_POLL_H
+# include <poll.h>
+#endif
+#if defined(HAVE_POLL) && !defined(POLLIN) && !defined(POLLNORM)
+# undef HAVE_POLL
+#endif
+
+#ifdef USE_LOCAL_H_ERRNO
+int h_errno;
+#endif
+
+/* We use the RFC 2553 interfaces.  If the functions don't exist in the library,
+   simulate them. */
+
+#ifndef INET_ADDRSTRLEN
+# define INET_ADDRSTRLEN 16
+#endif
+
+#ifndef INET6_ADDRSTRLEN
+# define INET6_ADDRSTRLEN 46
+#endif
+
+/**/
+#ifndef HAVE_INET_NTOP
+
+/**/
+mod_export char const *
+zsh_inet_ntop(int af, void const *cp, char *buf, size_t len)
+{       
+        if(af != AF_INET) {
+                errno = EAFNOSUPPORT;
+                return NULL;
+        } 
+        if(len < INET_ADDRSTRLEN) {
+                errno = ENOSPC;
+                return NULL;
+        }
+        strcpy(buf, inet_ntoa(*(struct in_addr *)cp));
+        return buf;
+}
+
+/**/
+#else /* !HAVE_INET_NTOP */
+
+/**/
+# define zsh_inet_ntop inet_ntop
+
+/**/
+#endif /* !HAVE_INET_NTOP */
+
+/**/
+#ifndef HAVE_INET_PTON
+
+/**/
+# ifndef HAVE_INET_ATON
+
+#  ifndef INADDR_NONE
+#   define INADDR_NONE 0xffffffffUL
+#  endif
+
+/**/
+mod_export int zsh_inet_aton(char const *src, struct in_addr *dst)
+{
+    return (dst->s_addr = inet_addr(src)) != INADDR_NONE;
+}
+
+/**/
+#else /* !HAVE_INET_ATON */
+
+/**/
+# define zsh_inet_aton inet_aton
+
+/**/
+# endif /* !HAVE_INET_ATON */
+
+/**/
+mod_export int
+zsh_inet_pton(int af, char const *src, void *dst)
+{
+        if(af != AF_INET) {
+                errno = EAFNOSUPPORT;
+                return -1;
+        }
+        return !!zsh_inet_aton(src, dst);
+}
+
+#else /* !HAVE_INET_PTON */
+
+# define zsh_inet_pton inet_pton
+
+/**/
+#endif /* !HAVE_INET_PTON */
+
+/**/
+#ifndef HAVE_GETIPNODEBYNAME
+
+/**/
+# ifndef HAVE_GETHOSTBYNAME2
+
+/**/
+mod_export struct hostent *
+zsh_gethostbyname2(char const *name, int af)
+{
+	if(af != AF_INET) {
+		h_errno = NO_RECOVERY;
+		return NULL;
+	}
+	return gethostbyname(name);
+}
+
+/**/
+#else /* !HAVE_GETHOSTBYNAME2 */
+
+/**/
+# define zsh_gethostbyname2 gethostbyname2
+
+/**/
+# endif /* !HAVE_GETHOSTBYNAME2 */
+
+/* note: this is not a complete implementation.  If ignores the flags,
+   and does not provide the memory allocation of the standard interface.
+   Each returned structure will overwrite the previous one. */
+
+/**/
+mod_export struct hostent *
+zsh_getipnodebyname(char const *name, int af, int flags, int *errorp)
+{
+	static struct hostent ahe;
+	static char nbuf[16];
+	static char *addrlist[] = { nbuf, NULL };
+# ifdef SUPPORT_IPV6
+	static char pbuf[INET6_ADDRSTRLEN];
+# else
+	static char pbuf[INET_ADDRSTRLEN];
+# endif
+	struct hostent *he;
+	if(zsh_inet_pton(af, name, nbuf) == 1) {
+		zsh_inet_ntop(af, nbuf, pbuf, sizeof(pbuf));
+		ahe.h_name = pbuf;
+		ahe.h_aliases = addrlist+1;
+		ahe.h_addrtype = af;
+		ahe.h_length = (af == AF_INET) ? 4 : 16;
+		ahe.h_addr_list = addrlist;
+		return &ahe;
+	}
+	he = zsh_gethostbyname2(name, af);
+	if(!he)
+		*errorp = h_errno;
+	return he;
+}
+
+/**/
+mod_export void
+freehostent(struct hostent *ptr)
+{
+}
+
+/**/
+#else /* !HAVE_GETIPNODEBYNAME */
+
+/**/
+# define zsh_getipnodebyname getipnodebyname
+
+/**/
+#endif /* !HAVE_GETIPNODEBYNAME */
+
+/**/
+mod_export int
+tcp_socket(int domain, int type, int protocol, Tcp_session sess)
+{
+	sess->fd = socket(domain, type, protocol);
+	return sess->fd;
+}
+
+static void
+tcp_cleanup(void)
+{
+}
+
+/* The load/unload routines required by the zsh library interface */
+
+/**/
+int
+setup_(Module m)
+{
+    return 0;
+}
+
+/**/
+int
+boot_(Module m)
+{
+    return 0;
+}
+
+/**/
+int
+cleanup_(Module m)
+{
+    tcp_cleanup();
+    return 0;
+}
+
+/**/
+int
+finish_(Module m)
+{
+    return 0;
+}
Index: Src/Modules/tcp.h
===================================================================
RCS file: tcp.h
diff -N tcp.h
--- /dev/null	Thu May 24 22:33:05 2001
+++ tcp.h	Wed Jun  6 09:17:46 2001
@@ -0,0 +1,67 @@
+/*
+ * tcp.h - builtin FTP client
+ *
+ * This file is part of zsh, the Z shell.
+ *
+ * Copyright (c) 1998-2001 Peter Stephenson
+ * All rights reserved.
+ *
+ * Permission is hereby granted, without written agreement and without
+ * license or royalty fees, to use, copy, modify, and distribute this
+ * software and to distribute modified versions of this software for any
+ * purpose, provided that the above copyright notice and the following
+ * two paragraphs appear in all copies of this software.
+ *
+ * In no event shall Peter Stephenson or the Zsh Development
+ * Group be liable to any party for direct, indirect, special, incidental,
+ * or consequential damages arising out of the use of this software and
+ * its documentation, even if Peter Stephenson, and the Zsh
+ * Development Group have been advised of the possibility of such damage.
+ *
+ * Peter Stephenson and the Zsh Development Group specifically
+ * disclaim any warranties, including, but not limited to, the implied
+ * warranties of merchantability and fitness for a particular purpose.  The
+ * software provided hereunder is on an "as is" basis, and Peter Stephenson
+ * and the Zsh Development Group have no obligation to provide maintenance,
+ * support, updates, enhancements, or modifications.
+ *
+ */
+
+#include "../../config.h"
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netdb.h>
+
+/* Is IPv6 supported by the library? */
+
+#if defined(AF_INET6) && defined(IN6ADDR_LOOPBACK_INIT) \
+	&& defined(HAVE_INET_NTOP) && defined(HAVE_INET_PTON)
+# define SUPPORT_IPV6 1
+#endif
+
+union tcp_sockaddr {
+	struct sockaddr a;
+	struct sockaddr_in in;
+#ifdef SUPPORT_IPV6
+	struct sockaddr_in6 in6;
+#endif
+};
+
+struct tcp_session {
+	int fd;				/* file descriptor */
+	union tcp_sockaddr sock;  	/* local address   */
+	union tcp_sockaddr peer;  	/* remote address  */
+};
+
+typedef struct tcp_session *Tcp_session;
+
+#include "tcp.mdh"
+#include "tcp.pro"
+
+#ifndef INET_ADDRSTRLEN
+# define INET_ADDRSTRLEN 16
+#endif
+
+#ifndef INET6_ADDRSTRLEN
+# define INET6_ADDRSTRLEN 46
+#endif
Index: Src/Modules/tcp.mdd
===================================================================
RCS file: tcp.mdd
diff -N tcp.mdd
--- /dev/null	Thu May 24 22:33:05 2001
+++ tcp.mdd	Wed Jun  6 09:17:46 2001
@@ -0,0 +1,5 @@
+name=zsh/tcp
+link=dynamic
+load=no
+
+objects="tcp.o"
Index: Src/Modules/zftp.c
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/zftp.c,v
retrieving revision 1.12
diff -u -r1.12 zftp.c
--- Src/Modules/zftp.c	2001/05/10 16:57:28	1.12
+++ Src/Modules/zftp.c	2001/06/06 16:17:47
@@ -46,7 +46,6 @@
 struct in_addr;
 struct sockaddr_in;
 struct sockaddr_in6;
-union zftp_sockaddr;
 struct zftp_session;
 typedef struct zftp_session *Zftp_session;
 
@@ -77,6 +76,7 @@
 #include <netinet/ip.h>
 #include <arpa/inet.h>
 
+#include "tcp.h"
 #include "zftp.mdh"
 #include "zftp.pro"
 
@@ -96,11 +96,9 @@
 # undef HAVE_POLL
 #endif
 
-/* Is IPv6 supported by the library? */
 
-#if defined(AF_INET6) && defined(IN6ADDR_LOOPBACK_INIT) \
-	&& defined(HAVE_INET_NTOP) && defined(HAVE_INET_PTON)
-# define SUPPORT_IPV6 1
+#ifdef USE_LOCAL_H_ERRNO
+int h_errno;
 #endif
 
 union zftp_sockaddr {
@@ -115,153 +113,6 @@
 int h_errno;
 #endif
 
-/* We use the RFC 2553 interfaces.  If the functions don't exist in the library,
-   simulate them. */
-
-#ifndef INET_ADDRSTRLEN
-# define INET_ADDRSTRLEN 16
-#endif
-
-#ifndef INET6_ADDRSTRLEN
-# define INET6_ADDRSTRLEN 46
-#endif
-
-/**/
-#ifndef HAVE_INET_NTOP
-
-/**/
-static char const *
-zsh_inet_ntop(int af, void const *cp, char *buf, size_t len)
-{       
-        if(af != AF_INET) {
-                errno = EAFNOSUPPORT;
-                return NULL;
-        } 
-        if(len < INET_ADDRSTRLEN) {
-                errno = ENOSPC;
-                return NULL;
-        }
-        strcpy(buf, inet_ntoa(*(struct in_addr *)cp));
-        return buf;
-}
-
-#else /* !HAVE_INET_NTOP */
-
-# define zsh_inet_ntop inet_ntop
-
-/**/
-#endif /* !HAVE_INET_NTOP */
-
-/**/
-#ifndef HAVE_INET_PTON
-
-/**/
-# ifndef HAVE_INET_ATON
-
-#  ifndef INADDR_NONE
-#   define INADDR_NONE 0xffffffffUL
-#  endif
-
-/**/
-static int zsh_inet_aton(char const *src, struct in_addr *dst)
-{
-    return (dst->s_addr = inet_addr(src)) != INADDR_NONE;
-}
-
-#else /* !HAVE_INET_ATON */
-
-# define zsh_inet_aton inet_aton
-
-/**/
-# endif /* !HAVE_INET_ATON */
-
-/**/
-static int
-zsh_inet_pton(int af, char const *src, void *dst)
-{
-        if(af != AF_INET) {
-                errno = EAFNOSUPPORT;
-                return -1;
-        }
-        return !!zsh_inet_aton(src, dst);
-}
-
-#else /* !HAVE_INET_PTON */
-
-# define zsh_inet_pton inet_pton
-
-/**/
-#endif /* !HAVE_INET_PTON */
-
-/**/
-#ifndef HAVE_GETIPNODEBYNAME
-
-/**/
-# ifndef HAVE_GETHOSTBYNAME2
-
-/**/
-static struct hostent *
-zsh_gethostbyname2(char const *name, int af)
-{
-	if(af != AF_INET) {
-		h_errno = NO_RECOVERY;
-		return NULL;
-	}
-	return gethostbyname(name);
-}
-
-#else /* !HAVE_GETHOSTBYNAME2 */
-
-# define zsh_gethostbyname2 gethostbyname2
-
-/**/
-# endif /* !HAVE_GETHOSTBYNAME2 */
-
-
-/* note: this is not a complete implementation.  If ignores the flags,
-   and does not provide the memory allocation of the standard interface.
-   Each returned structure will overwrite the previous one. */
-
-/**/
-static struct hostent *
-zsh_getipnodebyname(char const *name, int af, int flags, int *errorp)
-{
-	static struct hostent ahe;
-	static char nbuf[16];
-	static char *addrlist[] = { nbuf, NULL };
-# ifdef SUPPORT_IPV6
-	static char pbuf[INET6_ADDRSTRLEN];
-# else
-	static char pbuf[INET_ADDRSTRLEN];
-# endif
-	struct hostent *he;
-	if(zsh_inet_pton(af, name, nbuf) == 1) {
-		zsh_inet_ntop(af, nbuf, pbuf, sizeof(pbuf));
-		ahe.h_name = pbuf;
-		ahe.h_aliases = addrlist+1;
-		ahe.h_addrtype = af;
-		ahe.h_length = (af == AF_INET) ? 4 : 16;
-		ahe.h_addr_list = addrlist;
-		return &ahe;
-	}
-	he = zsh_gethostbyname2(name, af);
-	if(!he)
-		*errorp = h_errno;
-	return he;
-}
-/**/
-static void
-freehostent(struct hostent *ptr)
-{
-}
-
-#else /* !HAVE_GETIPNODEBYNAME */
-
-# define zsh_getipnodebyname getipnodebyname
-
-/**/
-#endif /* !HAVE_GETIPNODEBYNAME */
-
 /*
  * For FTP block mode
  *
@@ -476,10 +327,8 @@
     char *name;			/* name of session */
     char **params;		/* parameters ordered as in zfparams */
     char **userparams;		/* user parameters set by zftp_params */
-    int cfd;			/* control file descriptor */
     FILE *cin;			/* control input file */
-    union zftp_sockaddr sock;	/* this end of the control connection */
-    union zftp_sockaddr peer;	/* far end of the control connection */
+    struct tcp_session control;	/* the control connection */
     int dfd;			/* data connection */
     int has_size;		/* understands SIZE? */
     int has_mdtm;		/* understands MDTM? */
@@ -821,7 +670,7 @@
 		cmdbuf[0] = (char)IAC;
 		cmdbuf[1] = (char)DONT;
 		cmdbuf[2] = ch;
-		write(zfsess->cfd, cmdbuf, 3);
+		write(zfsess->control.fd, cmdbuf, 3);
 		continue;
 
 	    case DO:
@@ -831,7 +680,7 @@
 		cmdbuf[0] = (char)IAC;
 		cmdbuf[1] = (char)WONT;
 		cmdbuf[2] = ch;
-		write(zfsess->cfd, cmdbuf, 3);
+		write(zfsess->control.fd, cmdbuf, 3);
 		continue;
 
 	    case EOF:
@@ -881,7 +730,7 @@
     char line[256], *ptr, *verbose;
     int stopit, printing = 0, tmout;
 
-    if (zfsess->cfd == -1)
+    if (zfsess->control.fd == -1)
 	return 6;
     zsfree(lastmsg);
     lastmsg = NULL;
@@ -1009,7 +858,7 @@
      */
     int ret, tmout;
 
-    if (zfsess->cfd == -1)
+    if (zfsess->control.fd == -1)
 	return 6;
     tmout = getiparam("ZFTP_TMOUT");
     if (setjmp(zfalrmbuf)) {
@@ -1018,7 +867,7 @@
 	return 6;
     }
     zfalarm(tmout);
-    ret = write(zfsess->cfd, cmd, strlen(cmd));
+    ret = write(zfsess->control.fd, cmd, strlen(cmd));
     alarm(0);
 
     if (ret <= 0) {
@@ -1035,7 +884,7 @@
 
 /**/
 static int
-zfopendata(char *name, union zftp_sockaddr *zdsockp, int *is_passivep)
+zfopendata(char *name, union tcp_sockaddr *zdsockp, int *is_passivep)
 {
     if (!(zfprefs & (ZFPF_SNDP|ZFPF_PASV))) {
 	zwarnnam(name, "Must set preference S or P to transfer data", NULL, 0);
@@ -1052,7 +901,7 @@
 	int err, salen;
 
 #ifdef SUPPORT_IPV6
-	if(zfsess->peer.a.sa_family == AF_INET6)
+	if(zfsess->control.peer.a.sa_family == AF_INET6)
 	    psv_cmd = "EPSV\r\n";
 	else
 #endif /* SUPPORT_IPV6 */
@@ -1068,9 +917,9 @@
 	    zfclosedata();
 	    return zfopendata(name, zdsockp, is_passivep);
 	}
-	zdsockp->a.sa_family = zfsess->peer.a.sa_family;
+	zdsockp->a.sa_family = zfsess->control.peer.a.sa_family;
 #ifdef SUPPORT_IPV6
-	if(zfsess->peer.a.sa_family == AF_INET6) {
+	if(zfsess->control.peer.a.sa_family == AF_INET6) {
 	    /* see RFC 2428 for explanation */
 	    char const *ptr, *end;
 	    char delim, portbuf[6], *pbp;
@@ -1098,7 +947,7 @@
 	    portnum = strtoul(portbuf, &pbp, 10);
 	    if(*pbp || portnum > 65535UL)
 		goto bad_epsv;
-	    *zdsockp = zfsess->peer;
+	    *zdsockp = zfsess->control.peer;
 	    zdsockp->in6.sin6_port = htons((unsigned)portnum);
 	    salen = sizeof(struct sockaddr_in6);
 	} else
@@ -1157,7 +1006,7 @@
 	    return 1;
 	}
 
-	*zdsockp = zfsess->sock;
+	*zdsockp = zfsess->control.sock;
 #ifdef SUPPORT_IPV6
 	if(zdsockp->a.sa_family == AF_INET6) {
 	    zdsockp->in6.sin6_port = 0;	/* to be set by bind() */
@@ -1243,7 +1092,7 @@
 zfgetdata(char *name, char *rest, char *cmd, int getsize)
 {
     int len, newfd, is_passive;
-    union zftp_sockaddr zdsock;
+    union tcp_sockaddr zdsock;
 
     if (zfopendata(name, &zdsock, &is_passive))
 	return 1;
@@ -1831,8 +1680,8 @@
 
 	/* the following is black magic, as far as I'm concerned. */
 	/* what are we going to do if it fails?  not a lot, actually. */
-	send(zfsess->cfd, (char *)msg, 3, 0);
-	send(zfsess->cfd, (char *)msg+3, 1, MSG_OOB);
+	send(zfsess->control.fd, (char *)msg, 3, 0);
+	send(zfsess->control.fd, (char *)msg+3, 1, MSG_OOB);
 
 	zfsendcmd("ABOR\r\n");
 	if (lastcode == 226) {
@@ -1893,7 +1742,7 @@
      * Probably this is the safest thing to do.  It's possible
      * a `QUIT' will hang, though.
      */
-    if (zfsess->cfd != -1)
+    if (zfsess->control.fd != -1)
 	zfclose(0);
 
     /* this is going to give 0.  why bother? */
@@ -1950,24 +1799,24 @@
 	}
 	zfsetparam("ZFTP_HOST", ztrdup(zhostp->h_name), ZFPM_READONLY);
 
-	zfsess->peer.a.sa_family = af;
+	zfsess->control.peer.a.sa_family = af;
 #ifdef SUPPORT_IPV6
 	if(af == AF_INET6) {
-	    zfsess->peer.in6.sin6_port = zservp->s_port;
-	    zfsess->peer.in6.sin6_flowinfo = 0;
+	    zfsess->control.peer.in6.sin6_port = zservp->s_port;
+	    zfsess->control.peer.in6.sin6_flowinfo = 0;
 # ifdef HAVE_STRUCT_SOCKADDR_IN6_SIN6_SCOPE_ID
-	    zfsess->peer.in6.sin6_scope_id = 0;
+	    zfsess->control.peer.in6.sin6_scope_id = 0;
 # endif
 	    salen = sizeof(struct sockaddr_in6);
 	} else
 #endif /* SUPPORT_IPV6 */
 	{
-	    zfsess->peer.in.sin_port = zservp->s_port;
+	    zfsess->control.peer.in.sin_port = zservp->s_port;
 	    salen = sizeof(struct sockaddr_in);
 	}
 
-	zfsess->cfd = socket(af, SOCK_STREAM, 0);
-	if (zfsess->cfd < 0) {
+	tcp_socket(af, SOCK_STREAM, 0, &(zfsess->control));
+	if (zfsess->control.fd < 0) {
 	    freehostent(zhostp);
 	    zfunsetparam("ZFTP_HOST");
 	    FAILED();
@@ -1989,12 +1838,12 @@
 	for (addrp = zhostp->h_addr_list; err && *addrp; addrp++) {
 #ifdef SUPPORT_IPV6
 	    if(af == AF_INET6)
-		memcpy(&zfsess->peer.in6.sin6_addr, *addrp, zhostp->h_length);
+		memcpy(&zfsess->control.peer.in6.sin6_addr, *addrp, zhostp->h_length);
 	    else
 #endif /* SUPPORT_IPV6 */
-		memcpy(&zfsess->peer.in.sin_addr, *addrp, zhostp->h_length);
+		memcpy(&zfsess->control.peer.in.sin_addr, *addrp, zhostp->h_length);
 	    do {
-		err = connect(zfsess->cfd, (struct sockaddr *)&zfsess->peer,
+		err = connect(zfsess->control.fd, (struct sockaddr *)&zfsess->control.peer,
 			      salen);
 	    } while (err && errno == EINTR && !errflag);
 	    /* you can check whether it's worth retrying here */
@@ -2030,15 +1879,15 @@
      * Move the fd out of the user-visible range.  We need to do
      * this after the connect() on some systems.
      */
-    zfsess->cfd = zfmovefd(zfsess->cfd);
+    zfsess->control.fd = zfmovefd(zfsess->control.fd);
 
 #if defined(F_SETFD) && defined(FD_CLOEXEC)
     /* If the shell execs a program, we don't want this fd left open. */
-    fcntl(zfsess->cfd, F_SETFD, FD_CLOEXEC);
+    fcntl(zfsess->control.fd, F_SETFD, FD_CLOEXEC);
 #endif
 
-    len = sizeof(zfsess->sock);
-    if (getsockname(zfsess->cfd, (struct sockaddr *)&zfsess->sock, &len) < 0) {
+    len = sizeof(zfsess->control.sock);
+    if (getsockname(zfsess->control.fd, (struct sockaddr *)&zfsess->control.sock, &len) < 0) {
 	zwarnnam(name, "getsockname failed: %e", NULL, errno);
 	zfclose(0);
 	return 1;
@@ -2050,20 +1899,20 @@
      * do clever things with SIGURG.
      */
     len = 1;
-    setsockopt(zfsess->cfd, SOL_SOCKET, SO_OOBINLINE,
+    setsockopt(zfsess->control.fd, SOL_SOCKET, SO_OOBINLINE,
 	       (char *)&len, sizeof(len));
 #endif
 #if defined(IP_TOS) && defined(IPTOS_LOWDELAY)
     /* for control connection we want low delay.  please don't laugh. */
     len = IPTOS_LOWDELAY;
-    setsockopt(zfsess->cfd, IPPROTO_IP, IP_TOS, (char *)&len, sizeof(len));
+    setsockopt(zfsess->control.fd, IPPROTO_IP, IP_TOS, (char *)&len, sizeof(len));
 #endif
 
     /*
      * We use stdio with line buffering for convenience on input.
      * On output, we can just dump a complete message to the fd via write().
      */
-    zfsess->cin = fdopen(zfsess->cfd, "r");
+    zfsess->cin = fdopen(zfsess->control.fd, "r");
 
     if (!zfsess->cin) {
 	zwarnnam(name, "file handling error", NULL, 0);
@@ -2107,17 +1956,17 @@
 	unlink(fname);
     }
 
-    if (zfsess->cfd == -1) {
+    if (zfsess->control.fd == -1) {
 	/* final paranoid check */
 	return 1;
     }
 	
     zfsetparam("ZFTP_MODE", ztrdup("S"), ZFPM_READONLY);
     /* if remaining arguments, use them to log in. */
-    if (zfsess->cfd > -1 && *++args)
+    if (zfsess->control.fd > -1 && *++args)
 	return zftp_login(name, args, flags);
     /* if something wayward happened, connection was already closed */
-    return zfsess->cfd == -1;
+    return zfsess->control.fd == -1;
 }
 
 /*
@@ -2311,7 +2160,7 @@
     }
 
     zsfree(ucmd);
-    if (zfsess->cfd == -1)
+    if (zfsess->control.fd == -1)
 	return 1;
     if (stopit == 2 || (lastcode != 230 && lastcode != 202)) {
 	zwarnnam(name, "login failed", NULL, 0);
@@ -2390,7 +2239,7 @@
     struct timeval tv;
 # endif /* HAVE_POLL */
 
-    if (zfsess->cfd == -1)
+    if (zfsess->control.fd == -1)
 	return 1;
 
 # ifdef HAVE_POLL
@@ -2398,7 +2247,7 @@
     /* safety first, though I think POLLIN is more common */
 #   define POLLIN POLLNORM
 #  endif /* HAVE_POLL */
-    pfd.fd = zfsess->cfd;
+    pfd.fd = zfsess->control.fd;
     pfd.events = POLLIN;
     if ((ret = poll(&pfd, 1, 0)) < 0 && errno != EINTR && errno != EAGAIN)
 	zfclose(0);
@@ -2408,10 +2257,10 @@
     }
 # else
     FD_ZERO(&f);
-    FD_SET(zfsess->cfd, &f);
+    FD_SET(zfsess->control.fd, &f);
     tv.tv_sec = 0;
     tv.tv_usec = 0;
-    if ((ret = select(zfsess->cfd +1, (SELECT_ARG_2_T) &f,
+    if ((ret = select(zfsess->control.fd +1, (SELECT_ARG_2_T) &f,
 		      NULL, NULL, &tv)) < 0
 	&& errno != EINTR)
 	zfclose(0);
@@ -2420,8 +2269,8 @@
 	zfgetmsg();
     }
 # endif /* HAVE_POLL */
-    /* if we now have zfsess->cfd == -1, then we've just been dumped out. */
-    return (zfsess->cfd == -1) ? 2 : 0;
+    /* if we now have zfsess->control.fd == -1, then we've just been dumped out. */
+    return (zfsess->control.fd == -1) ? 2 : 0;
 #else
     zfwarnnam(name, "not supported on this system.", NULL, 0);
     return 3;
@@ -2843,7 +2692,7 @@
     char **aptr;
     Eprog prog;
 
-    if (zfsess->cfd == -1)
+    if (zfsess->control.fd == -1)
 	return;
 
     zfclosing = 1;
@@ -2859,10 +2708,10 @@
 	fclose(zfsess->cin);
 	zfsess->cin = NULL;
     }
-    if (zfsess->cfd != -1) {
+    if (zfsess->control.fd != -1) {
 	zfnopen--;
-	close(zfsess->cfd);
-	zfsess->cfd = -1;
+	close(zfsess->control.fd);
+	zfsess->control.fd = -1;
     }
 
     if (zfstatfd != -1) {
@@ -2935,7 +2784,7 @@
     if (!nptr) {
 	zfsess = (Zftp_session) zcalloc(sizeof(struct zftp_session));
 	zfsess->name = ztrdup(nm);
-	zfsess->cfd = zfsess->dfd = -1;
+	zfsess->control.fd = zfsess->dfd = -1;
 	zfsess->params = (char **) zcalloc(sizeof(zfparams));
 	zaddlinknode(zfsessions, zfsess);
 
@@ -3150,7 +2999,7 @@
 	int oldstatus = zfstatusp[zfsessno];
 	lseek(zfstatfd, 0, 0);
 	read(zfstatfd, (char *)zfstatusp, sizeof(int)*zfsesscnt);
-	if (zfsess->cfd != -1 && (zfstatusp[zfsessno] & ZFST_CLOS)) {
+	if (zfsess->control.fd != -1 && (zfstatusp[zfsessno] & ZFST_CLOS)) {
 	    /* got closed in subshell without us knowing */
 	    zcfinish = 2;
 	    zfclose(0);
@@ -3171,7 +3020,7 @@
 	}
     }
 #if defined(HAVE_SELECT) || defined (HAVE_POLL)
-    if (zfsess->cfd != -1 && !(zptr->flags & (ZFTP_TEST|ZFTP_SESS))) {
+    if (zfsess->control.fd != -1 && !(zptr->flags & (ZFTP_TEST|ZFTP_SESS))) {
 	/*
 	 * Test the connection for a bad fd or incoming message, but
 	 * only if the connection was last heard of open, and
@@ -3181,7 +3030,7 @@
 	ret = zftp_test("zftp test", NULL, 0);
     }
 #endif
-    if ((zptr->flags & ZFTP_CONN) && zfsess->cfd == -1) {
+    if ((zptr->flags & ZFTP_CONN) && zfsess->control.fd == -1) {
 	if (ret != 2) {
 	    /*
 	     * with ret == 2, we just got dumped out in the test,
Index: Src/Modules/zftp.mdd
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/zftp.mdd,v
retrieving revision 1.6
diff -u -r1.6 zftp.mdd
--- Src/Modules/zftp.mdd	2001/04/23 19:59:04	1.6
+++ Src/Modules/zftp.mdd	2001/06/06 16:17:47
@@ -5,4 +5,6 @@
 
 autobins="zftp"
 
+moddeps="zsh/tcp"
+
 objects="zftp.o"

Module namespaces? RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

> +name=zsh/tcp

I meant to mention it anyway.

What is the module nameing policy? It is somewhat silly to invent
hierarchical module namespace and put all modules under the same hood.

E.g. this one properly deserves something like net/tcp and zftp itself may
be sensibly moved into net/zftp as well.

-andrej

Re: Module namespaces? RE: PATCH: zftp/tcp abstraction

[Zefram]

Andrej Borsenkow wrote:
>What is the module nameing policy? It is somewhat silly to invent
>hierarchical module namespace and put all modules under the same hood.

My intention was that the first part of the hierarchical name identify
the publisher of the module.  For example, there could hypothetically
be a module named "redhat/foo" published by Red Hat.  The "zsh/" prefix
is intended for all (and only) modules published by the Zsh Development
Group.

-zefram

Re: Module namespaces? RE: PATCH: zftp/tcp abstraction

[Peter Stephenson]

Zefram wrote:
> Andrej Borsenkow wrote:
> >What is the module nameing policy? It is somewhat silly to invent
> >hierarchical module namespace and put all modules under the same hood.
> 
> My intention was that the first part of the hierarchical name identify
> the publisher of the module.  For example, there could hypothetically
> be a module named "redhat/foo" published by Red Hat.  The "zsh/" prefix
> is intended for all (and only) modules published by the Zsh Development
> Group.

We could use zsh/net/*.  I wouldn't want to move zftp there, however, since
it will be a major nuisance now.  But there's nothing to stop us making
zsh/net/tcp.  Zftp is directly useful to users, while tcp isn't, so hiding
the one at a lower level makes some kind of twisted sense.

I don't know offhand if the install scripts handle extra levels, but
zmodload won't have any problem.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: Module namespaces? RE: PATCH: zftp/tcp abstraction

[Clint Adams]

> We could use zsh/net/*.  I wouldn't want to move zftp there, however, since
> it will be a major nuisance now.  But there's nothing to stop us making
> zsh/net/tcp.  Zftp is directly useful to users, while tcp isn't, so hiding
> the one at a lower level makes some kind of twisted sense.
> 
> I don't know offhand if the install scripts handle extra levels, but
> zmodload won't have any problem.

make check is happy, at least.

Index: Src/Modules/tcp.mdd
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/tcp.mdd,v
retrieving revision 1.1
diff -u -r1.1 tcp.mdd
--- Src/Modules/tcp.mdd	2001/06/06 18:53:56	1.1
+++ Src/Modules/tcp.mdd	2001/06/06 19:10:49
@@ -1,4 +1,4 @@
-name=zsh/tcp
+name=zsh/net/tcp
 link=dynamic
 load=no
 
Index: Src/Modules/zftp.mdd
===================================================================
RCS file: /cvsroot/zsh/zsh/Src/Modules/zftp.mdd,v
retrieving revision 1.7
diff -u -r1.7 zftp.mdd
--- Src/Modules/zftp.mdd	2001/06/06 18:53:56	1.7
+++ Src/Modules/zftp.mdd	2001/06/06 19:10:49
@@ -5,6 +5,6 @@
 
 autobins="zftp"
 
-moddeps="zsh/tcp"
+moddeps="zsh/net/tcp"
 
 objects="zftp.o"

RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

bor@itsrm2% zmodload -u zsh/zftp
bor@itsrm2% zsh
bor@itsrm2% zmodload zsh/zftp
bor@itsrm2% zftp open localhost
ld.so: zsh: symbol not found: zsh_getipnodebyname
zsh: killed     zsh

-andrej

Modules dependencies RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

>
> bor@itsrm2% zmodload -u zsh/zftp
> bor@itsrm2% zsh
> bor@itsrm2% zmodload zsh/zftp
> bor@itsrm2% zftp open localhost
> ld.so: zsh: symbol not found: zsh_getipnodebyname
> zsh: killed     zsh
>

Index: Src/Modules/zftp.mdd
+moddeps="zsh/net/tcp"

The problem is, it is currently ignored for modules without auto stuff.
Dependencies are put only in main zsh in Src/bltinmods.list.

To make it work in general every module needs a standard function, let's
call it dep_, that is autogenerated out of moddeps and called before module
is attempted to be loaded to register dependencies. We may need the
counterpart to remove dependencies when module is unloaded (seems
reasonable).

-andrej

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> Index: Src/Modules/zftp.mdd
> +moddeps="zsh/net/tcp"
> 
> The problem is, it is currently ignored for modules without auto stuff.
> Dependencies are put only in main zsh in Src/bltinmods.list.
> 
> To make it work in general every module needs a standard function, let's
> call it dep_, that is autogenerated out of moddeps and called before module
> is attempted to be loaded to register dependencies. We may need the
> counterpart to remove dependencies when module is unloaded (seems
> reasonable).

Yes, possibly we should distinguish explicitly by checks in module.c
between modules which provide features directly (builtins, parameters,
etc.) and those which only exist as a dependency to provide support for
other modules.  Then we can tell what it's safe to unload.  Currently
modules are either one or the other --- maybe we can get away with keeping
it like this.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Zefram]

Andrej Borsenkow wrote:
>To make it work in general every module needs a standard function, let's
>call it dep_, that is autogenerated out of moddeps and called before module
>is attempted to be loaded to register dependencies.

Problem is you can't call that function without loading the module, and
you can't successfully load the module until you've got that information.
If there were an easy solution we'd have implemented it as soon as we
came up with the module system.

Basically we want to be able to read the list of dependencies before
we load the module.  The best idea I came up with was to concatenate
the list onto the end of the shared object file -- so it can be read
from the file before we invoke the dynamic linker -- but in the general
case this could interfere with the shared object file format enough that
the dynamic linker might reject it.  Another possibility is to put the
dependency list in a separate file, and install the two files (shared
object and dependency list) whenever we want to install the module.

Neither of my solutions is completely satisfactory, but it now seems that
we are making sufficient use of interdependent modules that we need some
form of implicit dependencies.

-zefram

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Sven Wischnowsky]

Zefram wrote:

> ...
> 
> Neither of my solutions is completely satisfactory, but it now seems that
> we are making sufficient use of interdependent modules that we need some
> form of implicit dependencies.

I've thought about this, too, when I started working with modules.  On
systems supporting RTLD_LAZY it should be possible to have the setup_
function call back the core to load needed modules, shouldn't it?

The other thing I was thinking about is to define a string or a symbol
with a defined prefix that contains the dependencies.  The core could
then scan the module binary to find out which modules to load first
(searching for the prefix and using the rest of the string).

But if we can't make sure that the string/symbol is near the beginning
of the module file, this isn't fun, of course.


Bye
  Sven


-- 
Sven Wischnowsky                         wischnow@informatik.hu-berlin.de

RE: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

> -----Original Message-----
> From: Sven Wischnowsky [mailto:wischnow@informatik.hu-berlin.de]
> Sent: Thursday, June 07, 2001 5:17 PM
> To: zsh-workers@sunsite.dk
> Subject: Re: Modules dependencies RE: PATCH: zftp/tcp abstraction
>
>
> Zefram wrote:
>
> > ...
> >
> > Neither of my solutions is completely satisfactory, but it now
> seems that
> > we are making sufficient use of interdependent modules that we need some
> > form of implicit dependencies.
>
> I've thought about this, too, when I started working with modules.  On
> systems supporting RTLD_LAZY it should be possible to have the setup_
> function call back the core to load needed modules, shouldn't it?
>

And if we could use ELF on all plattforms we could just add dependencies to
a section in ELF file.

No, I believe, the simplest solution is to have modele.dep for every
module.so. There may be some other things to put into (think about
autoloaded stuff - it would enable you to just drop module into directory
and let zsh scan $module_path on startup to add autoloaded objects) so
general name module.def might be better. We may even use .mdd file directly
but parsing it may be tricky.

Incidentally, this is what libtool does, so it is probably no the worst
solution :-)

One question - I do not quite understand logic in load_module. It looks like

if module not found
  load module
load module

One of these 'load module' looks redundant?

-andrej

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Bart Schaefer]

On Jun 7,  5:35pm, Andrej Borsenkow wrote:
}
} No, I believe, the simplest solution is to have modele.dep for every
} module.so. There may be some other things to put into (think about
} autoloaded stuff - it would enable you to just drop module into directory
} and let zsh scan $module_path on startup to add autoloaded objects)

That is exactly what I would NOT want to have happen.  Can you imagine
the confusion if zsh automatically discovered that `mv' should be loaded
from the files module?  I don't want zsh automatically picking up any
module I haven't explicitly told it about, thank you very much.

} One question - I do not quite understand logic in load_module. It looks like
} 
} if module not found
}   load module
} load module
} 
} One of these 'load module' looks redundant?

Look closely at the `if module not found' block.  It's possible that the
module may be linked but not loaded (no "handle") in that block.  In that
case a second call to load_module() will find the module, but still needs
to perform the actual loading.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

RE: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

>
> } One question - I do not quite understand logic in load_module.
> It looks like
> }
> } if module not found
> }   load module
> } load module
> }
> } One of these 'load module' looks redundant?
>
> Look closely at the `if module not found' block.  It's possible that the
> module may be linked but not loaded (no "handle") in that block.  In that
> case a second call to load_module() will find the module, but still needs
> to perform the actual loading.
>

No, I still do not understand it. The "if no found" block is basically just
an abridged version of what happens after. Moreover, remembering what Zefram
said: "Problem is you can't call that function without loading the module,
and you can't successfully load the module until you've got that
information." And look what happens here:

    if (!(node = find_module(name, 1, &name))) {
        if (!(linked = module_linked(name)) &&
            !(handle = do_load_module(name))) {
            unqueue_signals();
            return 0;
        }

i.e. we imediately try to dlopen() a module and in the worst case it
obviously fails because no dependecies were loaded. Something is wrong here.

-andrej

RE: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

>
> Andrej Borsenkow wrote:
> >To make it work in general every module needs a standard function, let's
> >call it dep_, that is autogenerated out of moddeps and called
> before module
> >is attempted to be loaded to register dependencies.
>
> Problem is you can't call that function without loading the module, and
> you can't successfully load the module until you've got that information.
> If there were an easy solution we'd have implemented it as soon as we
> came up with the module system.
>

Here is suggested change to current module system. It is actually pretty
trivial to implement and may have additional advantages ...

- module is represented by its module definition file. It is basically mdd
file with all makefile fragments stripped and any shell code replaced by
final value. To avoid confusion with original mdd, let's call it zmd (Zsh
Module Definition). The additional property is full name of binary DSO
object, e.g. soname.

- when you open a module, zsh opens zmd file. It then loads all needed
modules (as in moddeps) and after that loads modules object itself (as in
soname)

- zmd file is always installed for both linked-in and external modules. It
eliminates need to build extra list at compile-time.

- zsh _may_ scan $module_path and autoload objects from those modules with
load=yes at startup. Or it even can do it when $module_path changes (but see
*1). This makes it possible to have a single binary and simply drop new
module in place that may be of some use for packagers.


This does not seem very hard to implement (the ugliest part is parsing of
zmd files; do we have something ready in zsh so as not to reinvent the
wheel)? Comments?

-andrej

*1 When (re)scanning $module_path the problem is, what to do with currently
loaded modules. Consider the case:

module zsh/mod1 provides foo and bar and is loaded
module zsh/mod2 provides baz and bar and is being scanned

Zsh first adds baz, then it adds bar and gets an error. We possibly have to
remove baz now (else, it will try to load zsh/mod2 on access to baz and this
will silently redefine bar).

P.S. We can even completely ignore file name and rely on name= in zmd file.
That gives natural way to directly load any module by its path name.

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Bart Schaefer]

On Jun 26,  3:26pm, Andrej Borsenkow wrote:
}
} Here is suggested change to current module system. It is actually pretty
} trivial to implement and may have additional advantages ...

Jumping way ahead:

} This does not seem very hard to implement (the ugliest part is parsing of
} zmd files; do we have something ready in zsh so as not to reinvent the
} wheel)? Comments?

I object to any solution which requires that a statically linked binary
rely upon any external files.  I can't tell for certain whether that is
the case with your suggestion, but that's the impression I get.

Your remark about having something read to parse the extra files has
given me an idea:

} - module is represented by its module definition file.
} 
} - when you open a module, zsh opens zmd file. It then loads all needed
} modules (as in moddeps) and after that loads modules object itself (as in
} soname)
} 
} - zmd file is always installed for both linked-in and external modules.
} It eliminates need to build extra list at compile-time.

Why can't the module definition file itself be a loadable module?

For every module that has dependencies, we compile a separate little
shared object that has no dependencies itself.  It contains exactly one
function, which simply calls the module loading code on each of the
dependencies of the base module and then on the base module.  The name
of this object is derived from the name of the base module, e.g. for
zsh/zftp the name might be zsh/ld/zftp (and zsh/net/ld/tcp, etc.).

When the module loading code is invoked, it first inserts the /ld/ and
looks for that module and loads it if necessary, then calls the function
to finish the loading process.  If that fails, the /ld/ module has to be
unloaded again.

Of course directly loading the /ld/ module with `zmodload' would have
the same effect, so you end up with the base module and its dependencies
loaded as well.

If it doesn't find a /ld/ module, then it can go ahead and load the base
module directly.  If the /ld/ module is already loaded, it can assume
that the base module is also loaded.  (Of course we need a different
entry point that doesn't do this check, for the function inside the /ld/
module to call.)

And the /ld/ modules would be linked into the static binary just like
any other module, so this all continues to work there.

I'm not sure about "eliminates need to build extra list at compile-time."
I would think building the list once at compile time was preferable?

There remains the question of where aliasing interacts with this, e.g.,
do aliases get resolved after adding the /ld/ in the module code?  Is
one allowed to do `zmodload -A foo/ld/zap=foo/bar', and if so does the
foo/ld/bar module get loaded upon `zmodload foo/ld/zap'?  Etc.

} - zsh _may_ scan $module_path and autoload objects from those modules with
} load=yes at startup. Or it even can do it when $module_path changes (but see
} *1). This makes it possible to have a single binary and simply drop new
} module in place that may be of some use for packagers.

If this behavior is really desirable, then let's add an /auto/ module for
each base module that needs one.  Loading the /auto/ module installs all
the autoloads for the base module and then unloads itself again.

We have `zmodload -a' still hanging about as a synonym for `zmodload -ab';
it could be overloaded to look for and load the /auto/ module, otherwise
fall back on its old behavior.

If you give zmodload the -a and -u options together, then after unloading
the base module it reloads the corresponding /auto/ module; that addresses
Andrej's (?) earlier question about autoloads disappearing when a module
is unloaded.  (Presently -u -a is the same as -u -b; I suppose the /auto/
module could have two functions, one to add all the autoloads and one to
delete them again.)

} Zsh first adds baz, then it adds bar and gets an error. We possibly
} have to remove baz now (else, it will try to load zsh/mod2 on access
} to baz and this will silently redefine bar).

This problem exists independent of the scheme for manging dependencies.
So far we've required the user to resolve it manually, and I think that
is the only right way to go.
 
} P.S. We can even completely ignore file name and rely on name= in zmd
} file. That gives natural way to directly load any module by its path
} name.

I don't quite see what you mean is "natural" about this, but obviously
this is one point my scheme doesn't address.  I think the other benefits
outweigh this.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Clint Adams]

AB> - module is represented by its module definition file. It is basically mdd
AB> file with all makefile fragments stripped and any shell code replaced by

BS> Why can't the module definition file itself be a loadable module?

What about just embedding the module dependencies in the module itself?
If it could be scanned without loading the module, it would be less
messy than maintaining "helper files".  Otherwise, couldn't one
load the module, check the dependency symbol, then unload the module
and do things in the correct order?

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Peter Stephenson]

Clint Adams wrote:
> What about just embedding the module dependencies in the module itself?
> If it could be scanned without loading the module, it would be less
> messy than maintaining "helper files".

The other advantage of this is that, as Sven has pointed out, we can handle 
systems with RTLD_LAZY by loading first and asking questions afterwards.
The difference between these two methods is then transparent to the user
--- we don't have extra files on some systems and not on others.

At least, I hope we can handle RTLD_LAZY like that... only function objects
can be lazy, data objects can't be, at least under Solaris.  There may be
cases where we have data dependencies and this won't work.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

RE: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Andrej Borsenkow]

>
> Clint Adams wrote:
> > What about just embedding the module dependencies in the module itself?
> > If it could be scanned without loading the module, it would be less
> > messy than maintaining "helper files".
>
> The other advantage of this is that, as Sven has pointed out, we
> can handle
> systems with RTLD_LAZY by loading first and asking questions afterwards.
> The difference between these two methods is then transparent to the user
> --- we don't have extra files on some systems and not on others.
>

Zeram/14786

This was my original suggestion. Zefram had arguments against it.

> At least, I hope we can handle RTLD_LAZY like that... only
> function objects
> can be lazy, data objects can't be, at least under Solaris.  There may be
> cases where we have data dependencies and this won't work.
>

I never liked inter-module data references. Is there any chance we can get
rid of them and replace by wrappers/hooks/callbacks/what-ever? At least, as
long-term goal?

-andrej

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Peter Stephenson]

"Andrej Borsenkow" wrote:
> > The other advantage of this is that, as Sven has pointed out, we
> > can handle
> > systems with RTLD_LAZY by loading first and asking questions afterwards.
> > The difference between these two methods is then transparent to the user
> > --- we don't have extra files on some systems and not on others.
> >
> 
> Zeram/14786
> 
> This was my original suggestion. Zefram had arguments against it.

I'm not sure Zefram's objection applies when RTLD_LAZY is around?  He was
saying you can't load the module until you've fixed the depencies.  In this
particular case, you can.
 
> I never liked inter-module data references. Is there any chance we can get
> rid of them and replace by wrappers/hooks/callbacks/what-ever? At least, as
> long-term goal?

I'd be entirely in favour of this.  I'm not sure how easy it is, though.
I believe (without checking) that Sven has added plenty of scope for
hooks.

-- 
Peter Stephenson <pws@csr.com>                  Software Engineer
CSR Ltd., Unit 300, Science Park, Milton Road,
Cambridge, CB4 0XL, UK                          Tel: +44 (0)1223 392070


**********************************************************************
The information transmitted is intended only for the person or
entity to which it is addressed and may contain confidential 
and/or privileged material. 
Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by 
persons or entities other than the intended recipient is 
prohibited.  
If you received this in error, please contact the sender and 
delete the material from any computer.
**********************************************************************

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Bart Schaefer]

On Jun 27,  2:59pm, Peter Stephenson wrote:
} Subject: Re: Modules dependencies RE: PATCH: zftp/tcp abstraction
}
} Clint Adams wrote:
} > What about just embedding the module dependencies in the module itself?
} > If it could be scanned without loading the module, it would be less
} > messy than maintaining "helper files".
} 
} The other advantage of this is that, as Sven has pointed out, we can handle 
} systems with RTLD_LAZY by loading first and asking questions afterwards.

In addition to Zefram's remarks in 14786, it is also the case that on
some operating systems (Apollo Domain/OS is a specific if obscure example)
all executables and libraries are compressed and checksummed.  This makes
it (a) difficult[*] to read C strings out of the library without actually
loading it and (b) impossible to add any uncompressed data to the library
file and still load it as a library.

We can of course choose not to support dynamic modules on any such OS.

[*] I'm not aware of any API for reversing the compression on Domain/OS.

-- 
Bart Schaefer                                 Brass Lantern Enterprises
http://www.well.com/user/barts              http://www.brasslantern.com

Zsh: http://www.zsh.org | PHPerl Project: http://phperl.sourceforge.net   

Re: Modules dependencies RE: PATCH: zftp/tcp abstraction

[Zefram]

I generally support Andrej's suggestion, with just a couple of issues:

Bart Schaefer wrote:
>I object to any solution which requires that a statically linked binary
>rely upon any external files.  I can't tell for certain whether that is
>the case with your suggestion, but that's the impression I get.

It's only the case if there are external .zmd files for the internally
linked modules.  If we instead build this data into the executable,
then there is no problem.

>} - when you open a module, zsh opens zmd file. It then loads all needed
>} modules (as in moddeps) and after that loads modules object itself (as in
>} soname)

Here, we must be sure to load the module object from the proper directory
-- we need the filename, not the soname.

On Bart's suggestions:

>Why can't the module definition file itself be a loadable module?

Doing that would run into issues of symbol name clashes, but it is
surmountable, as we have already solved that problem for the modules
themselves.  However, module definition files being code just seems like
a bad idea; I'd be happier with them being plain old data, that can be
manipulated as data.  It doesn't seem that we gain anything by using
a shared object here, and we'd lose some flexibility (think what other
information we might want to put in the .zmd, and other ways we'd like
to use the information).

>of this object is derived from the name of the base module, e.g. for
>zsh/zftp the name might be zsh/ld/zftp (and zsh/net/ld/tcp, etc.).

Creating extra modules that are visible in the module namespace would
be a bad idea.  At minimum, it exposes too much of the implementation,
forcing us to retain this method of handling loadable modules even if
we come up with a better way.

-zefram