modules configuration patch

modules configuration patch

[Zefram]

-----BEGIN PGP SIGNED MESSAGE-----

This module stuff is impressive.  Good work, Zoltan.

Here is a patch to fix some configuration and building problems with
the new code.  The changes made are:

* configure.in:
    Currently, if the dynamic code is supposedly *dis*abled, dlopen()
    et al are checked for anyway.  But, as -ldl hasn't been tried,
    the functions are not found.  This can put bogus information into
    a cache file.  I solve this by making *all* the dynamic-specific
    stuff conditional on the configure option.

* configure.in, Src/Makefile.in:
    The "ifdef DYNAMIC" syntax is not universal to all makes.  So I
    replaced that bit with a more portable system: @D@ expands to D if
    dynamic stuff is enabled, or N otherwise, so variable references
    like $(@D@LEXT) have varying effect.  It's hacky, but portably doing
    anything complex in a Makefile is a bit of a black art anyway.

* Src/Makefile.in
    "make clean" should also delete *.so.

* Src/mod_example.c
    Include "zsh.h", in order to get all the type definitions, prototypes
    and so on that should be available.  Modules should do this some
    way or other, in order to allow proper interface checking.

* Src/module.c
    Changed the dlclose() substitute to emulate the correct semantics
    of the function.  It doesn't make any difference to the code currently
    there, but does guarantee no surprises when maintaining it.

Some other points that occurred to me:

The initialise/cleanup interface, with varying names that depend on
the filename of the module, is really ugly.  Is there a good reason to
have varying names?  Fixed names would be easier to implement.  And the
implementations of dlopen() that I've seen will automatically do this
anyway, using the names _init and _fini: are there implementations that
don't do this?  If we can rely on _init/_fini, the underscore configure
test would be unnecessary.

The interface to modload could use some more thought.  Is the argument
a filename, or a module name?  At the moment only a pathname starting
with /, ./ or ../ is treated as a filename.  It would be more consistent
to treat any pathname containing a / as an actual pathname, and only do
the path search and .so appending when only a bare name is specified.

And finally a more general question.  Are any non-trivial modules going
to be included in the zsh distribution?

 -zefram

 Index: configure.in
 ===================================================================
 RCS file: /home/zefram/usr/cvsroot/zsh/configure.in,v
 retrieving revision 1.19
 diff -c -r1.19 configure.in
 *** configure.in	1996/11/08 01:22:56	1.19
 --- configure.in	1996/11/08 01:55:02
 ***************
 *** 256,262 ****
   AC_CHECK_HEADERS(sys/time.h sys/times.h sys/select.h termcap.h termio.h \
   		 termios.h sys/param.h sys/filio.h string.h memory.h \
   		 limits.h fcntl.h libc.h sys/utsname.h sys/resource.h \
 ! 		 locale.h errno.h stdlib.h unistd.h dlfcn.h)
   
   dnl Some SCO systems cannot include both sys/time.h and sys/select.h
   if test $ac_cv_header_sys_time_h = yes -a $ac_cv_header_sys_select_h = yes; then
 --- 256,265 ----
   AC_CHECK_HEADERS(sys/time.h sys/times.h sys/select.h termcap.h termio.h \
   		 termios.h sys/param.h sys/filio.h string.h memory.h \
   		 limits.h fcntl.h libc.h sys/utsname.h sys/resource.h \
 ! 		 locale.h errno.h stdlib.h unistd.h)
 ! if test $dynamic = yes; then
 !   AC_CHECK_HEADERS(dlfcn.h)
 ! fi
   
   dnl Some SCO systems cannot include both sys/time.h and sys/select.h
   if test $ac_cv_header_sys_time_h = yes -a $ac_cv_header_sys_select_h = yes; then
 ***************
 *** 444,450 ****
                 getlogin setpgid gettimeofday gethostname mkfifo wait3 difftime \
                 sigblock sigsetmask sigrelse sighold killpg sigaction getrlimit \
                 sigprocmask setuid seteuid setreuid setresuid strerror nis_list \
 !               initgroups dlopen dlerror dlsym dlclose)
   
   dnl  Checking for working strcoll
   AC_CACHE_CHECK(for working strcoll, zsh_cv_func_strcoll,
 --- 447,456 ----
                 getlogin setpgid gettimeofday gethostname mkfifo wait3 difftime \
                 sigblock sigsetmask sigrelse sighold killpg sigaction getrlimit \
                 sigprocmask setuid seteuid setreuid setresuid strerror nis_list \
 !               initgroups)
 ! if test $dynamic = yes; then
 !   AC_CHECK_FUNCS(dlopen dlerror dlsym dlclose)
 ! fi
   
   dnl  Checking for working strcoll
   AC_CACHE_CHECK(for working strcoll, zsh_cv_func_strcoll,
 ***************
 *** 746,758 ****
 --- 752,767 ----
   fi
   
   if test "x$dynamic" = xyes; then
 +   D=D
     DYNAMIC=yes
     AC_DEFINE(DYNAMIC)dnl
   else
 +   D=N
     DYNAMIC=
   fi
   
   AC_DEFINE_UNQUOTED(DL_EXT, "$DL_EXT")dnl
 + AC_SUBST(D)dnl
   AC_SUBST(DYNAMIC)dnl
   AC_SUBST(DL_EXT)dnl
   AC_SUBST(DLLD)dnl
 Index: Src/Makefile.in
 ===================================================================
 RCS file: /home/zefram/usr/cvsroot/zsh/Src/Makefile.in,v
 retrieving revision 1.1.1.11
 diff -c -r1.1.1.11 Makefile.in
 *** Makefile.in	1996/11/08 01:19:37	1.1.1.11
 --- Makefile.in	1996/11/08 02:30:13
 ***************
 *** 55,60 ****
 --- 55,61 ----
   INCLUDES = -I.. -I. -I$(srcdir)
   
   COMPILE = $(CC) -c $(INCLUDES) $(CPPFLAGS) $(DEFS) $(CFLAGS)
 + LINK    = $(CC) $(LDFLAGS) $(EXTRA_LDFLAGS) -o $@
   
   INSTALL         = @INSTALL@
   INSTALL_PROGRAM = @INSTALL_PROGRAM@
 ***************
 *** 62,87 ****
   AWK = @AWK@
   SED = sed
   
 ! .SUFFIXES:
 ! 
 ! ifdef DYNAMIC
 ! .SUFFIXES: .c .o .$(DL_EXT) .pro
   
 ! LINK    = $(CC) $(LDFLAGS) $(EXTRA_LDFLAGS) -o $@
   
 ! .c.$(DL_EXT):
   	$(COMPILE) $(DLCFLAGS) -o $@.o $<
   	$(DLLD) $(LDFLAGS) $(DLLDFLAGS) -o $@ $@.o
 ! 	rm $@.o
 ! 
 ! DYNAMIC_OBJS = $Umodule.o
 ! else
 ! .SUFFIXES: .c .o .pro
 ! 
 ! LINK    = $(CC) $(LDFLAGS) $(EXTRA_LDFLAGS) -o $@
 ! 
 ! DYNAMIC_OBJS =
 ! endif
   
   .c.o:
   	$(COMPILE) $<
 --- 63,78 ----
   AWK = @AWK@
   SED = sed
   
 ! DLEXT=.$(DL_EXT)
 ! NLEXT=._foobarbaz_
   
 ! .SUFFIXES:
 ! .SUFFIXES: .c .o $(@D@LEXT) .pro
   
 ! .c$(@D@LEXT):
   	$(COMPILE) $(DLCFLAGS) -o $@.o $<
   	$(DLLD) $(LDFLAGS) $(DLLDFLAGS) -o $@ $@.o
 ! 	rm -f $@.o
   
   .c.o:
   	$(COMPILE) $<
 ***************
 *** 107,132 ****
   
   # zsh C source
   SRCS = builtin.c compat.c cond.c exec.c glob.c hashtable.c hist.c init.c \
 ! input.c jobs.c lex.c linklist.c loop.c math.c mem.c module.c params.c \
 ! parse.c signals.c subst.c text.c utils.c watch.c zle_bindings.c zle_hist.c \
 ! zle_main.c zle_misc.c zle_move.c zle_refresh.c zle_tricky.c zle_utils.c \
 ! zle_vi.c zle_word.c
   
   # generated prototypes
   PROTO = builtin.pro compat.pro cond.pro exec.pro glob.pro hashtable.pro \
   hist.pro init.pro input.pro jobs.pro lex.pro linklist.pro loop.pro \
 ! math.pro mem.pro module.pro params.pro parse.pro signals.pro subst.pro \
   text.pro utils.pro watch.pro zle_bindings.pro zle_hist.pro zle_main.pro \
   zle_misc.pro zle_move.pro zle_refresh.pro zle_tricky.pro zle_utils.pro \
 ! zle_vi.pro zle_word.pro
   
   # object files
   OBJS = $Ubuiltin.o $Ucompat.o $Ucond.o $Uexec.o $Uglob.o $Uhashtable.o \
   $Uhist.o $Uinit.o $Uinput.o $Ujobs.o $Ulex.o $Ulinklist.o $Uloop.o \
   $Umath.o $Umem.o $Uparams.o $Uparse.o $Usignals.o $Usubst.o $Utext.o \
   $Uutils.o $Uwatch.o $Uzle_bindings.o $Uzle_hist.o $Uzle_main.o \
   $Uzle_misc.o $Uzle_move.o $Uzle_refresh.o $Uzle_tricky.o $Uzle_utils.o \
 ! $Uzle_vi.o $Uzle_word.o $(DYNAMIC_OBJS)
   
   # auxiliary files
   AUX = Makefile.in .indent.pro signames.awk makepro.sed ansi2knr.c TAGS tags
 --- 98,127 ----
   
   # zsh C source
   SRCS = builtin.c compat.c cond.c exec.c glob.c hashtable.c hist.c init.c \
 ! input.c jobs.c lex.c linklist.c loop.c math.c mem.c module.c mod_example.c \
 ! params.c parse.c signals.c subst.c text.c utils.c watch.c zle_bindings.c \
 ! zle_hist.c zle_main.c zle_misc.c zle_move.c zle_refresh.c zle_tricky.c \
 ! zle_utils.c zle_vi.c zle_word.c
   
   # generated prototypes
 + DYNAMIC_PROTO = module.pro
 + NYNAMIC_PROTO =
   PROTO = builtin.pro compat.pro cond.pro exec.pro glob.pro hashtable.pro \
   hist.pro init.pro input.pro jobs.pro lex.pro linklist.pro loop.pro \
 ! math.pro mem.pro params.pro parse.pro signals.pro subst.pro \
   text.pro utils.pro watch.pro zle_bindings.pro zle_hist.pro zle_main.pro \
   zle_misc.pro zle_move.pro zle_refresh.pro zle_tricky.pro zle_utils.pro \
 ! zle_vi.pro zle_word.pro $(@D@YNAMIC_PROTO)
   
   # object files
 + DYNAMIC_OBJS = $Umodule.o
 + NYNAMIC_OBJS =
   OBJS = $Ubuiltin.o $Ucompat.o $Ucond.o $Uexec.o $Uglob.o $Uhashtable.o \
   $Uhist.o $Uinit.o $Uinput.o $Ujobs.o $Ulex.o $Ulinklist.o $Uloop.o \
   $Umath.o $Umem.o $Uparams.o $Uparse.o $Usignals.o $Usubst.o $Utext.o \
   $Uutils.o $Uwatch.o $Uzle_bindings.o $Uzle_hist.o $Uzle_main.o \
   $Uzle_misc.o $Uzle_move.o $Uzle_refresh.o $Uzle_tricky.o $Uzle_utils.o \
 ! $Uzle_vi.o $Uzle_word.o $(@D@YNAMIC_OBJS)
   
   # auxiliary files
   AUX = Makefile.in .indent.pro signames.awk makepro.sed ansi2knr.c TAGS tags
 ***************
 *** 152,157 ****
 --- 147,154 ----
   
   $(PROTO): makepro.sed
   
 + mod_example.$(DL_EXT): mod_example.pro
 + 
   # ========== DEPENDENCIES FOR INSTALLING ==========
   
   install: install.bin
 ***************
 *** 242,252 ****
   
   # ========== DEPENDENCIES FOR CLEANUP ==========
   
   mostlyclean:
   	rm -f core *.o *~
   
   clean: mostlyclean
 ! 	rm -f zsh ansi2knr signames.h _*.c *.pro
   
   distclean: clean
   	rm -f Makefile
 --- 239,252 ----
   
   # ========== DEPENDENCIES FOR CLEANUP ==========
   
 + DLCLEAN = *.$(DL_EXT)
 + NLCLEAN =
 + 
   mostlyclean:
   	rm -f core *.o *~
   
   clean: mostlyclean
 ! 	rm -f zsh ansi2knr $(@D@LCLEAN) signames.h _*.c *.pro
   
   distclean: clean
   	rm -f Makefile
 Index: Src/mod_example.c
 ===================================================================
 RCS file: /home/zefram/usr/cvsroot/zsh/Src/mod_example.c,v
 retrieving revision 1.2
 diff -c -r1.2 mod_example.c
 *** mod_example.c	1996/11/08 01:48:26	1.2
 --- mod_example.c	1996/11/08 02:27:17
 ***************
 *** 29,42 ****
    *
    */
   
 ! #include <stdio.h>
 ! 
 ! #define _(X) X
 ! 
 ! typedef int (*HandlerFunc) _((char *, char **, char *, int));
 ! 
 ! void addbuiltin _((char *, int, HandlerFunc, int, int, char *));
 ! void deletebuiltin _((char *nam));
   
   /**/
   int
 --- 29,36 ----
    *
    */
   
 ! #include "zsh.h"
 ! #include "mod_example.pro"
   
   /**/
   int
 Index: Src/module.c
 ===================================================================
 RCS file: /home/zefram/usr/cvsroot/zsh/Src/module.c,v
 retrieving revision 1.1.1.1
 diff -c -r1.1.1.1 module.c
 *** module.c	1996/11/08 01:19:58	1.1.1.1
 --- module.c	1996/11/08 02:25:10
 ***************
 *** 30,35 ****
 --- 30,36 ----
    */
   
   #include "zsh.h"
 + 
   #ifdef HAVE_DLFCN_H
   # include <dlfcn.h>
   #else
 ***************
 *** 37,47 ****
   # include <nlist.h>
   # include <link.h>
   #endif
   #ifndef RTLD_LAZY
 ! #define RTLD_LAZY 1
   #endif
   #ifndef HAVE_DLCLOSE
 ! # define dlclose(X)
   #endif
   
   static LinkList modules;
 --- 38,50 ----
   # include <nlist.h>
   # include <link.h>
   #endif
 + 
   #ifndef RTLD_LAZY
 ! # define RTLD_LAZY 1
   #endif
 + 
   #ifndef HAVE_DLCLOSE
 ! # define dlclose(X) ((X), 0)
   #endif
   
   static LinkList modules;

-----BEGIN PGP SIGNATURE-----
Version: 2.6.2

iQCVAwUBMoKiWHD/+HJTpU/hAQH2NAQAmwQk//1x2QdvLAbMp4DDKgi+9anJkUfr
NCylUpZVNNG5ysxUKrE9O17GBkbkRFAkVhYx3EmLuWc8gWuaqG8KLYf9MFiEwVk7
S8a8mpL7tiRPfHKTYIiwKqWOUCvwOnpzS20Dd8k4ay4j6ssUfnQLCX56DuVwoahz
vN359JiTno4=
=mCqp
-----END PGP SIGNATURE-----

Re: modules configuration patch

[Goran Larsson]

> The interface to modload could use some more thought.  Is the argument
> a filename, or a module name?  At the moment only a pathname starting
> with /, ./ or ../ is treated as a filename.  It would be more consistent
> to treat any pathname containing a / as an actual pathname, and only do
> the path search and .so appending when only a bare name is specified.

Please no. looking for xxx/yyy in the current directory just because it
contains a / is horrible. The normal method for zsh to search for ordinary
programs is to look in the PATH for xxx/yyy stuff, why not for modules
as well? Makes it more consistent. This was added by PF after I requested
it as it is perfect for managing the PATH namespace. It would be nice
to have this for modules as well.

-- 
 Goran Larsson                        mailto:hoh@approve.se
 I was an atheist,                    http://home1.swipnet.se/%7Ew-12153/
 until I found out I was God.

Re: modules configuration patch

[Zefram]

>Please no. looking for xxx/yyy in the current directory just because it
>contains a / is horrible. The normal method for zsh to search for ordinary
>programs is to look in the PATH for xxx/yyy stuff, why not for modules
>as well? Makes it more consistent. This was added by PF after I requested
>it as it is perfect for managing the PATH namespace. It would be nice
>to have this for modules as well.

Actually both approaches are available for PATH.  The default is the
one I suggested.  Perhaps (z|)modload should honour PATH_DIRS.

Might I also suggest that .so be appended always (whether or not the
path i searched), and the filename tried without the suffix only if
that fails.  This provides the expected behaviour in all but the most
freaky cases.

-zefram

Re: modules configuration patch

[Zoltan Hidvegi]

Zefram wrote:
[...]
>     like $(@D@LEXT) have varying effect.  It's hacky, but portably doing
>     anything complex in a Makefile is a bit of a black art anyway.

I was a bit afraid of that myself.  With a better make we can hack up some
automatic rules for K&R compilers and we may delete those _file.c rules.
This hack is a bit more difficult for modules.

> * Src/mod_example.c
>     Include "zsh.h", in order to get all the type definitions, prototypes
>     and so on that should be available.  Modules should do this some
>     way or other, in order to allow proper interface checking.

Of course you are right.  Here I just wanted to demonstrare that a module
can be compiled independently from zsh.

> Some other points that occurred to me:
> 
> The initialise/cleanup interface, with varying names that depend on
> the filename of the module, is really ugly.  Is there a good reason to
> have varying names?  Fixed names would be easier to implement.  And the

If we want to statically link modules to zsh we must use unique names.
I also think that the _init/_fini functions are not always available.

> The interface to modload could use some more thought.  Is the argument
> a filename, or a module name?  At the moment only a pathname starting
> with /, ./ or ../ is treated as a filename.  It would be more consistent
> to treat any pathname containing a / as an actual pathname, and only do
> the path search and .so appending when only a bare name is specified.

I know that the interface is not good at the mement but this is the easiest
part.  I wrote this interface a few months ago just to test the
capabilities of my ELF system.  The most difficult part is to make
everything portable and allow static linking of modules.  The user
interface part is really half finished I showed it to the public just to
give you some food for thought.

> And finally a more general question.  Are any non-trivial modules going
> to be included in the zsh distribution?

I hope so.  Perl comes with several modules and I think it would make zsh
popular if it came with a few powefull module.

Zoltan

Re: modules configuration patch

[Richard Coleman]

> >     like $(@D@LEXT) have varying effect.  It's hacky, but portably doing
> >     anything complex in a Makefile is a bit of a black art anyway.
> 
> I was a bit afraid of that myself.  With a better make we can hack up some
> automatic rules for K&R compilers and we may delete those _file.c rules.
> This hack is a bit more difficult for modules.

When I rewrote the Makefiles, I spent a good deal of time trying
to find a portable way to handle the conversion for K&R compilers.
In the end, my conclusion was that using the _file.c makefile rules
was the only way to go.  Nothing else was portable.  The only
alternative would be to require people to have GNU make (which has
the ability to do some fancy rules based on pattern matching).  I would
be against this.  The current way of using _file.c makefile rules is
simple and portable.  I don't see any need to change it.

rc