zsh-3.1 development: dynamic modules

zsh-3.1 development: dynamic modules

[Zoltan Hidvegi]

I have stared working on binary module support in zsh.  The preliminary
version is almost usable.  I tested it on ELF Linux SunOS and Solaris and
it seems to work.  Of course the most difficult part is to make it
portable.  I looked at the perl sources for that but I not yet added
everything which is there.

You can find my current version as a patch to zsh-3.0.1 in
ftp://ftp.cs.elte.hu/pub/zsh/testing/zsh-3.1.0-test1.diff.gz

The file date/size:

   15793 Nov  6 22:19 zsh-3.1.0-test1.diff.gz

md5sum:

2b83517e832b9318bc02016ff508ab23  zsh-3.1.0-test1.diff.gz

This is very experimental.  You have to give the --enable-dynamic option to
configure to compile in binary module support.  This adds a new builtin,
modload.  Whithout arguments it lists the currently loaded modules.
Otherwise it loads the given module.  The $module_path array is user to
seatch for the modules.  modload -u modue unloads the given module.

An example module is provided in mod_example.c.  Compile it with

make mod_example.so

Mosules may work on all systems with an ANSI C compiler and dlopen/dlsym
interface.  Other systems need some hacking.

This is an experimental code and I release it only to get it moving.  I've
done this several weeks ago but I had not much time to continue the work
and I'd like to hear others opinions about it.

At the moment there is no support to link a module statically to zsh.  Also
K&R C compilers cannot be used to compile the modules.

When the module code works it can provide completely new and exciting
extensions to zsh: database extensions, X windows etc.

Any new program may be built around zsh using zsh as a user interface and
command language.

Zoltan

Re: zsh-3.1 development: dynamic modules

[Peter Stephenson]

Zoltan Hidvegi wrote:
> I have stared working on binary module support in zsh.  The preliminary
> version is almost usable.  I tested it on ELF Linux SunOS and Solaris and
> it seems to work.  Of course the most difficult part is to make it
> portable.  I looked at the perl sources for that but I not yet added
> everything which is there.

Success on IRIX 5.3 (which is ELF) using gcc (haven't tried the native
compiler yet) with the following minor changes: (1) gcc didn't like
the hacked-up version of the prototype ANSIfier, it doesn't strip
enough parentheses (2) -shared was required on linking the example
module to prevent it trying to find `main' etc.  I would suggest that
all ELF/gcc combinations use -shared; maybe it's always necessary with
gcc but I don't know how that interacts with ld in the general case.
I've also added what must be a missing $ before a GCC test a few lines
later.  autoconf's not working here at the moment so I haven't
actually used the patched configure.in.

*** Src/mod_example.c.old	Thu Nov  7 16:45:55 1996
--- Src/mod_example.c	Thu Nov  7 16:39:28 1996
***************
*** 31,37 ****
  
  #include <stdio.h>
  
! #define _(X) (X)
  
  typedef int (*HandlerFunc) _((char *, char **, char *, int));
  
--- 31,37 ----
  
  #include <stdio.h>
  
! #define _(X) X
  
  typedef int (*HandlerFunc) _((char *, char **, char *, int));
  
*** configure.in.old	Thu Nov  7 16:46:45 1996
--- configure.in	Thu Nov  7 17:00:50 1996
***************
*** 666,675 ****
    DL_EXT="${DL_EXT=so}"
    if test $zsh_cv_sys_elf = yes; then
      DLLD="${DLLD=$CC}"
    else
      DLLD="${DLLD=ld}"
    fi
!   if test -n "GCC"; then
      DLCFLAGS="${DLCFLAGS=-fpic}"
    else
      case "$host_os" in
--- 666,678 ----
    DL_EXT="${DL_EXT=so}"
    if test $zsh_cv_sys_elf = yes; then
      DLLD="${DLLD=$CC}"
+     if test -n "$GCC"; then
+       DLLDFLAGS="${DLLDFLAGS=-shared}"
+     fi
    else
      DLLD="${DLLD=ld}"
    fi
!   if test -n "$GCC"; then
      DLCFLAGS="${DLCFLAGS=-fpic}"
    else
      case "$host_os" in

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77413
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: zsh-3.1 development: dynamic modules

[Bart Schaefer]

On Nov 6, 10:38pm, Zoltan Hidvegi wrote:
} Subject: zsh-3.1 development: dynamic modules
}
} You can find my current version as a patch to zsh-3.0.1 in
} ftp://ftp.cs.elte.hu/pub/zsh/testing/zsh-3.1.0-test1.diff.gz

The ChangeLog included with that patch lists several 3.0.1 patches that
don't actually seem to be present in the patch, although others are (as
evidenced by failure of the above patch to apply to my patched 3.0.1
sources).  The missing ones I spotted while perusing diffs included the
fixes for "." and ".." in directory scans.

Did you intentionally leave some things out of this patch?

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

(Apology) Re: zsh-3.1 development: dynamic modules

[Bart Schaefer]

Oops.  Never mind.  I got some stray *.orig files included in my diff.
Zoltan's patch is OK.

-- 
Bart Schaefer                             Brass Lantern Enterprises
http://www.well.com/user/barts            http://www.nbn.com/people/lantern

Re: zsh-3.1 development: dynamic modules

[Peter Stephenson]

Zoltan Hidvegi wrote:
> This is very experimental.  You have to give the --enable-dynamic option to
> configure to compile in binary module support.  This adds a new builtin,
> modload.  Whithout arguments it lists the currently loaded modules.
> Otherwise it loads the given module.  The $module_path array is user to
> seatch for the modules.  modload -u modue unloads the given module.

I just remembered... modload is an existing command on solaris to load
kernel modules.  It's not something the average user is going to be
doing, but I suggest it's probably better to start on the right foot
and avoid a clash and associated confusion (suppose the user looks for
a manual page on modload).  How about zmodload?  That's more explicit
about it being a zsh module.

*** Src/hashtable.h.old	Fri Nov  8 09:38:02 1996
--- Src/hashtable.h	Fri Nov  8 09:38:41 1996
***************
*** 304,310 ****
  #endif
  
  #ifdef DYNAMIC
!     {NULL, "modload", 0, bin_modload, 0, -1, 0, "uf", NULL},
  #endif
      {NULL, "popd", 0, bin_cd, 0, 2, BIN_POPD, NULL, NULL},
      {NULL, "print", BINF_PRINTOPTS, bin_print, 0, -1, BIN_PRINT, "RDPnrslzNu0123456789pioOcm-", NULL},
--- 304,310 ----
  #endif
  
  #ifdef DYNAMIC
!     {NULL, "zmodload", 0, bin_zmodload, 0, -1, 0, "uf", NULL},
  #endif
      {NULL, "popd", 0, bin_cd, 0, 2, BIN_POPD, NULL, NULL},
      {NULL, "print", BINF_PRINTOPTS, bin_print, 0, -1, BIN_PRINT, "RDPnrslzNu0123456789pioOcm-", NULL},
*** Src/module.c.old	Fri Nov  8 09:38:03 1996
--- Src/module.c	Fri Nov  8 09:38:26 1996
***************
*** 127,133 ****
  
  /**/
  int
! bin_modload(char *nam, char **args, char *ops, int func)
  {
      LinkNode node;
      Module m = NULL;
--- 127,133 ----
  
  /**/
  int
! bin_zmodload(char *nam, char **args, char *ops, int func)
  {
      LinkNode node;
      Module m = NULL;

-- 
Peter Stephenson <pws@ifh.de>       Tel: +49 33762 77366
WWW:  http://www.ifh.de/~pws/       Fax: +49 33762 77413
Deutches Electronen-Synchrotron --- Institut fuer Hochenergiephysik Zeuthen
DESY-IfH, 15735 Zeuthen, Germany.

Re: zsh-3.1 development: dynamic modules

[Andrej Borsenkow]

I tried it on Sinix 5.42. It is SVR4-derived system with -ldl library. At
least Perl 5.003 DynaLoad works O.K.

There are many problems about autoconf, but it could be solved in future.
The crucial problem is:

  on some systems (our at least; I take it to be true for other SVR4 as
  well) the statically linked symbols are *not* available to dynamic
  loader. I am not aware of any options to change it. Applied to 
  ZSH it means, that `addbuiltin' command (when called from loaded module)
  *is not found* :-(

Note, that Perl is built as two parts - "startup" command and library,
which contains actual Perl code. I think, the above problem is one reason
for it. Giving long development cycle of Perl, it probably the only
portable solution.

It means, that ZSH should be splitted in small kernel and (probably
several) dynamically loaded modules. At least those sysmbols, which need
to be accessable from modules, should be in dyna part. But I think, if
such rewrite is ever necessary, it would be nice to modularize code; as
example, whole zle_* part could probably be made separate module (and
loaded only in interactive shell).

BTW code of Tcl 7.5 is in *.so library as well.

I would be glad to here, that I am wrong.

greetings

-------------------------------------------------------------------------
Andrej Borsenkow 		Fax:   +7 (095) 252 01 05
SNI ITS Moscow			Tel:   +7 (095) 252 13 88

NERV:  borsenkow.msk		E-Mail: borsenkow.msk@sni.de
-------------------------------------------------------------------------

Re: zsh-3.1 development: dynamic modules

[Roderick Schertler]

On Sun, 10 Nov 1996 13:59:34 +0300 (MOW), Andrej Borsenkow <borsenkow.msk@sni.de> said:
> 
>   on some systems (our at least; I take it to be true for other SVR4
>   as well) the statically linked symbols are *not* available to
>   dynamic loader.

I'm practically certain that this is the way all SVR4 systems behave.

>   I am not aware of any options to change it.

Neither I.  It's for this reason that Perl has to build libperl.so
rather than libperl.a on SVR4 systems.

> I would be glad to [know?] here, that I am wrong.

Me, too, but oh well.

-- 
Roderick Schertler
roderick@gate.net