loading dynamic modules in a static shell

loading dynamic modules in a static shell

[Vincent]

Hi.

Is it supposed to be possible to compile a static shell with part of
the modules built in and to be able to load the remaining modules with
zmodload?  Having a separate link option for every module in
config.modules gives me the impression that I should be able to create
a mixed configuration, with some static and some dynamic, but I have
not been able to get it to work.  I don't know if there is another way
I am supposed to do it but If I tried setting some with link=dynamic
and some with link=static in config.modules it would not work.  It
only compiled either static or dynamic based on how the ldflags were
set when it was configured with configure.

I compiled zsh-4.0.9 configured with
"configure --enable-ldflags=-static", installed it, then reconfigured
it as shared, did a "make modules" and installed the dynamic modules.
However, when I try loading one of the dynamic modules that are not
statically linked in, such as zftp, it just says
"zsh: failed to load module: zsh/zftp".

I have searched the FAQ, mailing lists, and documentation, but have
not found any information that helps me.

I built a static shell to install in /bin on FreeBSD but did not want
it be so big with every module compiled in.

-- 
Avoid the VeriSign/Network Solutions domain registration trap!
http://www.InetAddresses.net

Re: loading dynamic modules in a static shell

[Dominic Mitchell]

On Sun, Jan 18, 2004 at 06:14:42AM -0600, Vincent wrote:
> Is it supposed to be possible to compile a static shell with part of
> the modules built in and to be able to load the remaining modules with
> zmodload?  Having a separate link option for every module in
> config.modules gives me the impression that I should be able to create
> a mixed configuration, with some static and some dynamic, but I have
> not been able to get it to work.  I don't know if there is another way
> I am supposed to do it but If I tried setting some with link=dynamic
> and some with link=static in config.modules it would not work.  It
> only compiled either static or dynamic based on how the ldflags were
> set when it was configured with configure.
> 
> I compiled zsh-4.0.9 configured with
> "configure --enable-ldflags=-static", installed it, then reconfigured
> it as shared, did a "make modules" and installed the dynamic modules.
> However, when I try loading one of the dynamic modules that are not
> statically linked in, such as zftp, it just says
> "zsh: failed to load module: zsh/zftp".
> 
> I have searched the FAQ, mailing lists, and documentation, but have
> not found any information that helps me.
> 
> I built a static shell to install in /bin on FreeBSD but did not want
> it be so big with every module compiled in.

I think it depends upon the capabilities of the platform's dynamic
loader.  Certainly for FreeBSD you won't be able to do this.  This is
because all dynamic loading is disabled when a static binary is used.
For the current release (5.2) the root filesystem has been made dynamic
in order to get this working (for the nsswitch stuff to work correctly).

I'm trying to remember the reasons why this is the case, but I can't
quite recall them.  But if you search the freebsd-current mailing list
archives for "dynamic root", that should provoke some enlightening
discussion.

-Dom

Re: loading dynamic modules in a static shell

[Vincent]

On Sun, Jan 18, 2004 at 03:31:07PM +0000, Dominic Mitchell wrote:
> On Sun, Jan 18, 2004 at 06:14:42AM -0600, Vincent wrote:
> > Is it supposed to be possible to compile a static shell with part of
> > the modules built in and to be able to load the remaining modules with
> > zmodload?

> > I compiled zsh-4.0.9 configured with
> > "configure --enable-ldflags=-static", installed it, then reconfigured
> > it as shared, did a "make modules" and installed the dynamic modules.
> > However, when I try loading one of the dynamic modules that are not
> > statically linked in, such as zftp, it just says
> > "zsh: failed to load module: zsh/zftp".
> >
> > I built a static shell to install in /bin on FreeBSD but did not want
> > it be so big with every module compiled in.
> 
> I think it depends upon the capabilities of the platform's dynamic
> loader.  Certainly for FreeBSD you won't be able to do this.  This is
> because all dynamic loading is disabled when a static binary is used.
> For the current release (5.2) the root filesystem has been made dynamic
> in order to get this working (for the nsswitch stuff to work correctly).

Thanks for the reply.  I did not realize that.  I could have sworn I
remembered compiling programs before with mixed linking, where some
libs I specified as "/usr/lib/xxx.a" while others were dynamic,
specified with "-lxxx".  Perhaps that was when I was running Linux.

I guess, for now, I just have to make a bloated executable in order to
have a statically linked shell.  Perhaps I will try again later when I
upgrade from FreeBSD-5.1 to 5.2.

> I'm trying to remember the reasons why this is the case, but I can't
> quite recall them.  But if you search the freebsd-current mailing list
> archives for "dynamic root", that should provoke some enlightening
> discussion.
> 
> -Dom

I scanned through some of the postings.  It did enlighten me on
another unrelated issue.  I was wandering why a lot of the newer ports
were failing to run because they were linked using
/libexec/ld-elf.so.1 rather than /usr/libexec/ld-elf.so.1.  Apparently
it is related to this changeover to a "dynamic root" that you
mentioned.  I though some of the ports maintainers were just goofing
up :-).

Thanks.
Vincent

-- 
Avoid the VeriSign/Network Solutions domain registration trap!
http://www.InetAddresses.net

Re: loading dynamic modules in a static shell

[Dan Nelson]

In the last episode (Jan 18), Vincent said:
> On Sun, Jan 18, 2004 at 03:31:07PM +0000, Dominic Mitchell wrote:
> > On Sun, Jan 18, 2004 at 06:14:42AM -0600, Vincent wrote:
> > > Is it supposed to be possible to compile a static shell with part
> > > of the modules built in and to be able to load the remaining
> > > modules with zmodload?
> > 
> > I think it depends upon the capabilities of the platform's dynamic
> > loader.  Certainly for FreeBSD you won't be able to do this.  This
> > is because all dynamic loading is disabled when a static binary is
> > used. For the current release (5.2) the root filesystem has been
> > made dynamic in order to get this working (for the nsswitch stuff
> > to work correctly).
> 
> Thanks for the reply.  I did not realize that.  I could have sworn I
> remembered compiling programs before with mixed linking, where some
> libs I specified as "/usr/lib/xxx.a" while others were dynamic,
> specified with "-lxxx".  Perhaps that was when I was running Linux.

You can do this, but note that this isn't a static binary; it's a
dynamic binary with some libraries loaded statically.  You'll still
have a dependency on ld.so, and depending on which libraries you pulled
in statically, you may have issues dlopen'ing shared objects that also
depend on those libraries.  Statically-linking a library only pulls in
the object files required for the link to complete, so if you link with
libc.a, you won't have all of libc in your executable.  If your shared
object also depends on libc (most would, I guess), it will dlopen
libc.so, but now you will have two copies of some functions and
structures in libc, and constructors may fire twice.

I did some googling, and I don't think Linux or Solaris like dlopening
from static binaries, either.

-- 
	Dan Nelson
	dnelson@allantgroup.com

Re: loading dynamic modules in a static shell

[Bart Schaefer]

On Jan 18,  6:14am, Vincent wrote:
> 
> Is it supposed to be possible to compile a static shell with part of
> the modules built in and to be able to load the remaining modules with
> zmodload?

Yes; in fact the default is for zsh/main to be static-linked and all the
other modules to be dynamic.

> not been able to get it to work.  I don't know if there is another way
> I am supposed to do it but If I tried setting some with link=dynamic
> and some with link=static in config.modules it would not work.

That's the way you're supposed to do it.  However, I think you may be a
little confused about the procedure:

> It only compiled either static or dynamic based on how the ldflags
> were set when it was configured with configure. I compiled zsh-4.0.9
> configured with "configure --enable-ldflags=-static"

That's the first mistake.  You want one of

	configure --enable-dynamic			(the default)
or
	configure --disable-dynamic

Do not specify the -static or -shared ldflags yourself directly; they are
independent of whether individual modules are static-linked.

For a mix of static and dynamic modules, you must start by configuring a
dynamic shell, then edit config.modules to select the modules that you
want linked static.

> installed it, then reconfigured it as shared, did a "make modules" and
> installed the dynamic modules. However, when I try loading one of the
> dynamic modules that are not statically linked in, such as zftp, it
> just says "zsh: failed to load module: zsh/zftp".

Did you ever actually edit config.modules to set "link=static" for zftp?
If so, did you also set "auto=no" so that re-running configure would not
clobber your changes?

> I have searched the FAQ, mailing lists, and documentation, but have
> not found any information that helps me.

The INSTALL file should tell you just about everything you need to know.

Re: loading dynamic modules in a static shell

[Vincent]

On Sun, Jan 18, 2004 at 10:39:01AM -0800, Bart Schaefer wrote:
> On Jan 18,  6:14am, Vincent wrote:
> > 
> > Is it supposed to be possible to compile a static shell with part of
> > the modules built in and to be able to load the remaining modules with
> > zmodload?
> 
> Yes; in fact the default is for zsh/main to be static-linked and all the
> other modules to be dynamic.
> 
> > not been able to get it to work.  I don't know if there is another way
> > I am supposed to do it but If I tried setting some with link=dynamic
> > and some with link=static in config.modules it would not work.
> 
> That's the way you're supposed to do it.  However, I think you may be a
> little confused about the procedure:
> 
> > It only compiled either static or dynamic based on how the ldflags
> > were set when it was configured with configure. I compiled zsh-4.0.9
> > configured with "configure --enable-ldflags=-static"
> 
> That's the first mistake.  You want one of
> 
> 	configure --enable-dynamic			(the default)
> or
> 	configure --disable-dynamic
> 
> Do not specify the -static or -shared ldflags yourself directly; they are
> independent of whether individual modules are static-linked.
> 
> For a mix of static and dynamic modules, you must start by configuring a
> dynamic shell, then edit config.modules to select the modules that you
> want linked static.

Hi Bart.  Thanks for the response.

The problem is, if I start by configuring a dynamic shell, then it is
dynamically linked with the system libraries.  ie.
    libncurses.so.5 => /usr/lib/libncurses.so.5 (0x2806b000)
    libm.so.2 => /usr/lib/libm.so.2 (0x280ab000)
    libc.so.5 => /usr/lib/libc.so.5 (0x280c8000)

I was trying to create a static shell for /bin on FreeBSD but I wanted
to still have part of the modules dynamically loaded to keep it from
being bloated.

> 
> > installed it, then reconfigured it as shared, did a "make modules" and
> > installed the dynamic modules. However, when I try loading one of the
> > dynamic modules that are not statically linked in, such as zftp, it
> > just says "zsh: failed to load module: zsh/zftp".
> 
> Did you ever actually edit config.modules to set "link=static" for zftp?
> If so, did you also set "auto=no" so that re-running configure would not
> clobber your changes?
> 
> > I have searched the FAQ, mailing lists, and documentation, but have
> > not found any information that helps me.
> 
> The INSTALL file should tell you just about everything you need to know.

Yes, I experimented with editing the config.modules file but, as you
mentioned, I could not get it to create any dynamic modules if I
configured the shell to be static.  The INSTALL file just mentions
dynamic vs. static modules.  It does not discuss creating a static
shell that is not dynamically linked with the system libraries.

Regards,
Vincent

-- 
Avoid the VeriSign/Network Solutions domain registration trap!
http://www.InetAddresses.net