[9fans] Dynamic loading et al (Was: Pegasus 2.6 is released)

[lucio@proxima.alt.za]

> it can't dynamically load libraries at run-time which
> is the normal extension mechanism for scripting languages on other
> platforms.

Let me sneak in here, irrespective of Erik's objections.

My wild attempts at getting GCC/G++ working have led me to believe
that the right idea is going to be, firstly, to get the Plan 9 kernel
to load static ELF executables and, quickly on its heels, to teach it
to load dynamically linked ELF objects.  So far, the follow on to Lua
and language interpretation, etc.  I now leave this subject about
which I know way too little for a diversion.

I have a three-step path ahead of me:

(a)	cross-build a GNU toolchain targetting Plan 9 instead of, say
	NetBSD, which is my GNU-oriented platform of choice.  The
	target binary format will be ELF, because my efforts at
	reshaping Binutils to produce Plan 9 a.out binaries were
	frustrated by an inability to grasp the complexity of the
	multiple levels of indirection (functions that call function
	through pointers that are overwritten by compiler flags :-(

	My solution does not remove the complications, but it re-uses
	the efforts of others, as ELF binaries are more or less native
	to the GNU toolchain and, hopefully, the best supported
	format.  And before I hear too many arguments against ELF, I
	have done a "careful" study of the format (as it pertains to
	the two targets I have at hand, NetBSD and Linux/Ubuntu) and
	it is not totally evil, even though there are some serious
	gaps/flaws in it.  Bottom line, I think I can use it.

(b)	Construct a start-up module for Plan 9's ELF libc (to be) that
	accepts arguments as passed by the kernel and sets up the
	execution environment required by, at this stage,
	statically-linked ELF binary modules.  I'll get back to this
	issue.  It then becomes possible to produce static ELF
	binaries and execute them under Plan 9.  Of course, all the
	Plan 9 libraries now require recompiling and it all becomes
	massive, but not much worse than previous attempts where
	similar compromises were made.

(c)	Construct a fresh version of the linking loader (I use
	NetBSD's /libexec/ld.elf_so, I'm getting closer to grasping
	more or less how it works) that does not use MMAP to do the
	job of allocating memory and loading the file image.  Once
	again, once this is coded, the kernel will need adjustment to
	trigger its use and, on the other end, a special start-up
	module to connect the kernel and the linking loader correctly.

Now, the tricky parts.  For one, even though assembler (as ageneric
development tool for small applications) is a favourite of mine and I
always felt I would be at home in any variation thereof, I'm having
enormous trouble with the AT&T variations both Plan 9 and the GNU
toolchain use.  Most likely because I have yet to find a comprehensive
manual as I used to be acquainted with when I dealt with Univac, IBM
and Intel Assemblers (to name a few, to me, remarkable ones).

That means that I am embarrassingly stumped both to understand the
existing code and to write fresh code to serve the purposes I
mentioned above.

In case it is not clear: HELP!  I need some tutoring to produce the
start-up modules.  I think there are only two: one for the linking
loader, one for ELF binaries.  I'm hoping that dynamic and static
linking behave identically, someone here may know better.

A totally distinct issue on which I am willing to accept advice is
whether Plan 9 actually needs an ELF version (statically linked) of
the linking loader.  The initial version will be a.out, why should it
not be when I'm re-writing it from scratch?  But an ELF version may
have many advantages, of which needing a third version of the very
same start-up module is not one :-(

Anyway, anyone (besides Cinap with whom I am already in touch), feel
free to mail me any helpful suggestions.

++L

PS: I may be missing something in the cross-compilation step, I have
not dug particularly deep in that direction.  I know David Hogan
needed the APE libraries to be able to generate the GNU toolchain
using itself.  I'm about to find out how hard it all is, although
that's my second step, the first is to produce the cross-toolchain on
NetBSD.  If anyone can save me some failures, please do.

PPS: And after all this, two plugs: firstly, a grateful thank you to
Russ Cox and Cinap Lenrek (I had a chance to discover his real name at
Volos, but I missed it) for providing a lot of guidance with the Linux
emulation code they produced and continue to support.  Secondly, Cinap
and I agree that based on the Linux emulation it becomes useful to
implement MMAP on Plan 9.  So don't be surprised if that is also going
to happen with a little bit of encouragement and assistance.  Oh,
Cinap did a pretty adequate job of MMAP in the Linux emulation (I
believe it postdates Russ' version, but I'm open to historical
correction) and he should be praised for doing it.