Re: [9fans] Binary format

Re: [9fans] Binary format

[Jacob Todd]

[-- Attachment #1: Type: text/plain, Size: 1446 bytes --]

On Wed, Feb 17, 2010 at 02:31:09PM +0100, Enrico Weigelt wrote:
> 
> Hi folks,
> 
> just curious: which binfmt does Plan9 use ?
> How are share libraries handled (if they exist at all) ?
> 
> Inspired by recent discussions @ gentoo-user, I'm thinking a bit
> how an simple and efficient binfmt could look like. Some key
> ideas are:
> 
> * purely runtime information (no debug stuff, etc)
> * everthing's sharable/relocatable module, with strict dependencies
> * on exec() the process image will be constructed the modules along
>   the dependency tree (the main program as root)
> * each module may have an entry point (main module w/o is allowed,
>   even if it wouldn't make much sense ;-o), these are called after
>   relocation, along the dependency tree, from leaf to root.
> * the whole loader sits in the kernel (maybe w/ some additional
>   helper deamon in userland), but userland can pass parameters
>   like search pathes, etc via env.
> 
> IMHO having the dynamic loader in kernel-land (in contrary to ELF 
> on GNU) not just removes the need for lots of syscalls, but allows
> a lot of optimizations, eg. code sharing w/ mmap() or caching
> relocation data.
> 
> 
Where was this thread at? I'm subscribed to gentoo-user, but I don't have
any threads about binary formats in my inbox. Was the discussion in
another thread (maybe the giant HAL thread)?

-- 
I am a man who does not exist for others.

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

[9fans] Binary format

[Enrico Weigelt]

Hi folks,

just curious: which binfmt does Plan9 use ?
How are share libraries handled (if they exist at all) ?

Inspired by recent discussions @ gentoo-user, I'm thinking a bit
how an simple and efficient binfmt could look like. Some key
ideas are:

* purely runtime information (no debug stuff, etc)
* everthing's sharable/relocatable module, with strict dependencies
* on exec() the process image will be constructed the modules along
  the dependency tree (the main program as root)
* each module may have an entry point (main module w/o is allowed,
  even if it wouldn't make much sense ;-o), these are called after
  relocation, along the dependency tree, from leaf to root.
* the whole loader sits in the kernel (maybe w/ some additional
  helper deamon in userland), but userland can pass parameters
  like search pathes, etc via env.

IMHO having the dynamic loader in kernel-land (in contrary to ELF
on GNU) not just removes the need for lots of syscalls, but allows
a lot of optimizations, eg. code sharing w/ mmap() or caching
relocation data.


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[Enrico Weigelt]

* Jacob Todd <jaketodd422@gmail.com> wrote:

> Where was this thread at? I'm subscribed to gentoo-user, but I don't have
> any threads about binary formats in my inbox. Was the discussion in
> another thread (maybe the giant HAL thread)?

Exactly this one ;-p

Oh, and we also talk about fat vs. tiny libraries, etc.
Maybe you'd like to join in ;-)


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[erik quanstrom]

> Oh, and we also talk about fat vs. tiny libraries, etc.
> Maybe you'd like to join in ;-)

please remember that this isn't a linux list.

- erik

Re: [9fans] Binary format

[Gorka Guardiola]

On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> wrote:
>
> Hi folks,
>
> just curious: which binfmt does Plan9 use ?
> How are share libraries handled (if they exist at all) ?

a.out(6)
no shared libraries.

>
> Inspired by recent discussions @ gentoo-user, I'm thinking a bit
> how an simple and efficient binfmt could look like. Some key
> ideas are:
>
> * purely runtime information (no debug stuff, etc)

just strip it.

> * everthing's sharable/relocatable module, with strict dependencies
> * on exec() the process image will be constructed the modules along
>  the dependency tree (the main program as root)

no shared libraries.

> * each module may have an entry point (main module w/o is allowed,
>  even if it wouldn't make much sense ;-o), these are called after
>  relocation, along the dependency tree, from leaf to root.

no modules.


-- 
- curiosity sKilled the cat

Re: [9fans] Binary format

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1236 bytes --]

And all of the below are some of the reasons I still love plan 9.
 Simplicity first is a really great way to work (I've learned over the
years).

Dave

On Wed, Feb 17, 2010 at 6:06 AM, Gorka Guardiola <paurea@gmail.com> wrote:

> On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> >
> > Hi folks,
> >
> > just curious: which binfmt does Plan9 use ?
> > How are share libraries handled (if they exist at all) ?
>
> a.out(6)
> no shared libraries.
>
> >
> > Inspired by recent discussions @ gentoo-user, I'm thinking a bit
> > how an simple and efficient binfmt could look like. Some key
> > ideas are:
> >
> > * purely runtime information (no debug stuff, etc)
>
> just strip it.
>
> > * everthing's sharable/relocatable module, with strict dependencies
> > * on exec() the process image will be constructed the modules along
> >  the dependency tree (the main program as root)
>
> no shared libraries.
>
> > * each module may have an entry point (main module w/o is allowed,
> >  even if it wouldn't make much sense ;-o), these are called after
> >  relocation, along the dependency tree, from leaf to root.
>
> no modules.
>
>
> --
> - curiosity sKilled the cat
>
>

[-- Attachment #2: Type: text/html, Size: 1789 bytes --]

Re: [9fans] Binary format

[Enrico Weigelt]

* Gorka Guardiola <paurea@gmail.com> wrote:
> On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> >
> > Hi folks,
> >
> > just curious: which binfmt does Plan9 use ?
> > How are share libraries handled (if they exist at all) ?
>
> a.out(6)
> no shared libraries.

ah, already suspected that ;-o

I know that plan9's standard libraries are so small, that they
dont really need to be shared. But when more and more applications
von *nix world get ported, the problem might arise again.

And another important feature of shared libraries is, that when
some lib is updated, importing programs dont have to be recompiled.
What's the Plan9 solution here ?


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[blstuart]

> * the whole loader sits in the kernel (maybe w/ some additional
>   helper deamon in userland), but userland can pass parameters
>   like search pathes, etc via env.
>
> IMHO having the dynamic loader in kernel-land (in contrary to ELF
> on GNU) not just removes the need for lots of syscalls, but allows
> a lot of optimizations, eg. code sharing w/ mmap() or caching
> relocation data.

If you're interested in how to do dynamic loading in a clean
and elegant way, take a look at Inferno.  Along the lines of
what you're suggesting, everything is a module.  The only
thing necessary to make a module suitable as a program to
run from the command line is the existence of a function
called init() with a particular signature.  After studying and
working with it, all of the other dynamic loading implementations
out there are... well the word that comes to mind is 'disgusting.'

BLS

Re: [9fans] Binary format

[Steve Simon]

> And another important feature of shared libraries is, that when
> some lib is updated, importing programs dont have to be recompiled.
> What's the Plan9 solution here ?

We recompile the relevant executables. The speed of kencc makes this
much less painful than you might expect. It also happens very rarely
on plan9 - I cannot remember the last time we had a "big" pull.

It also neatly sidesteps the issue that different
applications can need different versions of a single
shared library. This is a real problem on some OSs
(see 9fans passim).

-Steve

Re: [9fans] Binary format

[Enrico Weigelt]

* blstuart@bellsouth.net <blstuart@bellsouth.net> wrote:

> If you're interested in how to do dynamic loading in a clean
> and elegant way, take a look at Inferno.

hmm, isnt this an interpreter-based system ?

> Along the lines of what you're suggesting, everything is a module.
> The only thing necessary to make a module suitable as a program to
> run from the command line is the existence of a function
> called init() with a particular signature.

yes, definitivly a good way. reminds me a bit on Oberon, where
commands are simply parameter-less procedures in some module.


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[Enrico Weigelt]

* Steve Simon <steve@quintile.net> wrote:

> We recompile the relevant executables. The speed of kencc makes this
> much less painful than you might expect. It also happens very rarely
> on plan9 - I cannot remember the last time we had a "big" pull.

Okay, but then (as an admin) you have to know which apps have
to be recompiled. For a small system this might be okay, but
that doesnt scale well ;-o

A possible solution could be an package management system which
knows dependencies and so can tell what to rebuild.

> It also neatly sidesteps the issue that different applications
> can need different versions of a single shared library. This
> is a real problem on some OSs (see 9fans passim).

Actually, that's just a matter of clean dependency handling.
Include an API/ABI version in the filename, etc.


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[Robert Raschke]

[-- Attachment #1: Type: text/plain, Size: 1681 bytes --]

On Wed, Feb 17, 2010 at 2:33 PM, Enrico Weigelt <weigelt@metux.de> wrote:

> * Gorka Guardiola <paurea@gmail.com> wrote:
> > On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de>
> wrote:
> > >
> > > Hi folks,
> > >
> > > just curious: which binfmt does Plan9 use ?
> > > How are share libraries handled (if they exist at all) ?
> >
> > a.out(6)
> > no shared libraries.
>
> ah, already suspected that ;-o
>
> I know that plan9's standard libraries are so small, that they
> dont really need to be shared. But when more and more applications
> von *nix world get ported, the problem might arise again.
>

Well, I know that it is very tempting to "just" port across lots of
programs. But if you are looking to solve an actual problem (in a business
or "just do it" sense) and there is nothing on Plan 9 to help you with that,
then use Linux or Windows or OSX or ...

On the other hand, if you are really interested in providing functionality
to be used within Plan 9 that I'd strongly suggest investigating more than a
port. Re-implementing using the appropriate Plan 9 approach would be way
more beneficial.


> And another important feature of shared libraries is, that when
> some lib is updated, importing programs dont have to be recompiled.
> What's the Plan9 solution here ?
>

:-) Recompile.

(The usual argument to counter this approach is that security updates to
underlying libs are easier to get out across to everyone, while at the same
time completely ignoring the depths of DLL Hell and the myriads of
wonderfull new ways to break perfectly fine software all over the place. Me,
cynical, noooo! :-)


Robby

[-- Attachment #2: Type: text/html, Size: 2372 bytes --]

Re: [9fans] Binary format

[blstuart]

> * blstuart@bellsouth.net <blstuart@bellsouth.net> wrote:
>
>> If you're interested in how to do dynamic loading in a clean
>> and elegant way, take a look at Inferno.
>
> hmm, isnt this an interpreter-based system ?

The application language is Limbo which is compiled to
Dis machine code.  Dis is run in a virtual machine in the
kernel.  If you want, you can turn on a JIT compiler which
translates the Dis code to native code on the fly, but I
haven't usually found it to be all that much benefit.

BLS

Re: [9fans] Binary format

[erik quanstrom]

> > We recompile the relevant executables. The speed of kencc makes this
> > much less painful than you might expect. It also happens very rarely
> > on plan9 - I cannot remember the last time we had a "big" pull.
>
> Okay, but then (as an admin) you have to know which apps have
> to be recompiled. For a small system this might be okay, but
> that doesnt scale well ;-o

have you installed plan 9?

- erik

Re: [9fans] Binary format

[blstuart]

>> We recompile the relevant executables. The speed of kencc makes this
>> much less painful than you might expect. It also happens very rarely
>> on plan9 - I cannot remember the last time we had a "big" pull.
>
> Okay, but then (as an admin) you have to know which apps have
> to be recompiled. For a small system this might be okay, but
> that doesnt scale well ;-o

Keep in mind that Plan 9 isn't a new OS just waiting for thousands
of GNU apps to be ported.  Plan 9 is over 20 years old and has
a community that's developed native apps and a few selected
ports for the things the community finds useful.  So the kind of
apps explosion that would require it to scale well is anathama to
the culture.  Certainly there are things that would be welcomed
to be be written, and a few even ported.  But having 72 different
X11 widget "toolboxes" and 42 different GUI versions of diff
will meet with a resounding thud.  Eye candy, complexity, and
mainstream popularity don't impress the people in the Plan 9
community; elegance and originality do.

But beyond that, the main distribution includes both binaries
and source for updates.  So a pull will generally get everything
you need without needing to recompile.  And if you do want to
recompile everything, running mk in /sys/src will recompile
everything and not take all that long doing it, especially if you
keep the object files around.

BLS

Re: [9fans] Binary format

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

On Wed, Feb 17, 2010 at 6:55 AM, Steve Simon <steve@quintile.net> wrote:

> > And another important feature of shared libraries is, that when
> > some lib is updated, importing programs dont have to be recompiled.
> > What's the Plan9 solution here ?
>
> We recompile the relevant executables. The speed of kencc makes this
> much less painful than you might expect. It also happens very rarely
> on plan9 - I cannot remember the last time we had a "big" pull.


> It also neatly sidesteps the issue that different
> applications can need different versions of a single
> shared library. This is a real problem on some OSs
> (see 9fans passim).
>
>
The solution to needing different shared libraries for different
applications is of course to distribute them with the binary in a bundle,
which makes a lot less sense than just statically linking it to begin with.


> -Steve
>
>

[-- Attachment #2: Type: text/html, Size: 1526 bytes --]

Re: [9fans] Binary format

[Stuart Morrow]

On 2/17/10, Steve Simon <steve@quintile.net> wrote:
>> And another important feature of shared libraries is, that when
>> some lib is updated, importing programs dont have to be recompiled.
>> What's the Plan9 solution here ?
>
> We recompile the relevant executables.

also, plan 9 uses filesystems for many things that other systems use dlls for.
if there's a bug fixed in a 9p server, existing programs can still
talk to it without a recompile.

stu

Re: [9fans] Binary format

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 1613 bytes --]

On Wed, Feb 17, 2010 at 8:14 AM, Stuart Morrow <morrow.stuart@googlemail.com
> wrote:

> On 2/17/10, Steve Simon <steve@quintile.net> wrote:
> >> And another important feature of shared libraries is, that when
> >> some lib is updated, importing programs dont have to be recompiled.
> >> What's the Plan9 solution here ?
> >
> > We recompile the relevant executables.
>
> also, plan 9 uses filesystems for many things that other systems use dlls
> for.
> if there's a bug fixed in a 9p server, existing programs can still
> talk to it without a recompile.
>
> stu
>

A lot of "plug in" functionality you'll find on other platforms that
requires a shared library approach can be implemented via a file system
service technique.  Instead of using the C runtime and calling conventions +
a shared library loader to call on some routines that the program in
question did not originate, it can use 9P via the file system to make what
equates to RPC calls to another process that serves the file system itself.


This actually buys one a lot in terms of the following:

1. Isolation of concerns
2. Isolation of faults
3. Composition/Debugging/Testing of solutions via a ubiquitous API/tools

How much more convenient can it get when almost all the tools you need are a
shell, cat/cp/echo etc, and some namespaces in order to build a complex
system from smaller, simpler parts.

I don't know why everyone doesn't want to build software this way.  This is
the same philosophy Erlang and Haskell (and other functional programming
languages) encourage just to write programs.

Dave

[-- Attachment #2: Type: text/html, Size: 2167 bytes --]

Re: [9fans] Binary format

[lucio]

> And if you do want to
> recompile everything, running mk in /sys/src will recompile
> everything and not take all that long doing it, especially if you
> keep the object files around.

And I can vouch for the fact that this works just fine for the ARM
(impersonated in this case by a SheevaPlug) where binaries are not
part of a regular update; something that I imagine would be
considerably more complex for a Linux distribution.

++L

Re: [9fans] Binary format

[Enrico Weigelt]

* David Leimbach <leimy2k@gmail.com> wrote:

> A lot of "plug in" functionality you'll find on other platforms
> that requires a shared library approach can be implemented via
> a file system service technique.

Of course, and I would really like to see that approach in the GNU
world too (actually, I already did that in some projects). But it's
really not easy to convice collegues or clients to this approach
(often they dont even understand the concept of modularity - sad,
but true).

Even synthetic filesystems are good for moving bigger things to their
own services, there're many cases where that wouldnt make sense, for
example parsers. I doubt you'd really suggest putting an XML parser
to its own filesystem for real productional use ;-p (having such a
thing surely is a good idea for some cases, but for most cases an
library would most likely be much easier and efficient.

> I don't know why everyone doesn't want to build software this way.

Well, that's probably a psychological/social phenomenon. Maybe some
"bigger is better" ideology ? ;-o


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 2689 bytes --]

On Wed, Feb 17, 2010 at 10:21 AM, Enrico Weigelt <weigelt@metux.de> wrote:

> * David Leimbach <leimy2k@gmail.com> wrote:
>
> > A lot of "plug in" functionality you'll find on other platforms
> > that requires a shared library approach can be implemented via
> > a file system service technique.
>
> Of course, and I would really like to see that approach in the GNU
> world too (actually, I already did that in some projects). But it's
> really not easy to convice collegues or clients to this approach
> (often they dont even understand the concept of modularity - sad,
> but true).
>
> Even synthetic filesystems are good for moving bigger things to their
> own services, there're many cases where that wouldnt make sense, for
> example parsers. I doubt you'd really suggest putting an XML parser
> to its own filesystem for real productional use ;-p (having such a
> thing surely is a good idea for some cases, but for most cases an
> library would most likely be much easier and efficient.
>
>
No I'm not saying replace all library code with filesystems.  I don't know
why you'd want an RPC interface to an XML parser :-).

Libraries may not always be more efficient than services.  Imagine if Intel
released their 48 core machine today, how would your system utilize those
cores?  Sequential programming optimizations don't (always) work in the
parallel world, and in a lot of cases you actually want to work redundantly
across multiple processes in order to avoid communication overheads in a
well optimized parallel algorithm.

With all of that in mind, it can be better to distribute the services that
make up an application, or a work flow, as decomposed pieces running
together in concert via a sane RPC mechanism.  So, in the future, the Plan 9
way may be, in fact, more efficient than shared libs all piled up together.

Time will tell, but all indications from those who've been studying
computation for decades seems to point this way.

Dave


> > I don't know why everyone doesn't want to build software this way.
>
> Well, that's probably a psychological/social phenomenon. Maybe some
> "bigger is better" ideology ? ;-o
>
>
> cu
> --
> ----------------------------------------------------------------------
>  Enrico Weigelt, metux IT service -- http://www.metux.de/
>
>  phone:  +49 36207 519931  email: weigelt@metux.de
>  mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
> ----------------------------------------------------------------------
>  Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
> ----------------------------------------------------------------------
>
>

[-- Attachment #2: Type: text/html, Size: 3444 bytes --]

Re: [9fans] Binary format

[Lyndon Nerenberg (VE6BBM/VE7TFX)]

> Okay, but then (as an admin) you have to know which apps have
> to be recompiled. For a small system this might be okay, but
> that doesnt scale well ;-o

Plan 9 _is_ a small system.

Re: [9fans] Binary format

[Corey Thomasson]

>
> Even synthetic filesystems are good for moving bigger things to their
> own services, there're many cases where that wouldnt make sense, for
> example parsers. I doubt you'd really suggest putting an XML parser
> to its own filesystem for real productional use ;-p (having such a
> thing surely is a good idea for some cases, but for most cases an
> library would most likely be much easier and efficient.
>

Just because a library is the better idea doesn't mean it has to be shared.

Re: [9fans] Binary format

[Nick LaForge]

Enrico Weigelt <weigelt@metux.de> wrote:
>> And another important feature of shared libraries is, that when
>> some lib is updated, importing programs dont have to be recompiled.

Enrico Weigelt <weigelt@metux.de> wrote:
> Actually, that's just a matter of clean dependency handling.
> Include an API/ABI version in the filename, etc.

Why create single points of failure?  The complexity to justify
creating a bureaucracy delegating responsibility of libraries to
different people is not there in Plan9.  The libraries are for
convenience and orthogonality; they are not the definitive API.  That
is defined by the kernel and by 9P.  They really are the black boxes.
And shouldn't you be inspecting any new code paths when using new
code?  Why assume the library writer has tested against YOUR code in
all paths, and bump to the new version, risking regression?

This to me seems insane for production environments, and is all too
common in the open source world where people seem to be writing new
code for it's own sake and to everyone's detriment.

A nice benefit of taking the simple route early on is that it becomes
easier down the road to implement the functionality in one fell swoop,
top-down.  Maybe in the future the kernel could remove redundant code.
 9vx and it's ilk already do something somewhat orthogonal to this by
executing code that is shorter after it's been dynamically re-written.

Re: [9fans] Binary format

[Enrico Weigelt]

* David Leimbach <leimy2k@gmail.com> wrote:

> No I'm not saying replace all library code with filesystems.  I don't know
> why you'd want an RPC interface to an XML parser :-).

You need to implement an RPC protocol for that. And most likely you
want to get the parsed content into some convenient in-memory structure.
This probably ends up at an similar code size than the parser itself
would be, plus you have the communication overhead.

> Libraries may not always be more efficient than services.  Imagine if Intel
> released their 48 core machine today, how would your system utilize those
> cores?  Sequential programming optimizations don't (always) work in the
> parallel world, and in a lot of cases you actually want to work redundantly
> across multiple processes in order to avoid communication overheads in a
> well optimized parallel algorithm.

Wait, you cheated! You simply assumed that the application is question
can be parallized. In our XML example, there's great chance that the
application has to wait for the parser anyways, so the extra cpus/cores
would help much here. This perhaps would make sense if the file is
quite huge, the application is accessing the xml nodes quite sequentally
and does a lot of things in between. (coroutines would be fine here ;))


Unfortunately, this all is quite academical - I cant just take Plan9
for daily business uses, I just dont have the luxury of resources to
implement all the missing stuff on green land. :(
So I just can try to learn from it and take the concepts into "realworld"
step by step.


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[Enrico Weigelt]

* Enrico Weigelt <weigelt@metux.de> wrote:
> * David Leimbach <leimy2k@gmail.com> wrote:
>
> > No I'm not saying replace all library code with filesystems.  I don't know
> > why you'd want an RPC interface to an XML parser :-).
>
> You need to implement an RPC protocol for that. And most likely you
> want to get the parsed content into some convenient in-memory structure.
> This probably ends up at an similar code size than the parser itself
> would be, plus you have the communication overhead.

forget that part ... i hat a read error in your last statement ;-o


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Binary format

[Nathaniel W Filardo]

[-- Attachment #1: Type: text/plain, Size: 725 bytes --]

On Wed, Feb 17, 2010 at 03:06:57PM +0100, Gorka Guardiola wrote:
> > * each module may have an entry point (main module w/o is allowed,
> >  even if it wouldn't make much sense ;-o), these are called after
> >  relocation, along the dependency tree, from leaf to root.
> 
> no modules.

That's not entirely true; there's (experimental ?) work for dynld(2), and
the shipping compilers can already produce DLMs.  (That said, libdynld is
not yet part of the base system.)  dynld(2) provides a system reminiscent of
dlopen() and dlsym(), but no dynamic linkage is supported (only dynamic
loading).  It is quite tastefully done and is useful to have for some
applications (the python port springs to mind).

--nwf;

[-- Attachment #2: Type: application/pgp-signature, Size: 204 bytes --]

Re: [9fans] Binary format

[Jacob Todd]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

On Wed, Feb 17, 2010 at 03:33:03PM +0100, Enrico Weigelt wrote:
> 
> ah, already suspected that ;-o
> 
> I know that plan9's standard libraries are so small, that they 
> dont really need to be shared. But when more and more applications
> von *nix world get ported, the problem might arise again.
> 
> And another important feature of shared libraries is, that when
> some lib is updated, importing programs dont have to be recompiled.
> What's the Plan9 solution here ?
> 
> 
> cu
See [1] and [2].

[1] http://sta.li/faq
[2] http://harmful.cat-v.org/software/dynamic-linking/
-- 
I am a man who does not exist for others.

[-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --]

Re: [9fans] Binary format

[EBo]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 1698 bytes --]

Nathaniel W Filardo <nwf@cs.jhu.edu> said:

> On Wed, Feb 17, 2010 at 03:06:57PM +0100, Gorka Guardiola wrote:
> > > * each module may have an entry point (main module w/o is allowed,
> > > �even if it wouldn't make much sense ;-o), these are called after
> > > �relocation, along the dependency tree, from leaf to root.
> >
> > no modules.
>
> That's not entirely true; there's (experimental ?) work for dynld(2), and
> the shipping compilers can already produce DLMs.  (That said, libdynld is
> not yet part of the base system.)  dynld(2) provides a system reminiscent of
> dlopen() and dlsym(), but no dynamic linkage is supported (only dynamic
> loading).  It is quite tastefully done and is useful to have for some
> applications (the python port springs to mind).

Is dynamic loading and linkage something that people want?  I have some old
experimental code written in Spirit++ (a C++ template library which
functionally replaces lex and yacc, and reads like EBNF).  The old code is for
a runtime polymorphic parser for RS274 (CNC g-code).  I was experimenting with
parsing historic CNC code, and play with different geometry engines.  This old
project is WAY down on my to-get-around-to list, but it might be an
interesting testbed for something later on.  This would be useful for stuff
like a motion controller for the RepRap and DIY 3D Fab machines.

For that matter, is there anything like Spirit++ <http://boost-spirit.com> in
Plan9/Limbo?  When I developed the code for my fist thesis I developed a
parser for some oddball finite difference equations produced by some modeling
software.  It was nice to have the code read like EBNF.

  EBo --

Re: [9fans] Binary format

[David Leimbach]

[-- Attachment #1: Type: text/plain, Size: 2842 bytes --]

On Wed, Feb 17, 2010 at 3:16 PM, EBo <ebo@sandien.com> wrote:

> Nathaniel W Filardo <nwf@cs.jhu.edu> said:
>
> > On Wed, Feb 17, 2010 at 03:06:57PM +0100, Gorka Guardiola wrote:
> > > > * each module may have an entry point (main module w/o is allowed,
> > > >  even if it wouldn't make much sense ;-o), these are called after
> > > >  relocation, along the dependency tree, from leaf to root.
> > >
> > > no modules.
> >
> > That's not entirely true; there's (experimental ?) work for dynld(2), and
> > the shipping compilers can already produce DLMs.  (That said, libdynld is
> > not yet part of the base system.)  dynld(2) provides a system reminiscent
> of
> > dlopen() and dlsym(), but no dynamic linkage is supported (only dynamic
> > loading).  It is quite tastefully done and is useful to have for some
> > applications (the python port springs to mind).
>
> Is dynamic loading and linkage something that people want?  I have some old
> experimental code written in Spirit++ (a C++ template library which
> functionally replaces lex and yacc, and reads like EBNF).  The old code is
> for
> a runtime polymorphic parser for RS274 (CNC g-code).  I was experimenting
> with
> parsing historic CNC code, and play with different geometry engines.  This
> old
> project is WAY down on my to-get-around-to list, but it might be an
> interesting testbed for something later on.  This would be useful for stuff
> like a motion controller for the RepRap and DIY 3D Fab machines.
>
> For that matter, is there anything like Spirit++ <http://boost-spirit.com>
> in
> Plan9/Limbo?  When I developed the code for my fist thesis I developed a
> parser for some oddball finite difference equations produced by some
> modeling
> software.  It was nice to have the code read like EBNF.
>
>
Spirit is pretty amazing in that you can write very succinct and powerful
code with it, and yet it is terrible in that when you get an error message
you can end up with quite a disaster (due to the templates).  I particularly
like the Phoenix submodule in the new boost libraries for writing little
functional tidbits with STL looping constructs and iterators.

Speaking of which, Spirit appears to borrow pretty heavily from Parsec which
is a monadic parser combinator framework for Haskell that I believe will
work with Hugs, that also works on Plan 9.  (I think Andrey ported Hugs a
long time ago).

I'd LOVE to have GHC on Plan 9, but my first attempts to get anything
building with it were pretty frustrating.

There's scheme language implementations available for plan 9, and I think
one could either find or come up with a parsing framework like spirit for
scheme that relied on continuations (more "native" to scheme than monads,
though you could use monads too).

Dave


>  EBo --
>
>
>

[-- Attachment #2: Type: text/html, Size: 3606 bytes --]

Re: [9fans] Binary format

[Patrick Kelly]

On Feb 17, 2010, at 10:04 AM, Enrico Weigelt <weigelt@metux.de> wrote:

> * Steve Simon <steve@quintile.net> wrote:
>
>> We recompile the relevant executables. The speed of kencc makes this
>> much less painful than you might expect. It also happens very rarely
>> on plan9 - I cannot remember the last time we had a "big" pull.
>
> Okay, but then (as an admin) you have to know which apps have
> to be recompiled. For a small system this might be okay, but
> that doesnt scale well ;-o
>
> A possible solution could be an package management system which
> knows dependencies and so can tell what to rebuild.

Entia non sunt multiplicanda praeter necessitatem
>
>> It also neatly sidesteps the issue that different applications
>> can need different versions of a single shared library. This
>> is a real problem on some OSs (see 9fans passim).
>
> Actually, that's just a matter of clean dependency handling.
> Include an API/ABI version in the filename, etc.
>
>
> cu
> --
> ----------------------------------------------------------------------
> Enrico Weigelt, metux IT service -- http://www.metux.de/
>
> phone:  +49 36207 519931  email: weigelt@metux.de
> mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
> ----------------------------------------------------------------------
> Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
> ----------------------------------------------------------------------
>

Re: [9fans] Binary format

[Patrick Kelly]

On Feb 17, 2010, at 1:21 PM, Enrico Weigelt <weigelt@metux.de> wrote:

> * David Leimbach <leimy2k@gmail.com> wrote:
>
>> A lot of "plug in" functionality you'll find on other platforms
>> that requires a shared library approach can be implemented via
>> a file system service technique.
>
> Of course, and I would really like to see that approach in the GNU
> world too (actually, I already did that in some projects). But it's
> really not easy to convice collegues or clients to this approach
> (often they dont even understand the concept of modularity - sad,
> but true).
>
> Even synthetic filesystems are good for moving bigger things to their
> own services, there're many cases where that wouldnt make sense, for
> example parsers. I doubt you'd really suggest putting an XML parser
> to its own filesystem for real productional use ;-p (having such a
> thing surely is a good idea for some cases, but for most cases an
> library would most likely be much easier and efficient.
I just wouldnt suggest XML
>
>> I don't know why everyone doesn't want to build software this way.
>
> Well, that's probably a psychological/social phenomenon. Maybe some
> "bigger is better" ideology ? ;-o
More features is better, with no concern about stability or
useability...
>
>
> cu
> --
> ----------------------------------------------------------------------
> Enrico Weigelt, metux IT service -- http://www.metux.de/
>
> phone:  +49 36207 519931  email: weigelt@metux.de
> mobile: +49 174 7066481   icq:   210169427         skype: nekrad666
> ----------------------------------------------------------------------
> Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
> ----------------------------------------------------------------------
>