* [9fans] Binary format @ 2010-02-17 13:31 Enrico Weigelt 2010-02-17 8:50 ` Jacob Todd ` (2 more replies) 0 siblings, 3 replies; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 13:31 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs Hi folks, just curious: which binfmt does Plan9 use ? How are share libraries handled (if they exist at all) ? Inspired by recent discussions @ gentoo-user, I'm thinking a bit how an simple and efficient binfmt could look like. Some key ideas are: * purely runtime information (no debug stuff, etc) * everthing's sharable/relocatable module, with strict dependencies * on exec() the process image will be constructed the modules along the dependency tree (the main program as root) * each module may have an entry point (main module w/o is allowed, even if it wouldn't make much sense ;-o), these are called after relocation, along the dependency tree, from leaf to root. * the whole loader sits in the kernel (maybe w/ some additional helper deamon in userland), but userland can pass parameters like search pathes, etc via env. IMHO having the dynamic loader in kernel-land (in contrary to ELF on GNU) not just removes the need for lots of syscalls, but allows a lot of optimizations, eg. code sharing w/ mmap() or caching relocation data. cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 13:31 [9fans] Binary format Enrico Weigelt @ 2010-02-17 8:50 ` Jacob Todd 2010-02-17 13:58 ` Enrico Weigelt 2010-02-17 14:06 ` Gorka Guardiola 2010-02-17 14:38 ` blstuart 2 siblings, 1 reply; 32+ messages in thread From: Jacob Todd @ 2010-02-17 8:50 UTC (permalink / raw) To: weigelt, Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1446 bytes --] On Wed, Feb 17, 2010 at 02:31:09PM +0100, Enrico Weigelt wrote: > > Hi folks, > > just curious: which binfmt does Plan9 use ? > How are share libraries handled (if they exist at all) ? > > Inspired by recent discussions @ gentoo-user, I'm thinking a bit > how an simple and efficient binfmt could look like. Some key > ideas are: > > * purely runtime information (no debug stuff, etc) > * everthing's sharable/relocatable module, with strict dependencies > * on exec() the process image will be constructed the modules along > the dependency tree (the main program as root) > * each module may have an entry point (main module w/o is allowed, > even if it wouldn't make much sense ;-o), these are called after > relocation, along the dependency tree, from leaf to root. > * the whole loader sits in the kernel (maybe w/ some additional > helper deamon in userland), but userland can pass parameters > like search pathes, etc via env. > > IMHO having the dynamic loader in kernel-land (in contrary to ELF > on GNU) not just removes the need for lots of syscalls, but allows > a lot of optimizations, eg. code sharing w/ mmap() or caching > relocation data. > > Where was this thread at? I'm subscribed to gentoo-user, but I don't have any threads about binary formats in my inbox. Was the discussion in another thread (maybe the giant HAL thread)? -- I am a man who does not exist for others. [-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 8:50 ` Jacob Todd @ 2010-02-17 13:58 ` Enrico Weigelt 2010-02-17 14:01 ` erik quanstrom 0 siblings, 1 reply; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 13:58 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs * Jacob Todd <jaketodd422@gmail.com> wrote: > Where was this thread at? I'm subscribed to gentoo-user, but I don't have > any threads about binary formats in my inbox. Was the discussion in > another thread (maybe the giant HAL thread)? Exactly this one ;-p Oh, and we also talk about fat vs. tiny libraries, etc. Maybe you'd like to join in ;-) cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 13:58 ` Enrico Weigelt @ 2010-02-17 14:01 ` erik quanstrom 0 siblings, 0 replies; 32+ messages in thread From: erik quanstrom @ 2010-02-17 14:01 UTC (permalink / raw) To: weigelt, 9fans > Oh, and we also talk about fat vs. tiny libraries, etc. > Maybe you'd like to join in ;-) please remember that this isn't a linux list. - erik ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 13:31 [9fans] Binary format Enrico Weigelt 2010-02-17 8:50 ` Jacob Todd @ 2010-02-17 14:06 ` Gorka Guardiola 2010-02-17 14:13 ` David Leimbach ` (2 more replies) 2010-02-17 14:38 ` blstuart 2 siblings, 3 replies; 32+ messages in thread From: Gorka Guardiola @ 2010-02-17 14:06 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> wrote: > > Hi folks, > > just curious: which binfmt does Plan9 use ? > How are share libraries handled (if they exist at all) ? a.out(6) no shared libraries. > > Inspired by recent discussions @ gentoo-user, I'm thinking a bit > how an simple and efficient binfmt could look like. Some key > ideas are: > > * purely runtime information (no debug stuff, etc) just strip it. > * everthing's sharable/relocatable module, with strict dependencies > * on exec() the process image will be constructed the modules along > the dependency tree (the main program as root) no shared libraries. > * each module may have an entry point (main module w/o is allowed, > even if it wouldn't make much sense ;-o), these are called after > relocation, along the dependency tree, from leaf to root. no modules. -- - curiosity sKilled the cat ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:06 ` Gorka Guardiola @ 2010-02-17 14:13 ` David Leimbach 2010-02-17 14:33 ` Enrico Weigelt 2010-02-17 21:26 ` Nathaniel W Filardo 2 siblings, 0 replies; 32+ messages in thread From: David Leimbach @ 2010-02-17 14:13 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1236 bytes --] And all of the below are some of the reasons I still love plan 9. Simplicity first is a really great way to work (I've learned over the years). Dave On Wed, Feb 17, 2010 at 6:06 AM, Gorka Guardiola <paurea@gmail.com> wrote: > On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> wrote: > > > > Hi folks, > > > > just curious: which binfmt does Plan9 use ? > > How are share libraries handled (if they exist at all) ? > > a.out(6) > no shared libraries. > > > > > Inspired by recent discussions @ gentoo-user, I'm thinking a bit > > how an simple and efficient binfmt could look like. Some key > > ideas are: > > > > * purely runtime information (no debug stuff, etc) > > just strip it. > > > * everthing's sharable/relocatable module, with strict dependencies > > * on exec() the process image will be constructed the modules along > > the dependency tree (the main program as root) > > no shared libraries. > > > * each module may have an entry point (main module w/o is allowed, > > even if it wouldn't make much sense ;-o), these are called after > > relocation, along the dependency tree, from leaf to root. > > no modules. > > > -- > - curiosity sKilled the cat > > [-- Attachment #2: Type: text/html, Size: 1789 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:06 ` Gorka Guardiola 2010-02-17 14:13 ` David Leimbach @ 2010-02-17 14:33 ` Enrico Weigelt 2010-02-17 14:55 ` Steve Simon ` (2 more replies) 2010-02-17 21:26 ` Nathaniel W Filardo 2 siblings, 3 replies; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 14:33 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs * Gorka Guardiola <paurea@gmail.com> wrote: > On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> wrote: > > > > Hi folks, > > > > just curious: which binfmt does Plan9 use ? > > How are share libraries handled (if they exist at all) ? > > a.out(6) > no shared libraries. ah, already suspected that ;-o I know that plan9's standard libraries are so small, that they dont really need to be shared. But when more and more applications von *nix world get ported, the problem might arise again. And another important feature of shared libraries is, that when some lib is updated, importing programs dont have to be recompiled. What's the Plan9 solution here ? cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:33 ` Enrico Weigelt @ 2010-02-17 14:55 ` Steve Simon 2010-02-17 15:04 ` Enrico Weigelt ` (2 more replies) 2010-02-17 15:13 ` Robert Raschke 2010-02-17 21:32 ` Jacob Todd 2 siblings, 3 replies; 32+ messages in thread From: Steve Simon @ 2010-02-17 14:55 UTC (permalink / raw) To: weigelt, 9fans > And another important feature of shared libraries is, that when > some lib is updated, importing programs dont have to be recompiled. > What's the Plan9 solution here ? We recompile the relevant executables. The speed of kencc makes this much less painful than you might expect. It also happens very rarely on plan9 - I cannot remember the last time we had a "big" pull. It also neatly sidesteps the issue that different applications can need different versions of a single shared library. This is a real problem on some OSs (see 9fans passim). -Steve ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:55 ` Steve Simon @ 2010-02-17 15:04 ` Enrico Weigelt 2010-02-17 15:29 ` erik quanstrom ` (4 more replies) 2010-02-17 15:58 ` David Leimbach 2010-02-17 16:14 ` Stuart Morrow 2 siblings, 5 replies; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 15:04 UTC (permalink / raw) To: 9fans * Steve Simon <steve@quintile.net> wrote: > We recompile the relevant executables. The speed of kencc makes this > much less painful than you might expect. It also happens very rarely > on plan9 - I cannot remember the last time we had a "big" pull. Okay, but then (as an admin) you have to know which apps have to be recompiled. For a small system this might be okay, but that doesnt scale well ;-o A possible solution could be an package management system which knows dependencies and so can tell what to rebuild. > It also neatly sidesteps the issue that different applications > can need different versions of a single shared library. This > is a real problem on some OSs (see 9fans passim). Actually, that's just a matter of clean dependency handling. Include an API/ABI version in the filename, etc. cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 15:04 ` Enrico Weigelt @ 2010-02-17 15:29 ` erik quanstrom 2010-02-17 15:48 ` blstuart ` (3 subsequent siblings) 4 siblings, 0 replies; 32+ messages in thread From: erik quanstrom @ 2010-02-17 15:29 UTC (permalink / raw) To: weigelt, 9fans > > We recompile the relevant executables. The speed of kencc makes this > > much less painful than you might expect. It also happens very rarely > > on plan9 - I cannot remember the last time we had a "big" pull. > > Okay, but then (as an admin) you have to know which apps have > to be recompiled. For a small system this might be okay, but > that doesnt scale well ;-o have you installed plan 9? - erik ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 15:04 ` Enrico Weigelt 2010-02-17 15:29 ` erik quanstrom @ 2010-02-17 15:48 ` blstuart 2010-02-17 17:54 ` lucio 2010-02-17 18:37 ` Lyndon Nerenberg (VE6BBM/VE7TFX) ` (2 subsequent siblings) 4 siblings, 1 reply; 32+ messages in thread From: blstuart @ 2010-02-17 15:48 UTC (permalink / raw) To: weigelt, 9fans >> We recompile the relevant executables. The speed of kencc makes this >> much less painful than you might expect. It also happens very rarely >> on plan9 - I cannot remember the last time we had a "big" pull. > > Okay, but then (as an admin) you have to know which apps have > to be recompiled. For a small system this might be okay, but > that doesnt scale well ;-o Keep in mind that Plan 9 isn't a new OS just waiting for thousands of GNU apps to be ported. Plan 9 is over 20 years old and has a community that's developed native apps and a few selected ports for the things the community finds useful. So the kind of apps explosion that would require it to scale well is anathama to the culture. Certainly there are things that would be welcomed to be be written, and a few even ported. But having 72 different X11 widget "toolboxes" and 42 different GUI versions of diff will meet with a resounding thud. Eye candy, complexity, and mainstream popularity don't impress the people in the Plan 9 community; elegance and originality do. But beyond that, the main distribution includes both binaries and source for updates. So a pull will generally get everything you need without needing to recompile. And if you do want to recompile everything, running mk in /sys/src will recompile everything and not take all that long doing it, especially if you keep the object files around. BLS ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 15:48 ` blstuart @ 2010-02-17 17:54 ` lucio 0 siblings, 0 replies; 32+ messages in thread From: lucio @ 2010-02-17 17:54 UTC (permalink / raw) To: 9fans > And if you do want to > recompile everything, running mk in /sys/src will recompile > everything and not take all that long doing it, especially if you > keep the object files around. And I can vouch for the fact that this works just fine for the ARM (impersonated in this case by a SheevaPlug) where binaries are not part of a regular update; something that I imagine would be considerably more complex for a Linux distribution. ++L ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 15:04 ` Enrico Weigelt 2010-02-17 15:29 ` erik quanstrom 2010-02-17 15:48 ` blstuart @ 2010-02-17 18:37 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2010-02-17 19:00 ` Nick LaForge 2010-02-18 15:08 ` Patrick Kelly 4 siblings, 0 replies; 32+ messages in thread From: Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2010-02-17 18:37 UTC (permalink / raw) To: weigelt, 9fans > Okay, but then (as an admin) you have to know which apps have > to be recompiled. For a small system this might be okay, but > that doesnt scale well ;-o Plan 9 _is_ a small system. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 15:04 ` Enrico Weigelt ` (2 preceding siblings ...) 2010-02-17 18:37 ` Lyndon Nerenberg (VE6BBM/VE7TFX) @ 2010-02-17 19:00 ` Nick LaForge 2010-02-18 15:08 ` Patrick Kelly 4 siblings, 0 replies; 32+ messages in thread From: Nick LaForge @ 2010-02-17 19:00 UTC (permalink / raw) To: weigelt, Fans of the OS Plan 9 from Bell Labs Enrico Weigelt <weigelt@metux.de> wrote: >> And another important feature of shared libraries is, that when >> some lib is updated, importing programs dont have to be recompiled. Enrico Weigelt <weigelt@metux.de> wrote: > Actually, that's just a matter of clean dependency handling. > Include an API/ABI version in the filename, etc. Why create single points of failure? The complexity to justify creating a bureaucracy delegating responsibility of libraries to different people is not there in Plan9. The libraries are for convenience and orthogonality; they are not the definitive API. That is defined by the kernel and by 9P. They really are the black boxes. And shouldn't you be inspecting any new code paths when using new code? Why assume the library writer has tested against YOUR code in all paths, and bump to the new version, risking regression? This to me seems insane for production environments, and is all too common in the open source world where people seem to be writing new code for it's own sake and to everyone's detriment. A nice benefit of taking the simple route early on is that it becomes easier down the road to implement the functionality in one fell swoop, top-down. Maybe in the future the kernel could remove redundant code. 9vx and it's ilk already do something somewhat orthogonal to this by executing code that is shorter after it's been dynamically re-written. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 15:04 ` Enrico Weigelt ` (3 preceding siblings ...) 2010-02-17 19:00 ` Nick LaForge @ 2010-02-18 15:08 ` Patrick Kelly 4 siblings, 0 replies; 32+ messages in thread From: Patrick Kelly @ 2010-02-18 15:08 UTC (permalink / raw) To: weigelt, Fans of the OS Plan 9 from Bell Labs On Feb 17, 2010, at 10:04 AM, Enrico Weigelt <weigelt@metux.de> wrote: > * Steve Simon <steve@quintile.net> wrote: > >> We recompile the relevant executables. The speed of kencc makes this >> much less painful than you might expect. It also happens very rarely >> on plan9 - I cannot remember the last time we had a "big" pull. > > Okay, but then (as an admin) you have to know which apps have > to be recompiled. For a small system this might be okay, but > that doesnt scale well ;-o > > A possible solution could be an package management system which > knows dependencies and so can tell what to rebuild. Entia non sunt multiplicanda praeter necessitatem > >> It also neatly sidesteps the issue that different applications >> can need different versions of a single shared library. This >> is a real problem on some OSs (see 9fans passim). > > Actually, that's just a matter of clean dependency handling. > Include an API/ABI version in the filename, etc. > > > cu > -- > ---------------------------------------------------------------------- > Enrico Weigelt, metux IT service -- http://www.metux.de/ > > phone: +49 36207 519931 email: weigelt@metux.de > mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 > ---------------------------------------------------------------------- > Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme > ---------------------------------------------------------------------- > ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:55 ` Steve Simon 2010-02-17 15:04 ` Enrico Weigelt @ 2010-02-17 15:58 ` David Leimbach 2010-02-17 16:14 ` Stuart Morrow 2 siblings, 0 replies; 32+ messages in thread From: David Leimbach @ 2010-02-17 15:58 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 904 bytes --] On Wed, Feb 17, 2010 at 6:55 AM, Steve Simon <steve@quintile.net> wrote: > > And another important feature of shared libraries is, that when > > some lib is updated, importing programs dont have to be recompiled. > > What's the Plan9 solution here ? > > We recompile the relevant executables. The speed of kencc makes this > much less painful than you might expect. It also happens very rarely > on plan9 - I cannot remember the last time we had a "big" pull. > It also neatly sidesteps the issue that different > applications can need different versions of a single > shared library. This is a real problem on some OSs > (see 9fans passim). > > The solution to needing different shared libraries for different applications is of course to distribute them with the binary in a bundle, which makes a lot less sense than just statically linking it to begin with. > -Steve > > [-- Attachment #2: Type: text/html, Size: 1526 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:55 ` Steve Simon 2010-02-17 15:04 ` Enrico Weigelt 2010-02-17 15:58 ` David Leimbach @ 2010-02-17 16:14 ` Stuart Morrow 2010-02-17 16:28 ` David Leimbach 2 siblings, 1 reply; 32+ messages in thread From: Stuart Morrow @ 2010-02-17 16:14 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs On 2/17/10, Steve Simon <steve@quintile.net> wrote: >> And another important feature of shared libraries is, that when >> some lib is updated, importing programs dont have to be recompiled. >> What's the Plan9 solution here ? > > We recompile the relevant executables. also, plan 9 uses filesystems for many things that other systems use dlls for. if there's a bug fixed in a 9p server, existing programs can still talk to it without a recompile. stu ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 16:14 ` Stuart Morrow @ 2010-02-17 16:28 ` David Leimbach 2010-02-17 18:21 ` Enrico Weigelt 0 siblings, 1 reply; 32+ messages in thread From: David Leimbach @ 2010-02-17 16:28 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1613 bytes --] On Wed, Feb 17, 2010 at 8:14 AM, Stuart Morrow <morrow.stuart@googlemail.com > wrote: > On 2/17/10, Steve Simon <steve@quintile.net> wrote: > >> And another important feature of shared libraries is, that when > >> some lib is updated, importing programs dont have to be recompiled. > >> What's the Plan9 solution here ? > > > > We recompile the relevant executables. > > also, plan 9 uses filesystems for many things that other systems use dlls > for. > if there's a bug fixed in a 9p server, existing programs can still > talk to it without a recompile. > > stu > A lot of "plug in" functionality you'll find on other platforms that requires a shared library approach can be implemented via a file system service technique. Instead of using the C runtime and calling conventions + a shared library loader to call on some routines that the program in question did not originate, it can use 9P via the file system to make what equates to RPC calls to another process that serves the file system itself. This actually buys one a lot in terms of the following: 1. Isolation of concerns 2. Isolation of faults 3. Composition/Debugging/Testing of solutions via a ubiquitous API/tools How much more convenient can it get when almost all the tools you need are a shell, cat/cp/echo etc, and some namespaces in order to build a complex system from smaller, simpler parts. I don't know why everyone doesn't want to build software this way. This is the same philosophy Erlang and Haskell (and other functional programming languages) encourage just to write programs. Dave [-- Attachment #2: Type: text/html, Size: 2167 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 16:28 ` David Leimbach @ 2010-02-17 18:21 ` Enrico Weigelt 2010-02-17 18:34 ` David Leimbach ` (2 more replies) 0 siblings, 3 replies; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 18:21 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs * David Leimbach <leimy2k@gmail.com> wrote: > A lot of "plug in" functionality you'll find on other platforms > that requires a shared library approach can be implemented via > a file system service technique. Of course, and I would really like to see that approach in the GNU world too (actually, I already did that in some projects). But it's really not easy to convice collegues or clients to this approach (often they dont even understand the concept of modularity - sad, but true). Even synthetic filesystems are good for moving bigger things to their own services, there're many cases where that wouldnt make sense, for example parsers. I doubt you'd really suggest putting an XML parser to its own filesystem for real productional use ;-p (having such a thing surely is a good idea for some cases, but for most cases an library would most likely be much easier and efficient. > I don't know why everyone doesn't want to build software this way. Well, that's probably a psychological/social phenomenon. Maybe some "bigger is better" ideology ? ;-o cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 18:21 ` Enrico Weigelt @ 2010-02-17 18:34 ` David Leimbach 2010-02-17 19:02 ` Enrico Weigelt 2010-02-17 18:38 ` Corey Thomasson 2010-02-18 15:16 ` Patrick Kelly 2 siblings, 1 reply; 32+ messages in thread From: David Leimbach @ 2010-02-17 18:34 UTC (permalink / raw) To: weigelt, Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 2689 bytes --] On Wed, Feb 17, 2010 at 10:21 AM, Enrico Weigelt <weigelt@metux.de> wrote: > * David Leimbach <leimy2k@gmail.com> wrote: > > > A lot of "plug in" functionality you'll find on other platforms > > that requires a shared library approach can be implemented via > > a file system service technique. > > Of course, and I would really like to see that approach in the GNU > world too (actually, I already did that in some projects). But it's > really not easy to convice collegues or clients to this approach > (often they dont even understand the concept of modularity - sad, > but true). > > Even synthetic filesystems are good for moving bigger things to their > own services, there're many cases where that wouldnt make sense, for > example parsers. I doubt you'd really suggest putting an XML parser > to its own filesystem for real productional use ;-p (having such a > thing surely is a good idea for some cases, but for most cases an > library would most likely be much easier and efficient. > > No I'm not saying replace all library code with filesystems. I don't know why you'd want an RPC interface to an XML parser :-). Libraries may not always be more efficient than services. Imagine if Intel released their 48 core machine today, how would your system utilize those cores? Sequential programming optimizations don't (always) work in the parallel world, and in a lot of cases you actually want to work redundantly across multiple processes in order to avoid communication overheads in a well optimized parallel algorithm. With all of that in mind, it can be better to distribute the services that make up an application, or a work flow, as decomposed pieces running together in concert via a sane RPC mechanism. So, in the future, the Plan 9 way may be, in fact, more efficient than shared libs all piled up together. Time will tell, but all indications from those who've been studying computation for decades seems to point this way. Dave > > I don't know why everyone doesn't want to build software this way. > > Well, that's probably a psychological/social phenomenon. Maybe some > "bigger is better" ideology ? ;-o > > > cu > -- > ---------------------------------------------------------------------- > Enrico Weigelt, metux IT service -- http://www.metux.de/ > > phone: +49 36207 519931 email: weigelt@metux.de > mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 > ---------------------------------------------------------------------- > Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme > ---------------------------------------------------------------------- > > [-- Attachment #2: Type: text/html, Size: 3444 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 18:34 ` David Leimbach @ 2010-02-17 19:02 ` Enrico Weigelt 2010-02-17 19:35 ` Enrico Weigelt 0 siblings, 1 reply; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 19:02 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs * David Leimbach <leimy2k@gmail.com> wrote: > No I'm not saying replace all library code with filesystems. I don't know > why you'd want an RPC interface to an XML parser :-). You need to implement an RPC protocol for that. And most likely you want to get the parsed content into some convenient in-memory structure. This probably ends up at an similar code size than the parser itself would be, plus you have the communication overhead. > Libraries may not always be more efficient than services. Imagine if Intel > released their 48 core machine today, how would your system utilize those > cores? Sequential programming optimizations don't (always) work in the > parallel world, and in a lot of cases you actually want to work redundantly > across multiple processes in order to avoid communication overheads in a > well optimized parallel algorithm. Wait, you cheated! You simply assumed that the application is question can be parallized. In our XML example, there's great chance that the application has to wait for the parser anyways, so the extra cpus/cores would help much here. This perhaps would make sense if the file is quite huge, the application is accessing the xml nodes quite sequentally and does a lot of things in between. (coroutines would be fine here ;)) Unfortunately, this all is quite academical - I cant just take Plan9 for daily business uses, I just dont have the luxury of resources to implement all the missing stuff on green land. :( So I just can try to learn from it and take the concepts into "realworld" step by step. cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 19:02 ` Enrico Weigelt @ 2010-02-17 19:35 ` Enrico Weigelt 0 siblings, 0 replies; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 19:35 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs * Enrico Weigelt <weigelt@metux.de> wrote: > * David Leimbach <leimy2k@gmail.com> wrote: > > > No I'm not saying replace all library code with filesystems. I don't know > > why you'd want an RPC interface to an XML parser :-). > > You need to implement an RPC protocol for that. And most likely you > want to get the parsed content into some convenient in-memory structure. > This probably ends up at an similar code size than the parser itself > would be, plus you have the communication overhead. forget that part ... i hat a read error in your last statement ;-o cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 18:21 ` Enrico Weigelt 2010-02-17 18:34 ` David Leimbach @ 2010-02-17 18:38 ` Corey Thomasson 2010-02-18 15:16 ` Patrick Kelly 2 siblings, 0 replies; 32+ messages in thread From: Corey Thomasson @ 2010-02-17 18:38 UTC (permalink / raw) To: weigelt, Fans of the OS Plan 9 from Bell Labs > > Even synthetic filesystems are good for moving bigger things to their > own services, there're many cases where that wouldnt make sense, for > example parsers. I doubt you'd really suggest putting an XML parser > to its own filesystem for real productional use ;-p (having such a > thing surely is a good idea for some cases, but for most cases an > library would most likely be much easier and efficient. > Just because a library is the better idea doesn't mean it has to be shared. ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 18:21 ` Enrico Weigelt 2010-02-17 18:34 ` David Leimbach 2010-02-17 18:38 ` Corey Thomasson @ 2010-02-18 15:16 ` Patrick Kelly 2 siblings, 0 replies; 32+ messages in thread From: Patrick Kelly @ 2010-02-18 15:16 UTC (permalink / raw) To: weigelt, Fans of the OS Plan 9 from Bell Labs On Feb 17, 2010, at 1:21 PM, Enrico Weigelt <weigelt@metux.de> wrote: > * David Leimbach <leimy2k@gmail.com> wrote: > >> A lot of "plug in" functionality you'll find on other platforms >> that requires a shared library approach can be implemented via >> a file system service technique. > > Of course, and I would really like to see that approach in the GNU > world too (actually, I already did that in some projects). But it's > really not easy to convice collegues or clients to this approach > (often they dont even understand the concept of modularity - sad, > but true). > > Even synthetic filesystems are good for moving bigger things to their > own services, there're many cases where that wouldnt make sense, for > example parsers. I doubt you'd really suggest putting an XML parser > to its own filesystem for real productional use ;-p (having such a > thing surely is a good idea for some cases, but for most cases an > library would most likely be much easier and efficient. I just wouldnt suggest XML > >> I don't know why everyone doesn't want to build software this way. > > Well, that's probably a psychological/social phenomenon. Maybe some > "bigger is better" ideology ? ;-o More features is better, with no concern about stability or useability... > > > cu > -- > ---------------------------------------------------------------------- > Enrico Weigelt, metux IT service -- http://www.metux.de/ > > phone: +49 36207 519931 email: weigelt@metux.de > mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 > ---------------------------------------------------------------------- > Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme > ---------------------------------------------------------------------- > ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:33 ` Enrico Weigelt 2010-02-17 14:55 ` Steve Simon @ 2010-02-17 15:13 ` Robert Raschke 2010-02-17 21:32 ` Jacob Todd 2 siblings, 0 replies; 32+ messages in thread From: Robert Raschke @ 2010-02-17 15:13 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 1681 bytes --] On Wed, Feb 17, 2010 at 2:33 PM, Enrico Weigelt <weigelt@metux.de> wrote: > * Gorka Guardiola <paurea@gmail.com> wrote: > > On Wed, Feb 17, 2010 at 2:31 PM, Enrico Weigelt <weigelt@metux.de> > wrote: > > > > > > Hi folks, > > > > > > just curious: which binfmt does Plan9 use ? > > > How are share libraries handled (if they exist at all) ? > > > > a.out(6) > > no shared libraries. > > ah, already suspected that ;-o > > I know that plan9's standard libraries are so small, that they > dont really need to be shared. But when more and more applications > von *nix world get ported, the problem might arise again. > Well, I know that it is very tempting to "just" port across lots of programs. But if you are looking to solve an actual problem (in a business or "just do it" sense) and there is nothing on Plan 9 to help you with that, then use Linux or Windows or OSX or ... On the other hand, if you are really interested in providing functionality to be used within Plan 9 that I'd strongly suggest investigating more than a port. Re-implementing using the appropriate Plan 9 approach would be way more beneficial. > And another important feature of shared libraries is, that when > some lib is updated, importing programs dont have to be recompiled. > What's the Plan9 solution here ? > :-) Recompile. (The usual argument to counter this approach is that security updates to underlying libs are easier to get out across to everyone, while at the same time completely ignoring the depths of DLL Hell and the myriads of wonderfull new ways to break perfectly fine software all over the place. Me, cynical, noooo! :-) Robby [-- Attachment #2: Type: text/html, Size: 2372 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:33 ` Enrico Weigelt 2010-02-17 14:55 ` Steve Simon 2010-02-17 15:13 ` Robert Raschke @ 2010-02-17 21:32 ` Jacob Todd 2 siblings, 0 replies; 32+ messages in thread From: Jacob Todd @ 2010-02-17 21:32 UTC (permalink / raw) To: 9fans [-- Attachment #1: Type: text/plain, Size: 644 bytes --] On Wed, Feb 17, 2010 at 03:33:03PM +0100, Enrico Weigelt wrote: > > ah, already suspected that ;-o > > I know that plan9's standard libraries are so small, that they > dont really need to be shared. But when more and more applications > von *nix world get ported, the problem might arise again. > > And another important feature of shared libraries is, that when > some lib is updated, importing programs dont have to be recompiled. > What's the Plan9 solution here ? > > > cu See [1] and [2]. [1] http://sta.li/faq [2] http://harmful.cat-v.org/software/dynamic-linking/ -- I am a man who does not exist for others. [-- Attachment #2: Type: application/pgp-signature, Size: 205 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:06 ` Gorka Guardiola 2010-02-17 14:13 ` David Leimbach 2010-02-17 14:33 ` Enrico Weigelt @ 2010-02-17 21:26 ` Nathaniel W Filardo 2010-02-17 23:16 ` EBo 2 siblings, 1 reply; 32+ messages in thread From: Nathaniel W Filardo @ 2010-02-17 21:26 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 725 bytes --] On Wed, Feb 17, 2010 at 03:06:57PM +0100, Gorka Guardiola wrote: > > * each module may have an entry point (main module w/o is allowed, > > even if it wouldn't make much sense ;-o), these are called after > > relocation, along the dependency tree, from leaf to root. > > no modules. That's not entirely true; there's (experimental ?) work for dynld(2), and the shipping compilers can already produce DLMs. (That said, libdynld is not yet part of the base system.) dynld(2) provides a system reminiscent of dlopen() and dlsym(), but no dynamic linkage is supported (only dynamic loading). It is quite tastefully done and is useful to have for some applications (the python port springs to mind). --nwf; [-- Attachment #2: Type: application/pgp-signature, Size: 204 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 21:26 ` Nathaniel W Filardo @ 2010-02-17 23:16 ` EBo 2010-02-17 23:22 ` David Leimbach 0 siblings, 1 reply; 32+ messages in thread From: EBo @ 2010-02-17 23:16 UTC (permalink / raw) To: Fans of the OS Plan 9 from Bell Labs, Nathaniel W Filardo [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1: Type: text/plain, Size: 1698 bytes --] Nathaniel W Filardo <nwf@cs.jhu.edu> said: > On Wed, Feb 17, 2010 at 03:06:57PM +0100, Gorka Guardiola wrote: > > > * each module may have an entry point (main module w/o is allowed, > > > �even if it wouldn't make much sense ;-o), these are called after > > > �relocation, along the dependency tree, from leaf to root. > > > > no modules. > > That's not entirely true; there's (experimental ?) work for dynld(2), and > the shipping compilers can already produce DLMs. (That said, libdynld is > not yet part of the base system.) dynld(2) provides a system reminiscent of > dlopen() and dlsym(), but no dynamic linkage is supported (only dynamic > loading). It is quite tastefully done and is useful to have for some > applications (the python port springs to mind). Is dynamic loading and linkage something that people want? I have some old experimental code written in Spirit++ (a C++ template library which functionally replaces lex and yacc, and reads like EBNF). The old code is for a runtime polymorphic parser for RS274 (CNC g-code). I was experimenting with parsing historic CNC code, and play with different geometry engines. This old project is WAY down on my to-get-around-to list, but it might be an interesting testbed for something later on. This would be useful for stuff like a motion controller for the RepRap and DIY 3D Fab machines. For that matter, is there anything like Spirit++ <http://boost-spirit.com> in Plan9/Limbo? When I developed the code for my fist thesis I developed a parser for some oddball finite difference equations produced by some modeling software. It was nice to have the code read like EBNF. EBo -- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 23:16 ` EBo @ 2010-02-17 23:22 ` David Leimbach 0 siblings, 0 replies; 32+ messages in thread From: David Leimbach @ 2010-02-17 23:22 UTC (permalink / raw) To: ebo, Fans of the OS Plan 9 from Bell Labs [-- Attachment #1: Type: text/plain, Size: 2842 bytes --] On Wed, Feb 17, 2010 at 3:16 PM, EBo <ebo@sandien.com> wrote: > Nathaniel W Filardo <nwf@cs.jhu.edu> said: > > > On Wed, Feb 17, 2010 at 03:06:57PM +0100, Gorka Guardiola wrote: > > > > * each module may have an entry point (main module w/o is allowed, > > > > even if it wouldn't make much sense ;-o), these are called after > > > > relocation, along the dependency tree, from leaf to root. > > > > > > no modules. > > > > That's not entirely true; there's (experimental ?) work for dynld(2), and > > the shipping compilers can already produce DLMs. (That said, libdynld is > > not yet part of the base system.) dynld(2) provides a system reminiscent > of > > dlopen() and dlsym(), but no dynamic linkage is supported (only dynamic > > loading). It is quite tastefully done and is useful to have for some > > applications (the python port springs to mind). > > Is dynamic loading and linkage something that people want? I have some old > experimental code written in Spirit++ (a C++ template library which > functionally replaces lex and yacc, and reads like EBNF). The old code is > for > a runtime polymorphic parser for RS274 (CNC g-code). I was experimenting > with > parsing historic CNC code, and play with different geometry engines. This > old > project is WAY down on my to-get-around-to list, but it might be an > interesting testbed for something later on. This would be useful for stuff > like a motion controller for the RepRap and DIY 3D Fab machines. > > For that matter, is there anything like Spirit++ <http://boost-spirit.com> > in > Plan9/Limbo? When I developed the code for my fist thesis I developed a > parser for some oddball finite difference equations produced by some > modeling > software. It was nice to have the code read like EBNF. > > Spirit is pretty amazing in that you can write very succinct and powerful code with it, and yet it is terrible in that when you get an error message you can end up with quite a disaster (due to the templates). I particularly like the Phoenix submodule in the new boost libraries for writing little functional tidbits with STL looping constructs and iterators. Speaking of which, Spirit appears to borrow pretty heavily from Parsec which is a monadic parser combinator framework for Haskell that I believe will work with Hugs, that also works on Plan 9. (I think Andrey ported Hugs a long time ago). I'd LOVE to have GHC on Plan 9, but my first attempts to get anything building with it were pretty frustrating. There's scheme language implementations available for plan 9, and I think one could either find or come up with a parsing framework like spirit for scheme that relied on continuations (more "native" to scheme than monads, though you could use monads too). Dave > EBo -- > > > [-- Attachment #2: Type: text/html, Size: 3606 bytes --] ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 13:31 [9fans] Binary format Enrico Weigelt 2010-02-17 8:50 ` Jacob Todd 2010-02-17 14:06 ` Gorka Guardiola @ 2010-02-17 14:38 ` blstuart 2010-02-17 14:58 ` Enrico Weigelt 2 siblings, 1 reply; 32+ messages in thread From: blstuart @ 2010-02-17 14:38 UTC (permalink / raw) To: weigelt, 9fans > * the whole loader sits in the kernel (maybe w/ some additional > helper deamon in userland), but userland can pass parameters > like search pathes, etc via env. > > IMHO having the dynamic loader in kernel-land (in contrary to ELF > on GNU) not just removes the need for lots of syscalls, but allows > a lot of optimizations, eg. code sharing w/ mmap() or caching > relocation data. If you're interested in how to do dynamic loading in a clean and elegant way, take a look at Inferno. Along the lines of what you're suggesting, everything is a module. The only thing necessary to make a module suitable as a program to run from the command line is the existence of a function called init() with a particular signature. After studying and working with it, all of the other dynamic loading implementations out there are... well the word that comes to mind is 'disgusting.' BLS ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:38 ` blstuart @ 2010-02-17 14:58 ` Enrico Weigelt 2010-02-17 15:25 ` blstuart 0 siblings, 1 reply; 32+ messages in thread From: Enrico Weigelt @ 2010-02-17 14:58 UTC (permalink / raw) To: 9fans * blstuart@bellsouth.net <blstuart@bellsouth.net> wrote: > If you're interested in how to do dynamic loading in a clean > and elegant way, take a look at Inferno. hmm, isnt this an interpreter-based system ? > Along the lines of what you're suggesting, everything is a module. > The only thing necessary to make a module suitable as a program to > run from the command line is the existence of a function > called init() with a particular signature. yes, definitivly a good way. reminds me a bit on Oberon, where commands are simply parameter-less procedures in some module. cu -- ---------------------------------------------------------------------- Enrico Weigelt, metux IT service -- http://www.metux.de/ phone: +49 36207 519931 email: weigelt@metux.de mobile: +49 174 7066481 icq: 210169427 skype: nekrad666 ---------------------------------------------------------------------- Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme ---------------------------------------------------------------------- ^ permalink raw reply [flat|nested] 32+ messages in thread
* Re: [9fans] Binary format 2010-02-17 14:58 ` Enrico Weigelt @ 2010-02-17 15:25 ` blstuart 0 siblings, 0 replies; 32+ messages in thread From: blstuart @ 2010-02-17 15:25 UTC (permalink / raw) To: weigelt, 9fans > * blstuart@bellsouth.net <blstuart@bellsouth.net> wrote: > >> If you're interested in how to do dynamic loading in a clean >> and elegant way, take a look at Inferno. > > hmm, isnt this an interpreter-based system ? The application language is Limbo which is compiled to Dis machine code. Dis is run in a virtual machine in the kernel. If you want, you can turn on a JIT compiler which translates the Dis code to native code on the fly, but I haven't usually found it to be all that much benefit. BLS ^ permalink raw reply [flat|nested] 32+ messages in thread
end of thread, other threads:[~2010-02-18 15:16 UTC | newest] Thread overview: 32+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-02-17 13:31 [9fans] Binary format Enrico Weigelt 2010-02-17 8:50 ` Jacob Todd 2010-02-17 13:58 ` Enrico Weigelt 2010-02-17 14:01 ` erik quanstrom 2010-02-17 14:06 ` Gorka Guardiola 2010-02-17 14:13 ` David Leimbach 2010-02-17 14:33 ` Enrico Weigelt 2010-02-17 14:55 ` Steve Simon 2010-02-17 15:04 ` Enrico Weigelt 2010-02-17 15:29 ` erik quanstrom 2010-02-17 15:48 ` blstuart 2010-02-17 17:54 ` lucio 2010-02-17 18:37 ` Lyndon Nerenberg (VE6BBM/VE7TFX) 2010-02-17 19:00 ` Nick LaForge 2010-02-18 15:08 ` Patrick Kelly 2010-02-17 15:58 ` David Leimbach 2010-02-17 16:14 ` Stuart Morrow 2010-02-17 16:28 ` David Leimbach 2010-02-17 18:21 ` Enrico Weigelt 2010-02-17 18:34 ` David Leimbach 2010-02-17 19:02 ` Enrico Weigelt 2010-02-17 19:35 ` Enrico Weigelt 2010-02-17 18:38 ` Corey Thomasson 2010-02-18 15:16 ` Patrick Kelly 2010-02-17 15:13 ` Robert Raschke 2010-02-17 21:32 ` Jacob Todd 2010-02-17 21:26 ` Nathaniel W Filardo 2010-02-17 23:16 ` EBo 2010-02-17 23:22 ` David Leimbach 2010-02-17 14:38 ` blstuart 2010-02-17 14:58 ` Enrico Weigelt 2010-02-17 15:25 ` blstuart
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).