mailing list of musl libc
 help / color / mirror / code / Atom feed
* static build and dlopen
@ 2014-08-27 14:14 piranna
  2014-08-27 15:27 ` Szabolcs Nagy
  2014-08-27 16:43 ` Rich Felker
  0 siblings, 2 replies; 18+ messages in thread
From: piranna @ 2014-08-27 14:14 UTC (permalink / raw)
  To: musl

I have read that when an executable is compiled fully statically
linked with musl, a stub dlopen() is used instead of the real function
(http://www.openwall.com/lists/musl/2012/12/09/19), and seems it's
done in purpose since normally if you compile an executable as
statically linked is because you don't want to load dynamica libraries
(http://openwall.com/lists/musl/2012/12/08/4), but I've achieved an
use case where I need to do both things.

I'm working with others as a hobby on NodeOS (http://node-os.com/), an
operating system where Node.js is running on top of a naked Linux
kernel. To be able to use Node.js from the begging (as the '/init'
executable) it needs to be compiled as a fully statically linked
executable, and except for the OpenSSL support, so far so good thanks
to musl I was able to do it and worked as a charm :-)

Problem came when trying to use a Node.js compiled module ('.node'
files) so /init could be able to capture zombie processes
(https://github.com/piranna/node-century), since it was throwing an
error:


module.js:355
  Module._extensions[extension](this, filename);
                               ^
Error: Module did not self-register.
    at Error (native)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/tmp/ROOT/lib/node_modules/century/demo.js:3:15)
    at Module._compile (module.js:460:26)
    at Object.Module._extensions..js (module.js:478:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)


Since I had installed in my machine the same version of Node.js that I
compiled statically with musl (an Ubuntu package one, compiled
dynamically with glibc) and it worked, digging down I get to the point
that the problem was on the Node.js module loader
(https://github.com/joyent/node/blob/master/lib/module.js#L355), and
looking for the error message
(https://github.com/joyent/node/blob/master/src/node.cc#L2092), I
found it was the Node.js DLOpen() function
(https://github.com/joyent/node/blob/master/src/node.cc#L2056), that
thought libuv is calling to the system dlopen() function, it's said,
the musl stub for static builds, making it impossible to load the
Node.js compiled modules :-(

So I ask, does the dlopen() stub really makes sense (the linker is
intelligent enought to remove unused code on static builds...)? Is
there any alternative I can do? Is there a flag to allow to use the
real dlopen() function on static builds, or could it be possible to
add it? If not, could I be able to use an external dlopen library (and
could I be able to integrate it easily)?

Oh, and if you are thinking about it: yes, this is a farily similar
use case as when compiling OS kernels (at least hybrid ones, like
Linux) where you need to compile them statically so it can boot and
also has support to load compiled modules on runtime, since in this
case, Node.js is in fact the "kernel" and Linux is just a somewhat HAL
layer :-P



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 14:14 static build and dlopen piranna
@ 2014-08-27 15:27 ` Szabolcs Nagy
  2014-08-27 17:01   ` piranna
  2014-08-27 16:43 ` Rich Felker
  1 sibling, 1 reply; 18+ messages in thread
From: Szabolcs Nagy @ 2014-08-27 15:27 UTC (permalink / raw)
  To: musl

* piranna@gmail.com <piranna@gmail.com> [2014-08-27 16:14:07 +0200]:
...
> thought libuv is calling to the system dlopen() function, it's said,
> the musl stub for static builds, making it impossible to load the
> Node.js compiled modules :-(

dlopen does not work from a statically linked binary

(it can be made to work but it's more complicated than it seems:
the entire libc should be linked into the application so all
interfaces are present a loaded module may require which partly
defeats the purpose of static linking and there are toolchain
issues with this so it is not implemented)

> So I ask, does the dlopen() stub really makes sense (the linker is
> intelligent enought to remove unused code on static builds...)? Is
> there any alternative I can do? Is there a flag to allow to use the
> real dlopen() function on static builds, or could it be possible to
> add it? If not, could I be able to use an external dlopen library (and
> could I be able to integrate it easily)?
> 

if the loaded modules depend on the libc api (or transitively any
external dependency) then naive dlopen cannot work with static
linking and there is no way around this

(btw the same applies to all module systems that can load libraries
written in c, eg. if you tried a statically linked python you would
have the same issue: it works if all dependencies are linked in, it
fails if you load libraries at runtime)

> Oh, and if you are thinking about it: yes, this is a farily similar
> use case as when compiling OS kernels (at least hybrid ones, like
> Linux) where you need to compile them statically so it can boot and
> also has support to load compiled modules on runtime, since in this
> case, Node.js is in fact the "kernel" and Linux is just a somewhat HAL
> layer :-P

neither the kernel nor the loaded kernel modules depend on the libc


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 14:14 static build and dlopen piranna
  2014-08-27 15:27 ` Szabolcs Nagy
@ 2014-08-27 16:43 ` Rich Felker
  2014-08-27 17:10   ` piranna
  1 sibling, 1 reply; 18+ messages in thread
From: Rich Felker @ 2014-08-27 16:43 UTC (permalink / raw)
  To: musl

On Wed, Aug 27, 2014 at 04:14:07PM +0200, piranna@gmail.com wrote:
> So I ask, does the dlopen() stub really makes sense (the linker is
> intelligent enought to remove unused code on static builds...)? Is
> there any alternative I can do? Is there a flag to allow to use the
> real dlopen() function on static builds, or could it be possible to
> add it? If not, could I be able to use an external dlopen library (and
> could I be able to integrate it easily)?

As nsz explained, it's not that simple. At some point we want to have
a solution for usage cases like what you want, but it's actually quite
difficult to make it work correctly, and rather than using static
linking directly, it might actually be preferable, for supporting your
usage case, to have a tool which merges the main program, dynamic
linker/libc, and any .so's you want to include statically into one big
file. But this is not entirely trivial either.

As for a possible workaround, you can link your program dynamically
(possibly including most of the libraries that your modules _won't_
need to reference as static-linked in the main program binary) and
include a wrapper script, or wrapper static-linked-binary, to exec
your program explicitly via the dynamic linker, as in:

    /path/to/ld-musl-i386.so.1 -- /path/to/your/node "$@"

or similar. This avoids the need to have musl "installed" on the
target system; everything can be in a self-contained directory. I know
some users are already doing something like this for deployments;
maybe at some point we'll think about making some official tools to
make it easier.

Rich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 15:27 ` Szabolcs Nagy
@ 2014-08-27 17:01   ` piranna
  2014-08-27 17:20     ` Justin Cormack
  0 siblings, 1 reply; 18+ messages in thread
From: piranna @ 2014-08-27 17:01 UTC (permalink / raw)
  To: musl

>> thought libuv is calling to the system dlopen() function, it's said,
>> the musl stub for static builds, making it impossible to load the
>> Node.js compiled modules :-(
>
> dlopen does not work from a statically linked binary
>
> (it can be made to work but it's more complicated than it seems:
> the entire libc should be linked into the application so all
> interfaces are present a loaded module may require which partly
> defeats the purpose of static linking and there are toolchain
> issues with this so it is not implemented)

(facepalm) Ok, didn't though about this use case where the dynamically
loaded module would use components of the host binary.


>> So I ask, does the dlopen() stub really makes sense (the linker is
>> intelligent enought to remove unused code on static builds...)? Is
>> there any alternative I can do? Is there a flag to allow to use the
>> real dlopen() function on static builds, or could it be possible to
>> add it? If not, could I be able to use an external dlopen library (and
>> could I be able to integrate it easily)?
>>
>
> if the loaded modules depend on the libc api (or transitively any
> external dependency) then naive dlopen cannot work with static
> linking and there is no way around this
>
> (btw the same applies to all module systems that can load libraries
> written in c, eg. if you tried a statically linked python you would
> have the same issue: it works if all dependencies are linked in, it
> fails if you load libraries at runtime)

The de-facto standard in Node.js is that all the compiled modules are
static ones (.a files) with no external dependencies at all, just to
make it simpler to move to another environments. This contrast to how
Python works, where compiled modules are dynamic ones (.so). I'm
honestly not sure what are the low-level details of Node.js compiled
modules, but the general idea of Node.js modules is make them as
independent as possible and left to Node.js require() function to
manage the dependencies, so probably this would also apply to compiled
ones... Only point left here is if in fact .node files are in fact .a
files with a different extension or if they are wrapped someway, so in
that case dlopen() is efectively loading .a files (that would be a
surprise to me, but would left open the door to solve this problem...)


>> Oh, and if you are thinking about it: yes, this is a farily similar
>> use case as when compiling OS kernels (at least hybrid ones, like
>> Linux) where you need to compile them statically so it can boot and
>> also has support to load compiled modules on runtime, since in this
>> case, Node.js is in fact the "kernel" and Linux is just a somewhat HAL
>> layer :-P
>
> neither the kernel nor the loaded kernel modules depend on the libc

They don't depend on general-purpose libc, but they are compiled with
klibc or other stripped down versions... :-D


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 16:43 ` Rich Felker
@ 2014-08-27 17:10   ` piranna
  2014-08-27 18:48     ` Laurent Bercot
  0 siblings, 1 reply; 18+ messages in thread
From: piranna @ 2014-08-27 17:10 UTC (permalink / raw)
  To: musl

> As for a possible workaround, you can link your program dynamically
> (possibly including most of the libraries that your modules _won't_
> need to reference as static-linked in the main program binary) and
> include a wrapper script, or wrapper static-linked-binary, to exec
> your program explicitly via the dynamic linker, as in:
>
>     /path/to/ld-musl-i386.so.1 -- /path/to/your/node "$@"
>
> or similar. This avoids the need to have musl "installed" on the
> target system; everything can be in a self-contained directory. I know
> some users are already doing something like this for deployments;
> maybe at some point we'll think about making some official tools to
> make it easier.

Yes, I though about this option before, has a dumb statically linked
executable to work as PID 1 that just only exec Node.js and wait until
it finishes, so I can use a standard dynamically linked one and do
whatever I want, but it's more like an ugly hack and since I was
already thinking to use a statically linked Node.js to make simpler
the filesystem layout, then I followed that path. It's not what I
wanted, but seems it would be the easiest path. At least I've learn a
lot the last week while fighting with this things... :-)


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 17:01   ` piranna
@ 2014-08-27 17:20     ` Justin Cormack
  2014-08-27 20:07       ` piranna
  0 siblings, 1 reply; 18+ messages in thread
From: Justin Cormack @ 2014-08-27 17:20 UTC (permalink / raw)
  To: musl

On Wed, Aug 27, 2014 at 6:01 PM, piranna@gmail.com <piranna@gmail.com> wrote:
> The de-facto standard in Node.js is that all the compiled modules are
> static ones (.a files) with no external dependencies at all, just to
> make it simpler to move to another environments. This contrast to how
> Python works, where compiled modules are dynamic ones (.so). I'm
> honestly not sure what are the low-level details of Node.js compiled
> modules, but the general idea of Node.js modules is make them as
> independent as possible and left to Node.js require() function to
> manage the dependencies, so probably this would also apply to compiled
> ones... Only point left here is if in fact .node files are in fact .a
> files with a different extension or if they are wrapped someway, so in
> that case dlopen() is efectively loading .a files (that would be a
> surprise to me, but would left open the door to solve this problem...)

If this is really true, and they are static, then you should be able
to write a loader that has a function called dlopen etc but which is
not actually the full dynamic linker, just pretends to be, and will
work from a static binary. It could link in the modules at compile
time and just return static pointers (I have done this with Lua code,
pretending to be in a dynamic environment when actually in a static
one, just return pointers from dlsym calls that are fixed at compile
time).

Justin


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 17:10   ` piranna
@ 2014-08-27 18:48     ` Laurent Bercot
  2014-08-27 20:19       ` piranna
  0 siblings, 1 reply; 18+ messages in thread
From: Laurent Bercot @ 2014-08-27 18:48 UTC (permalink / raw)
  To: musl

On 27/08/2014 18:10, piranna@gmail.com wrote:
> Yes, I though about this option before, has a dumb statically linked
> executable to work as PID 1 that just only exec Node.js and wait until
> it finishes, so I can use a standard dynamically linked one and do
> whatever I want

  I don't understand why you can't do whatever you want anyway.
  You can run Node.js as PID 1 even if it is dynamically linked - you
just need to have the libc (and ld-musl.so) in the filesystem. It
will work. You can run anything as PID 1 provided all its dependencies
are there at boot time. Traditional inits are usually dynamically
linked - which I think is a very bad idea, but that's another subject.
If Node.js is a special case that cannot be treated that way, then I'm
interested in hearing why.

-- 
  Laurent


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 17:20     ` Justin Cormack
@ 2014-08-27 20:07       ` piranna
  0 siblings, 0 replies; 18+ messages in thread
From: piranna @ 2014-08-27 20:07 UTC (permalink / raw)
  To: musl

> If this is really true, and they are static, then you should be able
> to write a loader that has a function called dlopen etc but which is
> not actually the full dynamic linker, just pretends to be, and will
> work from a static binary. It could link in the modules at compile
> time and just return static pointers (I have done this with Lua code,
> pretending to be in a dynamic environment when actually in a static
> one, just return pointers from dlsym calls that are fixed at compile
> time).

I'm not fully sure how Node.js compiled modules works, only that the
standard is to generate them statically linked with no external
dependencies. This leads to have full libraries inside each package
(node-webrtc has a full copy of libjingle, for example...), but has
the advantage that the system is more stable to change of dependencies
or of enviroment, since each "project" is fully isolated from a code
and libraries point of view.

Regarding to your comment, are you suggesting to code a dlopen()
function that mimics the behaviour of the dynamic one, but returning
references to code already statically linked on the executable? If so,
it's the same as if I would tape the node modules to the executable
previously to know what the developer would use, and this is
impossible... Have I misunderstood something?

Good trick anyway, I'll pin your message for reference :-)

-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 18:48     ` Laurent Bercot
@ 2014-08-27 20:19       ` piranna
  2014-08-27 20:51         ` Rich Felker
  2014-08-27 20:59         ` Laurent Bercot
  0 siblings, 2 replies; 18+ messages in thread
From: piranna @ 2014-08-27 20:19 UTC (permalink / raw)
  To: musl

>> Yes, I though about this option before, has a dumb statically linked
>> executable to work as PID 1 that just only exec Node.js and wait until
>> it finishes, so I can use a standard dynamically linked one and do
>> whatever I want
>
>  I don't understand why you can't do whatever you want anyway.
>  You can run Node.js as PID 1 even if it is dynamically linked - you
> just need to have the libc (and ld-musl.so) in the filesystem. It
> will work. You can run anything as PID 1 provided all its dependencies
> are there at boot time. Traditional inits are usually dynamically
> linked - which I think is a very bad idea, but that's another subject.
> If Node.js is a special case that cannot be treated that way, then I'm
> interested in hearing why.

I tried to do it that way, but didn't worked. Seems on Linux when you
are using a dynamically linked executable this is not run directly,
but instead it is exec internally /lib/ld-linux.so.2, that's a program
that load and link your executable and it's dynamic libraries and
later exec it, so it takes the PID 1 and when it finish and give
control to your dynamically linked executable (Node.js in this case),
then since PID 1 has exited, the kernel has a kernel panic. The same
happens if using /usr/bin/env as she-bang, since it will get the PID 1
and fail. You can read all that I have learned about this topics on
https://github.com/NodeOS/NodeOS-Docker/pull/12.

By the way, based on this example
(http://www.cs.utah.edu/dept/old/texinfo/glibc-manual-0.02/library_23.html#SEC410)
I've done a dumb /init program that just exec the dynamically linked
Node.js with the real /init in Javascript and it worked. Ugly hack,
but at least it does its job :-)


-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 20:19       ` piranna
@ 2014-08-27 20:51         ` Rich Felker
  2014-08-27 20:59         ` Laurent Bercot
  1 sibling, 0 replies; 18+ messages in thread
From: Rich Felker @ 2014-08-27 20:51 UTC (permalink / raw)
  To: musl

On Wed, Aug 27, 2014 at 10:19:18PM +0200, piranna@gmail.com wrote:
> >> Yes, I though about this option before, has a dumb statically linked
> >> executable to work as PID 1 that just only exec Node.js and wait until
> >> it finishes, so I can use a standard dynamically linked one and do
> >> whatever I want
> >
> >  I don't understand why you can't do whatever you want anyway.
> >  You can run Node.js as PID 1 even if it is dynamically linked - you
> > just need to have the libc (and ld-musl.so) in the filesystem. It
> > will work. You can run anything as PID 1 provided all its dependencies
> > are there at boot time. Traditional inits are usually dynamically
> > linked - which I think is a very bad idea, but that's another subject.
> > If Node.js is a special case that cannot be treated that way, then I'm
> > interested in hearing why.
> 
> I tried to do it that way, but didn't worked. Seems on Linux when you
> are using a dynamically linked executable this is not run directly,
> but instead it is exec internally /lib/ld-linux.so.2, that's a program
> that load and link your executable and it's dynamic libraries and
> later exec it, so it takes the PID 1 and when it finish and give
> control to your dynamically linked executable (Node.js in this case),
> then since PID 1 has exited, the kernel has a kernel panic. The same
> happens if using /usr/bin/env as she-bang, since it will get the PID 1
> and fail. You can read all that I have learned about this topics on
> https://github.com/NodeOS/NodeOS-Docker/pull/12.

This analysis is incorrect. Dynamic-linked programs do not involve
multiple processes where the dynamic linker runs in the parent process
and exits once the child is loaded; this would break absolutely
everything. Whether you took the approach I described (a wrapper
program that exec's ld-musl with the right arguments) or just using
the dynamic-linked program directly (if you can ensure that the
dynamic linker is available at the right pathname, which I think you
can), it will run as PID 1 and everything will work as expected.

> By the way, based on this example
> (http://www.cs.utah.edu/dept/old/texinfo/glibc-manual-0.02/library_23.html#SEC410)
> I've done a dumb /init program that just exec the dynamically linked
> Node.js with the real /init in Javascript and it worked. Ugly hack,
> but at least it does its job :-)

I think you could just directly use the dynamically linked Node.js
here, without the wrapper; you haven't given any indication of why
this would not work.

Rich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 20:19       ` piranna
  2014-08-27 20:51         ` Rich Felker
@ 2014-08-27 20:59         ` Laurent Bercot
  2014-08-27 21:04           ` Rich Felker
  2014-08-27 22:54           ` Kurt H Maier
  1 sibling, 2 replies; 18+ messages in thread
From: Laurent Bercot @ 2014-08-27 20:59 UTC (permalink / raw)
  To: musl

> I tried to do it that way, but didn't worked. Seems on Linux when you
> are using a dynamically linked executable this is not run directly,
> but instead it is exec internally /lib/ld-linux.so.2, that's a program
> that load and link your executable and it's dynamic libraries and
> later exec it, so it takes the PID 1 and when it finish and give
> control to your dynamically linked executable (Node.js in this case),
> then since PID 1 has exited, the kernel has a kernel panic.

  No, the dynamic linker will run your executable with the same PID.
What happened in your case was that for some reason either the
dynamic linker failed to load your executable, and exited, or
Node.js exited. In any case, the system didn't work as it was
intended to; I can assure you it's supposed to work.
  Remember that with musl, the dynamic linker is called
/lib/ld-musl-${ARCH}.so.1 instead of /lib/ld-linux.so.2 so you might
want to check your setup again.

  That said, unless your Node.js really takes care of everything, it's
a good idea to not run it as PID 1, if only to make your filesystem
read-only at shutdown time.


> I've done a dumb /init program that just exec the dynamically linked
> Node.js with the real /init in Javascript and it worked. Ugly hack,
> but at least it does its job :-)

  By doing this, you are effectively using Node.js as process 1, since
it is your Javascript interpreter. So there is no reason you cannot
run it directly without your hack.

  I'm still concerned about Node.js' ability to shutdown your machine
properly though.

-- 
  Laurent



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 20:59         ` Laurent Bercot
@ 2014-08-27 21:04           ` Rich Felker
  2014-08-27 22:54           ` Kurt H Maier
  1 sibling, 0 replies; 18+ messages in thread
From: Rich Felker @ 2014-08-27 21:04 UTC (permalink / raw)
  To: musl

On Wed, Aug 27, 2014 at 09:59:55PM +0100, Laurent Bercot wrote:
> >I've done a dumb /init program that just exec the dynamically linked
> >Node.js with the real /init in Javascript and it worked. Ugly hack,
> >but at least it does its job :-)
> 
>  By doing this, you are effectively using Node.js as process 1, since
> it is your Javascript interpreter. So there is no reason you cannot
> run it directly without your hack.
> 
>  I'm still concerned about Node.js' ability to shutdown your machine
> properly though.

It could probably be done with the right modules. The goal here seems
to be to have a "pure javascript" system, so I would assume there are
Node.js modules allowing all the system-level stuff to be written in
js.

Rich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 20:59         ` Laurent Bercot
  2014-08-27 21:04           ` Rich Felker
@ 2014-08-27 22:54           ` Kurt H Maier
  2014-08-27 23:24             ` Laurent Bercot
  1 sibling, 1 reply; 18+ messages in thread
From: Kurt H Maier @ 2014-08-27 22:54 UTC (permalink / raw)
  To: musl

Quoting Laurent Bercot <ska-dietlibc@skarnet.org>:

>  I'm still concerned about Node.js' ability to shutdown your machine
> properly though.


Node servers do not shut down; they just devop into the cloud.

Node.js spends a lot of time throwing uncaught exceptions; I suspect
transitioning to S5 will be less common than node just dying.
Perhaps this would be ok with a stateless netbooted ramdisk as a
root fs.

khm



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 22:54           ` Kurt H Maier
@ 2014-08-27 23:24             ` Laurent Bercot
  2014-08-27 23:36               ` Brent Cook
  0 siblings, 1 reply; 18+ messages in thread
From: Laurent Bercot @ 2014-08-27 23:24 UTC (permalink / raw)
  To: musl

> Node.js spends a lot of time throwing uncaught exceptions; I suspect
> transitioning to S5 will be less common than node just dying.

  Then it's all the more reason to *not* run it as process 1.
If Node.js dies unexpectedly, you want to restart it automatically,
or at least to reboot; you do not want a kernel panic and a
maintenance call.


> Perhaps this would be ok with a stateless netbooted ramdisk as a
> root fs.

  Kernel + libc + Node + all the needed node modules, netbooted ?
Now I know why the terminals at my train station are so slow. :P

-- 
  Laurent



^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 23:24             ` Laurent Bercot
@ 2014-08-27 23:36               ` Brent Cook
  2014-09-01 23:38                 ` piranna
  0 siblings, 1 reply; 18+ messages in thread
From: Brent Cook @ 2014-08-27 23:36 UTC (permalink / raw)
  To: musl


On Aug 27, 2014, at 6:24 PM, Laurent Bercot <ska-dietlibc@skarnet.org> wrote:

>> Node.js spends a lot of time throwing uncaught exceptions; I suspect
>> transitioning to S5 will be less common than node just dying.
> 
> Then it's all the more reason to *not* run it as process 1.
> If Node.js dies unexpectedly, you want to restart it automatically,
> or at least to reboot; you do not want a kernel panic and a
> maintenance call.
> 
> 
>> Perhaps this would be ok with a stateless netbooted ramdisk as a
>> root fs.
> 
> Kernel + libc + Node + all the needed node modules, netbooted ?
> Now I know why the terminals at my train station are so slow. :P
> 
> -- 
> Laurent
> 

IIRC the pfsense ands m0n0wall firewalls run PHP as process 1, so I
guess something like this is not unheard of. But, they do run from a
ramdisk as well.


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-08-27 23:36               ` Brent Cook
@ 2014-09-01 23:38                 ` piranna
  2014-09-02  0:38                   ` Rich Felker
  0 siblings, 1 reply; 18+ messages in thread
From: piranna @ 2014-09-01 23:38 UTC (permalink / raw)
  To: musl

Sorry for the delay, I have been busy these days.

First of all, you were right: It possible to use dynamically linked
executables for PID 1, seems I was missing one of the libraries
(probably /lib/ld-linux.so.2... :-/ ). Now a Node.js script is working
as PID 1 and in fact, the /usr/bin/env she-bang is working too, that's
amazing :-D Sorry for have been doing so dumb questions this days,
it's the first time I'm digging so down on Operating System
architecture and I've missed this point (other guys go to the beach on
their holidays... :-P ). Now my changes are uploaded and QEmu is also
able to mount a hard disk image and access to it, all of it from
Node.js (with the help of some compiled Node.js modules, of course).

Regarding to the kernel low-level access, the compiled modules are
working just as bridges between C & Javascript. There are not so much
of them, but you can mount filesystems
(https://github.com/groundwater/node-src-mount), configure network
interfaces (https://github.com/groundwater/node-src-sockios,
https://github.com/groundwater/node-bin-ifconfig), ioctl
(https://github.com/groundwater/node-src-ioctl)... Oh, and also
shutdown the machine
(https://github.com/groundwater/node-bin-shutdown,
https://github.com/groundwater/node-src-reboot) ;-) Obviously at this
moment is really pre-alpha, but seems stable and could be useful to
gain some performance on Node.js machines since all the bloatware is
removed, but the idea is to take advantage of Node.js features,
capabilities and LXC containers (like Docker) to make isolation and
the feeling of the developer that "all the machine is theirs" as added
killer features :-) Obviously all of them must be run with setuid 0,
but at this moment I only needed it only for mounting the filesystem,
no more.

Oh, and if you want harder drugs, you should take a look at
http://runtimejs.org, where some developers works somewhat actively
(like groundwater) on both projects: where NodeOS aims to create a
Linux based operating system where all the user-space is build on top
of Node.js, runtime.js aims to develop an OS kernel in pure Javascript
on top of raw v8 isolates and get some extra performance by delegating
memory protection to them so all process can run on ring 0 (no
context-switchs) and message-based IPC by just exchanging a pointer.
Innovative? yes. Crazy? maybe. Awesome? don't doubt it ;-)

2014-08-28 1:36 GMT+02:00 Brent Cook <busterb@gmail.com>:
>
> On Aug 27, 2014, at 6:24 PM, Laurent Bercot <ska-dietlibc@skarnet.org> wrote:
>
>>> Node.js spends a lot of time throwing uncaught exceptions; I suspect
>>> transitioning to S5 will be less common than node just dying.
>>
>> Then it's all the more reason to *not* run it as process 1.
>> If Node.js dies unexpectedly, you want to restart it automatically,
>> or at least to reboot; you do not want a kernel panic and a
>> maintenance call.
>>
>>
>>> Perhaps this would be ok with a stateless netbooted ramdisk as a
>>> root fs.
>>
>> Kernel + libc + Node + all the needed node modules, netbooted ?
>> Now I know why the terminals at my train station are so slow. :P
>>
>> --
>> Laurent
>>
>
> IIRC the pfsense ands m0n0wall firewalls run PHP as process 1, so I
> guess something like this is not unheard of. But, they do run from a
> ramdisk as well.



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-09-01 23:38                 ` piranna
@ 2014-09-02  0:38                   ` Rich Felker
  2014-09-02  0:49                     ` piranna
  0 siblings, 1 reply; 18+ messages in thread
From: Rich Felker @ 2014-09-02  0:38 UTC (permalink / raw)
  To: musl

On Tue, Sep 02, 2014 at 01:38:55AM +0200, piranna@gmail.com wrote:
> Sorry for the delay, I have been busy these days.
> 
> First of all, you were right: It possible to use dynamically linked
> executables for PID 1, seems I was missing one of the libraries
> (probably /lib/ld-linux.so.2... :-/ ).

A heads-up: This should not exist if you're using musl. If it seems to
be needed, you incorrectly linked against glibc, not musl, or possibly
even a mix of the two, and stuff is likely to break very badly. The
musl dynamic linker is named /lib/ld-musl-$(ARCH).so.1 where $(ARCH)
would be i386 or x86_64 for 32- or 64-bit x86 systems, and something
like mips, mipsel, arm, armhf, etc. for other common systems.

Rich


^ permalink raw reply	[flat|nested] 18+ messages in thread

* Re: static build and dlopen
  2014-09-02  0:38                   ` Rich Felker
@ 2014-09-02  0:49                     ` piranna
  0 siblings, 0 replies; 18+ messages in thread
From: piranna @ 2014-09-02  0:49 UTC (permalink / raw)
  To: musl

Yes I know, I put it as example of the glibc case before I started
using musl to try to solve it since glibc static link didn't worked
for me, I know that musl one is /lib/ld-musl-i386.so.1.

2014-09-02 2:38 GMT+02:00 Rich Felker <dalias@libc.org>:
> On Tue, Sep 02, 2014 at 01:38:55AM +0200, piranna@gmail.com wrote:
>> Sorry for the delay, I have been busy these days.
>>
>> First of all, you were right: It possible to use dynamically linked
>> executables for PID 1, seems I was missing one of the libraries
>> (probably /lib/ld-linux.so.2... :-/ ).
>
> A heads-up: This should not exist if you're using musl. If it seems to
> be needed, you incorrectly linked against glibc, not musl, or possibly
> even a mix of the two, and stuff is likely to break very badly. The
> musl dynamic linker is named /lib/ld-musl-$(ARCH).so.1 where $(ARCH)
> would be i386 or x86_64 for 32- or 64-bit x86 systems, and something
> like mips, mipsel, arm, armhf, etc. for other common systems.
>
> Rich



-- 
"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton de sitios diferentes, simplemente escribe un sistema operativo
Unix."
– Linus Tordvals, creador del sistema operativo Linux


^ permalink raw reply	[flat|nested] 18+ messages in thread

end of thread, other threads:[~2014-09-02  0:49 UTC | newest]

Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-08-27 14:14 static build and dlopen piranna
2014-08-27 15:27 ` Szabolcs Nagy
2014-08-27 17:01   ` piranna
2014-08-27 17:20     ` Justin Cormack
2014-08-27 20:07       ` piranna
2014-08-27 16:43 ` Rich Felker
2014-08-27 17:10   ` piranna
2014-08-27 18:48     ` Laurent Bercot
2014-08-27 20:19       ` piranna
2014-08-27 20:51         ` Rich Felker
2014-08-27 20:59         ` Laurent Bercot
2014-08-27 21:04           ` Rich Felker
2014-08-27 22:54           ` Kurt H Maier
2014-08-27 23:24             ` Laurent Bercot
2014-08-27 23:36               ` Brent Cook
2014-09-01 23:38                 ` piranna
2014-09-02  0:38                   ` Rich Felker
2014-09-02  0:49                     ` piranna

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/musl/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).