From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/5937 Path: news.gmane.org!not-for-mail From: "piranna@gmail.com" Newsgroups: gmane.linux.lib.musl.general Subject: Re: static build and dlopen Date: Wed, 27 Aug 2014 19:01:37 +0200 Message-ID: References: <20140827152735.GB22308@port70.net> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Trace: ger.gmane.org 1409158945 28169 80.91.229.3 (27 Aug 2014 17:02:25 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Wed, 27 Aug 2014 17:02:25 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-5944-gllmg-musl=m.gmane.org@lists.openwall.com Wed Aug 27 19:02:18 2014 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1XMgbu-00086D-Lq for gllmg-musl@plane.gmane.org; Wed, 27 Aug 2014 19:02:14 +0200 Original-Received: (qmail 14236 invoked by uid 550); 27 Aug 2014 17:02:14 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 14226 invoked from network); 27 Aug 2014 17:02:14 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type:content-transfer-encoding; bh=5Z628LjsQ6sy38pG+cYtpb3+9Q2iUQ1Few+vSKwexxQ=; b=lJAI6jazmBUttk/YOFXFYuvUKI4Fo6VBYIYC7w5/LK/sDllSZHQHlzmgOYiocnQHKU mtjSv4rpiWAA9yN/+6IwDUlJFXRNMEpoaGbdqyEVQfkcEQKOLeEB3PNYTNZnd0ZgA0w7 gjEw1GqSiHMKYHm6NCzmmvxmN8xA83a/uphiENLtijavJoNxxLdgHQJL+GOMhZ9z5VPf Sc8x6rgxuejl5yk4XJd8Bems7fbITxd+nxN7So9NEhqI7QvlFMtksIWI3w760dYhzJO9 mODdbd3u7ttG2AcDxosvMxODiDyoTRpvk9/oXHmV3gxCFygQgfK+EKNYvHnSmUDYC84p vUBw== X-Received: by 10.194.103.200 with SMTP id fy8mr4429296wjb.123.1409158918096; Wed, 27 Aug 2014 10:01:58 -0700 (PDT) In-Reply-To: <20140827152735.GB22308@port70.net> Xref: news.gmane.org gmane.linux.lib.musl.general:5937 Archived-At: >> thought libuv is calling to the system dlopen() function, it's said, >> the musl stub for static builds, making it impossible to load the >> Node.js compiled modules :-( > > dlopen does not work from a statically linked binary > > (it can be made to work but it's more complicated than it seems: > the entire libc should be linked into the application so all > interfaces are present a loaded module may require which partly > defeats the purpose of static linking and there are toolchain > issues with this so it is not implemented) (facepalm) Ok, didn't though about this use case where the dynamically loaded module would use components of the host binary. >> So I ask, does the dlopen() stub really makes sense (the linker is >> intelligent enought to remove unused code on static builds...)? Is >> there any alternative I can do? Is there a flag to allow to use the >> real dlopen() function on static builds, or could it be possible to >> add it? If not, could I be able to use an external dlopen library (and >> could I be able to integrate it easily)? >> > > if the loaded modules depend on the libc api (or transitively any > external dependency) then naive dlopen cannot work with static > linking and there is no way around this > > (btw the same applies to all module systems that can load libraries > written in c, eg. if you tried a statically linked python you would > have the same issue: it works if all dependencies are linked in, it > fails if you load libraries at runtime) The de-facto standard in Node.js is that all the compiled modules are static ones (.a files) with no external dependencies at all, just to make it simpler to move to another environments. This contrast to how Python works, where compiled modules are dynamic ones (.so). I'm honestly not sure what are the low-level details of Node.js compiled modules, but the general idea of Node.js modules is make them as independent as possible and left to Node.js require() function to manage the dependencies, so probably this would also apply to compiled ones... Only point left here is if in fact .node files are in fact .a files with a different extension or if they are wrapped someway, so in that case dlopen() is efectively loading .a files (that would be a surprise to me, but would left open the door to solve this problem...) >> Oh, and if you are thinking about it: yes, this is a farily similar >> use case as when compiling OS kernels (at least hybrid ones, like >> Linux) where you need to compile them statically so it can boot and >> also has support to load compiled modules on runtime, since in this >> case, Node.js is in fact the "kernel" and Linux is just a somewhat HAL >> layer :-P > > neither the kernel nor the loaded kernel modules depend on the libc They don't depend on general-purpose libc, but they are compiled with klibc or other stripped down versions... :-D --=20 "Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton de sitios diferentes, simplemente escribe un sistema operativo Unix." =E2=80=93 Linus Tordvals, creador del sistema operativo Linux