* Question regarding dynamic loader @ 2018-11-21 13:55 Gernot Reisinger 2018-11-21 14:25 ` Rich Felker ` (2 more replies) 0 siblings, 3 replies; 8+ messages in thread From: Gernot Reisinger @ 2018-11-21 13:55 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 823 bytes --] Hi, I recently stumbled upon an issue with preloading a shared object into a Go application (see related Go ticket https://github.com/golang/go/issues/28909 ). In short - Go comes with an internal linker which will not link crt code to the application. The entry point will directly execute Go standard library code. As musl libc calls shared object constructors in crt code, the shared objects constructors subsequently will never be invoked. Things will work on glibc systems / processes. it It seems to be a subtle - but in this case wide reaching - behavioral difference to glibc. I wonder if calling constructor functions from crt code is an intended musl libc behavior. My personal - non expert - gut feeling considers glibc behavior "more correct". Is there a chance that musl will change this behavior? br Gernot [-- Attachment #2: Type: text/html, Size: 1013 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 13:55 Question regarding dynamic loader Gernot Reisinger @ 2018-11-21 14:25 ` Rich Felker 2018-11-21 15:52 ` Gernot Reisinger 2018-11-21 16:14 ` Rich Felker 2018-11-21 14:46 ` Szabolcs Nagy 2018-11-23 9:29 ` Florian Weimer 2 siblings, 2 replies; 8+ messages in thread From: Rich Felker @ 2018-11-21 14:25 UTC (permalink / raw) To: musl; +Cc: Gernot Reisinger On Wed, Nov 21, 2018 at 02:55:19PM +0100, Gernot Reisinger wrote: > Hi, > I recently stumbled upon an issue with preloading a shared object into a Go > application (see related Go ticket https://github.com/golang/go/issues/28909 > ). > > In short - Go comes with an internal linker which will not link crt code to > the application. The entry point will directly execute Go standard library > code. As musl libc calls shared object constructors in crt code, the shared I don't think this assessment of what musl does is correct. It calls the (initially loaded) shared object constructor via __libc_start_main. If the program is not entered via __libc_start_main, libc is not usable. Necessary initialization will have been bypassed. This has little to do with whether the crt code was linked, except that *crt1.o is normally responsible for calling __libc_start_main. If the linking process bypasses crt1, it needs to ensure that __libc_start_main ends up getting called in some other way. As far as I know this is also true for glibc, so I'm not sure why it differs. > objects constructors subsequently will never be invoked. Things will work > on glibc systems / processes. it It seems to be a subtle - but in this case > wide reaching - behavioral difference to glibc. > > I wonder if calling constructor functions from crt code is an intended musl > libc behavior. My personal - non expert - gut feeling considers glibc > behavior "more correct". Is there a chance that musl will change this > behavior? The musl behavior here is intentional. For FDPIC targets, it's impossible to run *any* application code, in the main application or shared libraries, before the main application's crt1 has executed, because there are (essentially -- the equivalent of) self-relocations performed at that stage that the dynamic linker can't see. If any ctors were invoked directly by the dynamic linker before passing control the the main application's entry point, they would run without these relocations in the main application having been performed, possibly resulting in runaway-wrong execution. I believe Go is doing some bad hacks here with regard to its C FFI, but it's likely fixable in some reasonable way. We should get more eyes looking at it. Rich ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 14:25 ` Rich Felker @ 2018-11-21 15:52 ` Gernot Reisinger 2018-11-21 16:41 ` Szabolcs Nagy 2018-11-21 16:14 ` Rich Felker 1 sibling, 1 reply; 8+ messages in thread From: Gernot Reisinger @ 2018-11-21 15:52 UTC (permalink / raw) To: dalias; +Cc: musl [-- Attachment #1: Type: text/plain, Size: 2934 bytes --] Thanks for your swift and extensive reply. Your explanations make a lot of sense. (sorry for my sloppy description - __libc_start_main invoked by *crt is the place where constructor calls happen as you did outline). I did no extensive research how glibc executes these constructor calls. At least the call stack indicates that they are partially executed in dynamic linker context - _dl_start_user () in /lib64/ld-linux-x86-64.so calling _dl_init. I will add a reference to reply to the Go ticket. Am Mi., 21. Nov. 2018 um 15:25 Uhr schrieb Rich Felker <dalias@libc.org>: > On Wed, Nov 21, 2018 at 02:55:19PM +0100, Gernot Reisinger wrote: > > Hi, > > I recently stumbled upon an issue with preloading a shared object into a > Go > > application (see related Go ticket > https://github.com/golang/go/issues/28909 > > ). > > > > In short - Go comes with an internal linker which will not link crt code > to > > the application. The entry point will directly execute Go standard > library > > code. As musl libc calls shared object constructors in crt code, the > shared > > I don't think this assessment of what musl does is correct. It calls > the (initially loaded) shared object constructor via > __libc_start_main. If the program is not entered via > __libc_start_main, libc is not usable. Necessary initialization will > have been bypassed. This has little to do with whether the crt code > was linked, except that *crt1.o is normally responsible for calling > __libc_start_main. If the linking process bypasses crt1, it needs to > ensure that __libc_start_main ends up getting called in some other > way. As far as I know this is also true for glibc, so I'm not sure why > it differs. > > > objects constructors subsequently will never be invoked. Things will work > > on glibc systems / processes. it It seems to be a subtle - but in this > case > > wide reaching - behavioral difference to glibc. > > > > I wonder if calling constructor functions from crt code is an intended > musl > > libc behavior. My personal - non expert - gut feeling considers glibc > > behavior "more correct". Is there a chance that musl will change this > > behavior? > > The musl behavior here is intentional. For FDPIC targets, it's > impossible to run *any* application code, in the main application or > shared libraries, before the main application's crt1 has executed, > because there are (essentially -- the equivalent of) self-relocations > performed at that stage that the dynamic linker can't see. If any > ctors were invoked directly by the dynamic linker before passing > control the the main application's entry point, they would run without > these relocations in the main application having been performed, > possibly resulting in runaway-wrong execution. > > I believe Go is doing some bad hacks here with regard to its C FFI, > but it's likely fixable in some reasonable way. We should get more > eyes looking at it. > > Rich > > [-- Attachment #2: Type: text/html, Size: 3587 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 15:52 ` Gernot Reisinger @ 2018-11-21 16:41 ` Szabolcs Nagy 2018-11-23 11:34 ` Gernot Reisinger 0 siblings, 1 reply; 8+ messages in thread From: Szabolcs Nagy @ 2018-11-21 16:41 UTC (permalink / raw) To: musl; +Cc: dalias, Gernot Reisinger * Gernot Reisinger <Gernot.Reisinger@omnino.at> [2018-11-21 16:52:53 +0100]: > I did no extensive research how glibc executes these constructor calls. At > least the call stack indicates that they are partially executed in dynamic > linker context - _dl_start_user () in /lib64/ld-linux-x86-64.so > calling _dl_init. the dynamic linker runs the - preinit_array functions of the main executable, - the init_array and DT_INIT functions of shared libraries. then via __libc_start_main the _init and init_array functions of the main executable are run by libc_nonshared.a code that is linked into the executable. so part of the initialization (main exe) does require entry via __libc_start_main (but this is not an issue for go). however this design can change when glibc introduces a new symbol version for __libc_start_main, so i don't see how go can rely on any of this. ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 16:41 ` Szabolcs Nagy @ 2018-11-23 11:34 ` Gernot Reisinger 0 siblings, 0 replies; 8+ messages in thread From: Gernot Reisinger @ 2018-11-23 11:34 UTC (permalink / raw) To: musl, dalias [-- Attachment #1: Type: text/plain, Size: 1203 bytes --] Thanks a lot for this exhaustive explanation - helps a lot to understand the different initialization stages. I agree, one should not assume a specific execution sequence of these initialization routines. Am Mi., 21. Nov. 2018 um 17:41 Uhr schrieb Szabolcs Nagy <nsz@port70.net>: > * Gernot Reisinger <Gernot.Reisinger@omnino.at> [2018-11-21 16:52:53 > +0100]: > > I did no extensive research how glibc executes these constructor calls. > At > > least the call stack indicates that they are partially executed in > dynamic > > linker context - _dl_start_user () in /lib64/ld-linux-x86-64.so > > calling _dl_init. > > the dynamic linker runs the > - preinit_array functions of the main executable, > - the init_array and DT_INIT functions of shared libraries. > > then via __libc_start_main the _init and init_array functions > of the main executable are run by libc_nonshared.a code that > is linked into the executable. > > so part of the initialization (main exe) does require entry > via __libc_start_main (but this is not an issue for go). > > however this design can change when glibc introduces a new > symbol version for __libc_start_main, so i don't see how > go can rely on any of this. > > [-- Attachment #2: Type: text/html, Size: 1618 bytes --] ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 14:25 ` Rich Felker 2018-11-21 15:52 ` Gernot Reisinger @ 2018-11-21 16:14 ` Rich Felker 1 sibling, 0 replies; 8+ messages in thread From: Rich Felker @ 2018-11-21 16:14 UTC (permalink / raw) To: musl On Wed, Nov 21, 2018 at 09:25:50AM -0500, Rich Felker wrote: > On Wed, Nov 21, 2018 at 02:55:19PM +0100, Gernot Reisinger wrote: > > I wonder if calling constructor functions from crt code is an intended musl > > libc behavior. My personal - non expert - gut feeling considers glibc > > behavior "more correct". Is there a chance that musl will change this > > behavior? > > The musl behavior here is intentional. For FDPIC targets, it's > impossible to run *any* application code, in the main application or > shared libraries, before the main application's crt1 has executed, > because there are (essentially -- the equivalent of) self-relocations > performed at that stage that the dynamic linker can't see. If any > ctors were invoked directly by the dynamic linker before passing > control the the main application's entry point, they would run without > these relocations in the main application having been performed, > possibly resulting in runaway-wrong execution. For reference, this was initially done in commit c87a52103399135d2f57a91a8bcc749d8cb2ca83. Of course these code paths have changed significantly since then, but it gives some historical context. Rich ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 13:55 Question regarding dynamic loader Gernot Reisinger 2018-11-21 14:25 ` Rich Felker @ 2018-11-21 14:46 ` Szabolcs Nagy 2018-11-23 9:29 ` Florian Weimer 2 siblings, 0 replies; 8+ messages in thread From: Szabolcs Nagy @ 2018-11-21 14:46 UTC (permalink / raw) To: musl; +Cc: Gernot Reisinger * Gernot Reisinger <Gernot.Reisinger@omnino.at> [2018-11-21 14:55:19 +0100]: > Hi, > I recently stumbled upon an issue with preloading a shared object into a Go > application (see related Go ticket https://github.com/golang/go/issues/28909 > ). > > In short - Go comes with an internal linker which will not link crt code to > the application. The entry point will directly execute Go standard library then calling into the c runtime later is undefined. crt is required for the c runtime setup. > code. As musl libc calls shared object constructors in crt code, the shared this is not true, the crt code is a tiny stub that calls the __libc_start_main setup code in libc.so, where ctors are run (there are several mechanisms to do ctors, but that's an elf thing: musl supports both _init and initarry style initializers, former is passed as argument to __libc_start_main the latter requires begin/end symbols for the .initarray section, in glibc is similar, but part of the initialization happens in the dynamic linker and part of it in libc_nonshared.a code which should be linked to the application and not in libc.so. static linking is another story but i assume you are using dynamic linking). > objects constructors subsequently will never be invoked. Things will work > on glibc systems / processes. it It seems to be a subtle - but in this case > wide reaching - behavioral difference to glibc. this is libc internal implementation detail that callers should not try to guess or rely on. (however it has to be abi stable within one libc implementation because old crt1.o linked into an executable must work with new libc.so, otoh in glibc the abi can changed over time using symbol versioning for backward compatibility, and there were talks about doing exactly that because the way it runs ctor code is a perfect gadget for rop attacks and present in every executable) > I wonder if calling constructor functions from crt code is an intended musl > libc behavior. My personal - non expert - gut feeling considers glibc > behavior "more correct". Is there a chance that musl will change this what made you think musl calls ctors from crt code? > behavior? > br > Gernot ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Question regarding dynamic loader 2018-11-21 13:55 Question regarding dynamic loader Gernot Reisinger 2018-11-21 14:25 ` Rich Felker 2018-11-21 14:46 ` Szabolcs Nagy @ 2018-11-23 9:29 ` Florian Weimer 2 siblings, 0 replies; 8+ messages in thread From: Florian Weimer @ 2018-11-23 9:29 UTC (permalink / raw) To: Gernot Reisinger; +Cc: musl * Gernot Reisinger: > I recently stumbled upon an issue with preloading a shared object into > a Go application (see related Go ticket > https://github.com/golang/go/issues/28909). The bug here is that Go does not call __libc_start_main at all and does not link crt1.o. That is a major toolchain bug. It's surprising that this works at all. Thanks, Florian ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2018-11-23 11:34 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-11-21 13:55 Question regarding dynamic loader Gernot Reisinger 2018-11-21 14:25 ` Rich Felker 2018-11-21 15:52 ` Gernot Reisinger 2018-11-21 16:41 ` Szabolcs Nagy 2018-11-23 11:34 ` Gernot Reisinger 2018-11-21 16:14 ` Rich Felker 2018-11-21 14:46 ` Szabolcs Nagy 2018-11-23 9:29 ` Florian Weimer
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).