Re: [musl] building statically linked DSOs with musl

["Alex Xu (Hello71)" <alex_y_xu@yahoo.ca>]

Excerpts from Joakim Sindholt's message of January 1, 2022 7:56 am:
> clock_gettime(), which gettimeofday() uses internally now, wants to
> check if linux provides a user space fast path through the
> injected-into-every-process VDSO.
> When the dynamic linker, in this case glibc's ld.so, loads your binary
> it will do a bunch of internal setup in addition to symbol stitchup and
> whatever else needs to be done. Here you've run into the first of many
> brick walls. Musl will set up its own internal global libc structure
> with a bunch of values during the initial dynamic loading phase; among
> the members is libc.auxv, which is what __vdsosym will look through when
> trying to find the VDSO. Since you never ran musl's dynamic linker (and
> even if your host binary was musl-based, not the one that would have
> initialized the libc.auxv baked in to your statically linked DSO) it
> won't have set up this and a whole host of other things.
> 
> It's not just a question of "just run this setup code and it will work"
> either, as a lot of libc functionality by necessity depends on exclusive
> access and implementation details that can't cross the barrier you've
> set up.
> 
> Best I can figure your only solution, if you want to ship your own
> ecosystem, is to start another process and talk to it over a
> socket/pipe.
> I'm sure this wasn't a very helpful post. Maybe someone else on this
> list has a solution that better fits your needs.
> 
> Joakim
> 

Yes, this is fundamentally impossible. You could avoid much of this 
difficulty by attempting to statically link or otherwise forcibly bind 
your library's dependencies to your alternative malloc; however, this 
merely decreases the size of the issue rather than removing it. First, 
malloc implementations (may) use a process-global resource: (s)brk. It 
will cause havoc if multiple implementations attempt to simultaneously 
manipulate it. Second, even if no more than one implementation uses 
(s)brk, they will still have incompatible metadata. Consider the case 
where your plugin allocates some memory with malloc, then passes the 
pointer to Python which then frees it. Since an incompatible 
implementation is processing it, it will most likely destroy the heap or 
crash.

In conclusion, without extremely careful preparation of the entire 
program and all its DSOs, it is mandatory to use a single libc and 
malloc implementation in all DSOs.

Cheers,
Alex.