Making a shared library that intercepts call to main work with musl compiled binary

Making a shared library that intercepts call to main work with musl compiled binary

[Riyad Parvez]

[-- Attachment #1: Type: text/plain, Size: 579 bytes --]

Hi All,

I've shared library [1] which intercepts loading of a binary and does some
pre-processing with the command-line arguments via "LD_PRELOAD" technique.
The shared library works fine with glibc compiled binary i.e., it can
intercept the call to main and does some preprocessing. But it can't
intercept call to "main" in musl-compiled binary. I've compiled my binary
in static mode with musl "CC=/musl-gcc -static". How can I make the shared
library work with musl-compiled binary?

Thanks
Riyad

[1]: https://github.com/dslab-epfl/s2e/blob/master/guest/init_env/init_env.c

[-- Attachment #2: Type: text/html, Size: 827 bytes --]

Re: Making a shared library that intercepts call to main work with musl compiled binary

[Rich Felker]

On Mon, Jul 06, 2015 at 01:00:52PM -0400, Riyad Parvez wrote:
> Hi All,
> 
> I've shared library [1] which intercepts loading of a binary and does some
> pre-processing with the command-line arguments via "LD_PRELOAD" technique.
> The shared library works fine with glibc compiled binary i.e., it can
> intercept the call to main and does some preprocessing. But it can't
> intercept call to "main" in musl-compiled binary. I've compiled my binary
> in static mode with musl "CC=/musl-gcc -static". How can I make the shared
> library work with musl-compiled binary?
> 
> Thanks
> Riyad
> 
> [1]: https://github.com/dslab-epfl/s2e/blob/master/guest/init_env/init_env.c

I haven't read this code in detail, so I don't have any advice on
whether there are other problems, but the basic issue seems to be that
you're static linking. The same would happen if you use glibc and
static link. If you want the ability to use LD_PRELOAD you need to
link dynamically.

Rich

Re: Making a shared library that intercepts call to main work with musl compiled binary

[Alexander Monakov]

On Mon, 6 Jul 2015, Rich Felker wrote:
> On Mon, Jul 06, 2015 at 01:00:52PM -0400, Riyad Parvez wrote:
> > Hi All,
> > 
> > I've shared library [1] which intercepts loading of a binary and does some
> > pre-processing with the command-line arguments via "LD_PRELOAD" technique.
> > The shared library works fine with glibc compiled binary i.e., it can
> > intercept the call to main and does some preprocessing. But it can't
> > intercept call to "main" in musl-compiled binary. I've compiled my binary
> > in static mode with musl "CC=/musl-gcc -static". How can I make the shared
> > library work with musl-compiled binary?
> > 
> > Thanks
> > Riyad
> > 
> > [1]: https://github.com/dslab-epfl/s2e/blob/master/guest/init_env/init_env.c
> 
> I haven't read this code in detail, so I don't have any advice on
> whether there are other problems, but the basic issue seems to be that
> you're static linking. The same would happen if you use glibc and
> static link. If you want the ability to use LD_PRELOAD you need to
> link dynamically.

Furthermore, even with dynamic linking you cannot interpose 'main' in the
executable via LD_PRELOAD.  Even more baffling is that the linked code
interposes __libc_start_main; I'm lost for words that it happens to work with
glibc.

I think a cleaner solution can be obtained by converting __s2e_init_env in the
linked code to a constructor that examines environment variables, and moving
the command line processing to an separate wrapper executable.

Alexander

Re: Making a shared library that intercepts call to main work with musl compiled binary

[Rich Felker]

On Mon, Jul 06, 2015 at 10:00:41PM +0300, Alexander Monakov wrote:
> On Mon, 6 Jul 2015, Rich Felker wrote:
> > On Mon, Jul 06, 2015 at 01:00:52PM -0400, Riyad Parvez wrote:
> > > Hi All,
> > > 
> > > I've shared library [1] which intercepts loading of a binary and does some
> > > pre-processing with the command-line arguments via "LD_PRELOAD" technique.
> > > The shared library works fine with glibc compiled binary i.e., it can
> > > intercept the call to main and does some preprocessing. But it can't
> > > intercept call to "main" in musl-compiled binary. I've compiled my binary
> > > in static mode with musl "CC=/musl-gcc -static". How can I make the shared
> > > library work with musl-compiled binary?
> > > 
> > > Thanks
> > > Riyad
> > > 
> > > [1]: https://github.com/dslab-epfl/s2e/blob/master/guest/init_env/init_env.c
> > 
> > I haven't read this code in detail, so I don't have any advice on
> > whether there are other problems, but the basic issue seems to be that
> > you're static linking. The same would happen if you use glibc and
> > static link. If you want the ability to use LD_PRELOAD you need to
> > link dynamically.
> 
> Furthermore, even with dynamic linking you cannot interpose 'main' in the
> executable via LD_PRELOAD.  Even more baffling is that the linked code
> interposes __libc_start_main; I'm lost for words that it happens to work with
> glibc.

Presumably it interposes __libc_start_main because it can't interpose
main. As far as I can tell the only reason it does this is to inject
fake argv. I'm skeptical as to whether it works correctly.

> I think a cleaner solution can be obtained by converting __s2e_init_env in the
> linked code to a constructor that examines environment variables, and moving
> the command line processing to an separate wrapper executable.

Certainly.

Rich

Re: Making a shared library that intercepts call to main work with musl compiled binary

[Alexander Monakov]

> > Furthermore, even with dynamic linking you cannot interpose 'main' in the
> > executable via LD_PRELOAD.  Even more baffling is that the linked code
> > interposes __libc_start_main; I'm lost for words that it happens to work with
> > glibc.
> 
> Presumably it interposes __libc_start_main because it can't interpose
> main. As far as I can tell the only reason it does this is to inject
> fake argv. I'm skeptical as to whether it works correctly.

I was wrong about interposition on __libc_start_main being somehow surprising
-- I missed that it's called from the main executable's startup routine, not
from the dynamic linker (thanks to Rich for explaining that on IRC!).  So as a
result I expect that original code should usually work in practice.

Alexander