Program with constructor function segfaults frequently with musl

Program with constructor function segfaults frequently with musl

[Bracken Dawson]

[-- Attachment #1: Type: text/plain, Size: 2436 bytes --]

I have been having trouble getting a cgo program to run with musl, it has
been segfaulting frequently and with 'No stack' when run under gdb.

I have managed to reproduce such a failure in pure c with a very small
example:

```
#include <stdio.h>
#include <stdlib.h>
#include <getopt.h>

__attribute__((constructor)) void enter_namespace(int argc, char *argv[]) {
    struct option long_options[] = {
        {"some-option", required_argument, 0, 's'},
        {0,0,0,0}
    };
    int option_index, c, pid;
    while ((c = getopt_long_only(argc, argv, "m:", long_options,
&option_index)) != -1) {
        switch (c) {
            case 's':
                pid = atoi(optarg);
                if (pid < 1) {
                    fprintf(stderr, "Invalid some-option: %s\n", optarg);
                    exit(1);
                }
                break;
            case 0:
                break;
        }
    }
}

int main(void) {
  return 0;
}
```

Run with or without options will segfault frequently:
```
~ # gcc -g test.c
~ # ./a.out
Segmentation fault
~ # ./a.out
~ # ./a.out
~ # ./a.out
~ # ./a.out
Segmentation fault
~ #
~ # gdb ./a.out
GNU gdb (GDB) 7.12.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-alpine-linux-musl".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./a.out...done.
(gdb) r
Starting program: /root/a.out
warning: Error disabling address space randomization: Operation not
permitted
During startup program terminated with signal SIGSEGV, Segmentation fault.
(gdb) bt
No stack.
(gdb)
```
It appears that having any code in a constructor function leads to this
problem, having the same code in the main function does not segfault.

I'm not sure how to take this any further without a backtrace.

I am using musl 1.1.16-r14 on alpine 3.6.0. The code above works with glibc.

Regards,
Bracken Dawson.

:wq

[-- Attachment #2: Type: text/html, Size: 6654 bytes --]

Re: Program with constructor function segfaults frequently with musl

[Szabolcs Nagy]

* Bracken Dawson <abdawson@gmail.com> [2018-03-15 10:38:31 +0000]:
> I have been having trouble getting a cgo program to run with musl, it has
> been segfaulting frequently and with 'No stack' when run under gdb.
> 
> I have managed to reproduce such a failure in pure c with a very small
> example:
> 
> ```
> #include <stdio.h>
> #include <stdlib.h>
> #include <getopt.h>
> 
> __attribute__((constructor)) void enter_namespace(int argc, char *argv[]) {

the arguments passed to ctors are not part of the elf abi
http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#init_fini
(and it cannot really work for dynamically loaded libraries anyway:
the application can arbitrarily clobber argv by that time)

glibc passes these arguments as an extension (the semantics
for dlopened libraries is unclear), which happens to work
since the calling convention of functions with no arguments
allows this on all supported targets.

(note that there are security hardenning solutions that check
the call site function signature against the callee and abort on
mismatch and such extension would not work with that)

is this cgo that tries to capture argv in a ctor or some other
c library? (in either case you should first try to solve it
portably without depending on the glibc extension)

Re: Program with constructor function segfaults frequently with musl

[Szabolcs Nagy]

* Szabolcs Nagy <nsz@port70.net> [2018-03-15 12:01:44 +0100]:
> * Bracken Dawson <abdawson@gmail.com> [2018-03-15 10:38:31 +0000]:
> > I have been having trouble getting a cgo program to run with musl, it has
> > been segfaulting frequently and with 'No stack' when run under gdb.
> > 
> > I have managed to reproduce such a failure in pure c with a very small
> > example:
> > 
> > ```
> > #include <stdio.h>
> > #include <stdlib.h>
> > #include <getopt.h>
> > 
> > __attribute__((constructor)) void enter_namespace(int argc, char *argv[]) {
> 
> the arguments passed to ctors are not part of the elf abi
> http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#init_fini

ah this does not explain the type signature, the right link is
http://www.sco.com/developers/gabi/latest/ch4.sheader.html#init_array

> (and it cannot really work for dynamically loaded libraries anyway:
> the application can arbitrarily clobber argv by that time)
> 
> glibc passes these arguments as an extension (the semantics
> for dlopened libraries is unclear), which happens to work
> since the calling convention of functions with no arguments
> allows this on all supported targets.
> 
> (note that there are security hardenning solutions that check
> the call site function signature against the callee and abort on
> mismatch and such extension would not work with that)
> 
> is this cgo that tries to capture argv in a ctor or some other
> c library? (in either case you should first try to solve it
> portably without depending on the glibc extension)

Re: Program with constructor function segfaults frequently with musl

[Bracken Dawson]

[-- Attachment #1: Type: text/plain, Size: 2056 bytes --]

Sadly my use case is to set a given mnt namespace before go becomes
multi-threaded, which happens before the go main() function, so I do depend
on reading argv in the constructor, I mean I could use a file or something
else, but would rather not.

I guess this is just something I can get away with today in glibc that musl
will never support.

Thanks for looking though.

:wq

On 15 March 2018 at 11:17, Szabolcs Nagy <nsz@port70.net> wrote:

> * Szabolcs Nagy <nsz@port70.net> [2018-03-15 12:01:44 +0100]:
> > * Bracken Dawson <abdawson@gmail.com> [2018-03-15 10:38:31 +0000]:
> > > I have been having trouble getting a cgo program to run with musl, it
> has
> > > been segfaulting frequently and with 'No stack' when run under gdb.
> > >
> > > I have managed to reproduce such a failure in pure c with a very small
> > > example:
> > >
> > > ```
> > > #include <stdio.h>
> > > #include <stdlib.h>
> > > #include <getopt.h>
> > >
> > > __attribute__((constructor)) void enter_namespace(int argc, char
> *argv[]) {
> >
> > the arguments passed to ctors are not part of the elf abi
> > http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#init_fini
>
> ah this does not explain the type signature, the right link is
> http://www.sco.com/developers/gabi/latest/ch4.sheader.html#init_array
>
> > (and it cannot really work for dynamically loaded libraries anyway:
> > the application can arbitrarily clobber argv by that time)
> >
> > glibc passes these arguments as an extension (the semantics
> > for dlopened libraries is unclear), which happens to work
> > since the calling convention of functions with no arguments
> > allows this on all supported targets.
> >
> > (note that there are security hardenning solutions that check
> > the call site function signature against the callee and abort on
> > mismatch and such extension would not work with that)
> >
> > is this cgo that tries to capture argv in a ctor or some other
> > c library? (in either case you should first try to solve it
> > portably without depending on the glibc extension)
>

[-- Attachment #2: Type: text/html, Size: 3179 bytes --]

Re: Program with constructor function segfaults frequently with musl

[Szabolcs Nagy]

* Bracken Dawson <abdawson@gmail.com> [2018-03-15 11:28:56 +0000]:
> Sadly my use case is to set a given mnt namespace before go becomes
> multi-threaded, which happens before the go main() function, so I do depend
> on reading argv in the constructor, I mean I could use a file or something
> else, but would rather not.
> 
> I guess this is just something I can get away with today in glibc that musl
> will never support.
> 

i don't think it is guaranteed that the process is single threaded by
the time your ctor is called (another ctor that runs earlier can easily
create other threads)

so either you have full control over the runtime, in which case you
should be able to do whatever you want before things become multithread
without any hacks, or you have no control over the runtime, which happens
when you use a high level language like go, but then you should not make
any assumptions what happens under the hood since (c)go runtime may
change and break your assumptions in the future.

i think a better solution is e.g. having a simple executable written
in c that does whatever you want and execs the real go binary after
setting things up the right way.

> Thanks for looking though.
> 
> :wq
> 
> On 15 March 2018 at 11:17, Szabolcs Nagy <nsz@port70.net> wrote:
> 
> > * Szabolcs Nagy <nsz@port70.net> [2018-03-15 12:01:44 +0100]:
> > > * Bracken Dawson <abdawson@gmail.com> [2018-03-15 10:38:31 +0000]:
> > > > I have been having trouble getting a cgo program to run with musl, it
> > has
> > > > been segfaulting frequently and with 'No stack' when run under gdb.
> > > >
> > > > I have managed to reproduce such a failure in pure c with a very small
> > > > example:
> > > >
> > > > ```
> > > > #include <stdio.h>
> > > > #include <stdlib.h>
> > > > #include <getopt.h>
> > > >
> > > > __attribute__((constructor)) void enter_namespace(int argc, char
> > *argv[]) {
> > >
> > > the arguments passed to ctors are not part of the elf abi
> > > http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#init_fini
> >
> > ah this does not explain the type signature, the right link is
> > http://www.sco.com/developers/gabi/latest/ch4.sheader.html#init_array
> >
> > > (and it cannot really work for dynamically loaded libraries anyway:
> > > the application can arbitrarily clobber argv by that time)
> > >
> > > glibc passes these arguments as an extension (the semantics
> > > for dlopened libraries is unclear), which happens to work
> > > since the calling convention of functions with no arguments
> > > allows this on all supported targets.
> > >
> > > (note that there are security hardenning solutions that check
> > > the call site function signature against the callee and abort on
> > > mismatch and such extension would not work with that)
> > >
> > > is this cgo that tries to capture argv in a ctor or some other
> > > c library? (in either case you should first try to solve it
> > > portably without depending on the glibc extension)
> >

Re: Program with constructor function segfaults frequently with musl

[Rich Felker]

On Thu, Mar 15, 2018 at 11:28:56AM +0000, Bracken Dawson wrote:
> Sadly my use case is to set a given mnt namespace before go becomes
> multi-threaded, which happens before the go main() function, so I do depend
> on reading argv in the constructor, I mean I could use a file or something
> else, but would rather not.
> 
> I guess this is just something I can get away with today in glibc that musl
> will never support.
> 
> Thanks for looking though.

If you really insist on trying to access the command line from a ctor,
you can open /proc/self/cmdline and read it in. However as noted
before there are problems with assuming the process is not yet
multithreaded just because you're in a ctor and other approaches (exec
via helper) would probably be better.

Rich


> On 15 March 2018 at 11:17, Szabolcs Nagy <nsz@port70.net> wrote:
> 
> > * Szabolcs Nagy <nsz@port70.net> [2018-03-15 12:01:44 +0100]:
> > > * Bracken Dawson <abdawson@gmail.com> [2018-03-15 10:38:31 +0000]:
> > > > I have been having trouble getting a cgo program to run with musl, it
> > has
> > > > been segfaulting frequently and with 'No stack' when run under gdb.
> > > >
> > > > I have managed to reproduce such a failure in pure c with a very small
> > > > example:
> > > >
> > > > ```
> > > > #include <stdio.h>
> > > > #include <stdlib.h>
> > > > #include <getopt.h>
> > > >
> > > > __attribute__((constructor)) void enter_namespace(int argc, char
> > *argv[]) {
> > >
> > > the arguments passed to ctors are not part of the elf abi
> > > http://www.sco.com/developers/gabi/latest/ch5.dynamic.html#init_fini
> >
> > ah this does not explain the type signature, the right link is
> > http://www.sco.com/developers/gabi/latest/ch4.sheader.html#init_array
> >
> > > (and it cannot really work for dynamically loaded libraries anyway:
> > > the application can arbitrarily clobber argv by that time)
> > >
> > > glibc passes these arguments as an extension (the semantics
> > > for dlopened libraries is unclear), which happens to work
> > > since the calling convention of functions with no arguments
> > > allows this on all supported targets.
> > >
> > > (note that there are security hardenning solutions that check
> > > the call site function signature against the callee and abort on
> > > mismatch and such extension would not work with that)
> > >
> > > is this cgo that tries to capture argv in a ctor or some other
> > > c library? (in either case you should first try to solve it
> > > portably without depending on the glibc extension)
> >

Re: Program with constructor function segfaults frequently with musl

[Markus Wichmann]

On Thu, Mar 15, 2018 at 11:28:56AM +0000, Bracken Dawson wrote:
> Sadly my use case is to set a given mnt namespace before go becomes
> multi-threaded, which happens before the go main() function, so I do depend
> on reading argv in the constructor, I mean I could use a file or something
> else, but would rather not.
> 
> I guess this is just something I can get away with today in glibc that musl
> will never support.
> 
> Thanks for looking though.
> 

How about unshare(1)? Suddenly, no hacks are needed anymore...

Ciao,
Markus