a possible need for MAP_FIXED in ldso/dynlink.c ?

a possible need for MAP_FIXED in ldso/dynlink.c ?

[u-uy74]

Hello,

Running musl-based (1.1.16) Linux binaries (i386) under Linux ABI
on FreeBSD (11.0-RELEASE amd64), with explicit use of the loader like
 /..../libc.so --library-path <something> <prog> <args>
fails when mmap() returns a different address than requested
which is rejected by the musl loader when mapping the executable:
"Not a valid dynamic program",
due to:
        map = ....
                : mmap((void *)addr_min, map_len, prot,
                        MAP_PRIVATE, fd, off_start);
 ...
        /* If the loaded file is not relocatable and the requested address is
         * not available, then the load operation must fail. */
        if (eh->e_type != ET_DYN && addr_min && map!=(void *)addr_min) {
                errno = EBUSY;
                goto error;
 ...

mmap() returning a different address does not necessarily mean that
the requested one is not available.

I wonder whether adding MAP_FIXED to MAP_PRIVATE above would be a
useful approach (conditionally on eh->e_type==ET_EXEC ?).

Adding the MAP_FIXED flag, both conditionally or not, seems to work
around the particular problem but I am unsure about all its implications
and consequences, among others under the current Linux implementation
of the Lunux ABI.

Regards,
Rune

Re: a possible need for MAP_FIXED in ldso/dynlink.c ?

[Rich Felker]

On Sat, Jun 10, 2017 at 12:51:51PM +0200, u-uy74@aetey.se wrote:
> Hello,
> 
> Running musl-based (1.1.16) Linux binaries (i386) under Linux ABI
> on FreeBSD (11.0-RELEASE amd64), with explicit use of the loader like
>  /..../libc.so --library-path <something> <prog> <args>
> fails when mmap() returns a different address than requested
> which is rejected by the musl loader when mapping the executable:
> "Not a valid dynamic program",
> due to:
>         map = ....
>                 : mmap((void *)addr_min, map_len, prot,
>                         MAP_PRIVATE, fd, off_start);
>  ...
>         /* If the loaded file is not relocatable and the requested address is
>          * not available, then the load operation must fail. */
>         if (eh->e_type != ET_DYN && addr_min && map!=(void *)addr_min) {
>                 errno = EBUSY;
>                 goto error;
>  ...
> 
> mmap() returning a different address does not necessarily mean that
> the requested one is not available.
> 
> I wonder whether adding MAP_FIXED to MAP_PRIVATE above would be a
> useful approach (conditionally on eh->e_type==ET_EXEC ?).
> 
> Adding the MAP_FIXED flag, both conditionally or not, seems to work
> around the particular problem but I am unsure about all its implications
> and consequences, among others under the current Linux implementation
> of the Lunux ABI.

Use of MAP_FIXED with a memory range you don't already own is an
invalid and unsafe operation. You may end up mapping over top of
yourself, even. Implementations should honor the requested address
passed to mmap and only fail to provide it if it's already in use.

Basically MAP_FIXED is analogous to dup2, and mmap with a preferred
address but no MAP_FIXED is analogous to fcntl F_DUPFD. Breaking the
latter is popular among security snakeoil products but really has no
benefits, since applications that don't have a reason for requesting a
particular address should, and do, pass 0 as the request.

Rich

Re: a possible need for MAP_FIXED in ldso/dynlink.c ?

[u-uy74]

On Sat, Jun 10, 2017 at 08:26:10AM -0400, Rich Felker wrote:
> On Sat, Jun 10, 2017 at 12:51:51PM +0200, u-uy74@aetey.se wrote:
> > Adding the MAP_FIXED flag, both conditionally or not, seems to work
> > around the particular problem but I am unsure about all its implications
> > and consequences, among others under the current Linux implementation
> > of the Lunux ABI.

> Use of MAP_FIXED with a memory range you don't already own is an
> invalid and unsafe operation. You may end up mapping over top of
> yourself, even.

But the latter should be possible to avoid as long as we know where
ourself is located (?) Or do we?

> Implementations should honor the requested address
> passed to mmap and only fail to provide it if it's already in use.

I did not see this statement anywhere in the mmap() documentation
(and even if I am misinformed, you say "should" not "must").

E.g. http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
says
"When MAP_FIXED is not set, the implementation uses addr in an
implementation-defined manner to arrive at pa."

> applications that don't have a reason for requesting a
> particular address should, and do, pass 0 as the request.

Notwithstanding this, I find the API very unclear about how to
safely check whether the use of a certain address range is impossible
or just differs from the particular "implementation-defined manner"
to derive the returned value.

Does this mean that musl can not implement the explicit dynamic loader
with mere Posix mmap(), only with the "more tightly specified Linux mmap()"?

Regards,
Rune

Re: a possible need for MAP_FIXED in ldso/dynlink.c ?

[Rich Felker]

On Sat, Jun 10, 2017 at 04:32:25PM +0200, u-uy74@aetey.se wrote:
> On Sat, Jun 10, 2017 at 08:26:10AM -0400, Rich Felker wrote:
> > On Sat, Jun 10, 2017 at 12:51:51PM +0200, u-uy74@aetey.se wrote:
> > > Adding the MAP_FIXED flag, both conditionally or not, seems to work
> > > around the particular problem but I am unsure about all its implications
> > > and consequences, among others under the current Linux implementation
> > > of the Lunux ABI.
> 
> > Use of MAP_FIXED with a memory range you don't already own is an
> > invalid and unsafe operation. You may end up mapping over top of
> > yourself, even.
> 
> But the latter should be possible to avoid as long as we know where
> ourself is located (?) Or do we?

Not easily. And you don't know that something else isn't mapped there,
like perhaps a data page needed by the vdso or some other
kernel-mapped code (perhaps sigreturn trampolines). The only way to
know that would be nasty hacks like parsing /proc/self/maps. And the
code that does the mapping is not restricted to running before program
startup. At any later time, any check is subject to TOCTOU races. Use
of MAP_FIXED is simply not safe/not valid except when the caller
already "owns" the address range to be mapped. It's not sufficient to
require that "nothing else be using it"; you really need "I own it and
I authorize it to be replaced with something else".

> > Implementations should honor the requested address
> > passed to mmap and only fail to provide it if it's already in use.
> 
> I did not see this statement anywhere in the mmap() documentation
> (and even if I am misinformed, you say "should" not "must").
> 
> E.g. http://pubs.opengroup.org/onlinepubs/9699919799/functions/mmap.html
> says
> "When MAP_FIXED is not set, the implementation uses addr in an
> implementation-defined manner to arrive at pa."

Indeed, not all implementations will define this in a way that admits
mapping programs at a hard-coded address, but then they're not
suitable for dynamic loading of non-PIE programs.

BTW one reason the standard can't really formally define loading at a
requested address is that pointers that don't point to an existing
object or null are not even a valid concept in the C language.

> > applications that don't have a reason for requesting a
> > particular address should, and do, pass 0 as the request.
> 
> Notwithstanding this, I find the API very unclear about how to
> safely check whether the use of a certain address range is impossible
> or just differs from the particular "implementation-defined manner"
> to derive the returned value.
> 
> Does this mean that musl can not implement the explicit dynamic loader
> with mere Posix mmap(), only with the "more tightly specified Linux mmap()"?

Yes, I think that's accurate. Of course you can use PIE binaries and
then you don't have to care about it, since they can be loaded at any
address.

Rich

Re: a possible need for MAP_FIXED in ldso/dynlink.c ?

[u-uy74]

On Sat, Jun 10, 2017 at 11:01:18AM -0400, Rich Felker wrote:
> On Sat, Jun 10, 2017 at 04:32:25PM +0200, u-uy74@aetey.se wrote:
> > On Sat, Jun 10, 2017 at 08:26:10AM -0400, Rich Felker wrote:
> > > Use of MAP_FIXED with a memory range you don't already own is an
> > > invalid and unsafe operation. You may end up mapping over top of
> > > yourself, even.
> > 
> > But the latter should be possible to avoid as long as we know where
> > ourself is located (?) Or do we?
> 
> Not easily. And you don't know that something else isn't mapped there,
> like perhaps a data page needed by the vdso or some other
> kernel-mapped code (perhaps sigreturn trampolines). The only way to
> know that would be nasty hacks like parsing /proc/self/maps. And the
> code that does the mapping is not restricted to running before program
> startup. At any later time, any check is subject to TOCTOU races. Use

I see. Thanks for explaining this.

> > "When MAP_FIXED is not set, the implementation uses addr in an
> > implementation-defined manner to arrive at pa."
> 
> Indeed, not all implementations will define this in a way that admits
> mapping programs at a hard-coded address, but then they're not
> suitable for dynamic loading of non-PIE programs.

Ok.

> BTW one reason the standard can't really formally define loading at a
> requested address is that pointers that don't point to an existing
> object or null are not even a valid concept in the C language.

Interesting. This thought did not strike me before.

> > Does this mean that musl can not implement the explicit dynamic loader
> > with mere Posix mmap(), only with the "more tightly specified Linux mmap()"?
> 
> Yes, I think that's accurate.

Sigh.

> Of course you can use PIE binaries and
> then you don't have to care about it, since they can be loaded at any
> address.

This does not sound too bad in the end.

I'll see if I can convince everything to be built as PIE. Thanks for
the clarifications!

By the way, even though I do not like adding the extra bytes,
what about something like

--- ldso/dynlink.c.orig   2017-06-10 18:33:45.114169773 +0200
+++ ldso/dynlink.c        2017-06-10 18:36:10.592548594 +0200
@@ -1428,7 +1428,7 @@
                runtime = 1;
                Ehdr *ehdr = (void *)map_library(fd, &app);
                if (!ehdr) {
-                       dprintf(2, "%s: %s: Not a valid dynamic program\n", ldname, argv[0]);
+                       dprintf(2, "%s: %s: Failed to map the dynamic program\n", ldname, argv[0]);
                        _exit(1);
                }
                runtime = 0;

(The dynamic program is not necessarily at fault, it is the kernel
who for unknown reasons decides not to grant the address request,
it may even decide differently at the next try, who knows.)

Regards,
Rune