dl_iterate_phdr() behaves differently on musl and glibc

dl_iterate_phdr() behaves differently on musl and glibc

[Bob B.]

Hi list.

I noticed a difference in behavior of dl_iterate_phdr() when playing with musl and glibc.
The program I tried was the example on the bottom of the page: https://linux.die.net/man/3/dl_iterate_phdr

What happens is that musl returns the program itself when run + shared objects.
glibc only returns the shared objects.

Not sure which one is right. Or if both are acceptable.
Any light to clear why the different behavior is appreciated.

Thanks all.

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Rich Felker]

On Mon, May 28, 2018 at 03:44:15PM +0200, Bob B. wrote:
> Hi list.
> 
> I noticed a difference in behavior of dl_iterate_phdr() when playing with musl and glibc.
> The program I tried was the example on the bottom of the page: https://linux.die.net/man/3/dl_iterate_phdr
> 
> What happens is that musl returns the program itself when run + shared objects.
> glibc only returns the shared objects.
> 
> Not sure which one is right. Or if both are acceptable.
> Any light to clear why the different behavior is appreciated.

I was unaware of that difference, but omitting the main program seems
like a serious limitation that makes the interface a lot less useful.
I'm not sure if there's any other way to get the information.

The manual page documents the function as enumerating the "shared
objects", which one could interpret only as libraries, but I usually
consider a dynamic program as a "shared object" itself.

Is there a reason musl's behavior is problematic here? Even without
the ambiguity I'd be hesitant to change it since programs may be
relying on it.

Rich

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Bob B.]

> Is there a reason musl's behavior is problematic here? Even without
> the ambiguity I'd be hesitant to change it since programs may be
> relying on it.

Looking around I found that it caused confusion with libwhich[0] (a program like `which` for dynamic libraries).
There's an issue opened[1] about it working differently on musl then it does on other systems.
But I wonder if it causes any problems...

Looking at the code, it looks like Apple systems do the same thing as musl and they treated it individually by skipping the first program on the report (itself)[2]

Might be worth checking what other systems do (BSDs especially I think) before making the decision.

A clarification on POSIX should also take place. Is that a too bureaucratic process?


[0] https://github.com/vtjnash/libwhich
[1] https://github.com/vtjnash/libwhich/issues/6
[2] https://github.com/vtjnash/libwhich/blob/master/libwhich.c#L175

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Rich Felker]

On Mon, May 28, 2018 at 11:39:59PM +0200, Bob B. wrote:
> > Is there a reason musl's behavior is problematic here? Even without
> > the ambiguity I'd be hesitant to change it since programs may be
> > relying on it.
> 
> Looking around I found that it caused confusion with libwhich[0] (a program like `which` for dynamic libraries).
> There's an issue opened[1] about it working differently on musl then it does on other systems.
> But I wonder if it causes any problems...
> 
> Looking at the code, it looks like Apple systems do the same thing as musl and they treated it individually by skipping the first program on the report (itself)[2]

Rather than special-casing knowledge of particular systems I think the
code should just be fixed so that it doesn't break if the main program
is included.

> Might be worth checking what other systems do (BSDs especially I think) before making the decision.
> 
> A clarification on POSIX should also take place. Is that a too bureaucratic process?

POSIX is not relevant since this interface is not defined by POSIX.
I'm rather surprised Apple has it either since it's pretty
ELF-specific...

Rich

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Bob B.]

> Rather than special-casing knowledge of particular systems I think the
> code should just be fixed so that it doesn't break if the main program
> is included.

I'm CC'ing the libwhich people so let them know and discuss.
For context - http://www.openwall.com/lists/musl/2018/05/28/1

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Luc Van Oostenryck]

On Mon, May 28, 2018 at 04:02:43PM -0400, Rich Felker wrote:
> On Mon, May 28, 2018 at 03:44:15PM +0200, Bob B. wrote:
> > Hi list.
> > 
> > I noticed a difference in behavior of dl_iterate_phdr() when playing with musl and glibc.
> > The program I tried was the example on the bottom of the page: https://linux.die.net/man/3/dl_iterate_phdr
> > 
> > What happens is that musl returns the program itself when run + shared objects.
> > glibc only returns the shared objects.
> > 
> > Not sure which one is right. Or if both are acceptable.
> > Any light to clear why the different behavior is appreciated.
>
> I was unaware of that difference, but omitting the main program seems
> like a serious limitation that makes the interface a lot less useful.
> I'm not sure if there's any other way to get the information.
> 
> The manual page documents the function as enumerating the "shared
> objects", which one could interpret only as libraries, but I usually
> consider a dynamic program as a "shared object" itself.
> 

On a recent Ubuntu, the example in the manpage clearly shows that
the entry for the program itself is also present. And I confirm
it effective does:
	Name: "" (9 segments)
	     0: [0x5582f02f5040; memsz:    1f8] flags: 0x4; PT_PHDR
	     1: [0x5582f02f5238; memsz:     1c] flags: 0x4; PT_INTERP
	     2: [0x5582f02f5000; memsz:    bf8] flags: 0x5; PT_LOAD
	     3: [0x5582f04f5da0; memsz:    278] flags: 0x6; PT_LOAD
	     4: [0x5582f04f5db0; memsz:    1f0] flags: 0x6; PT_DYNAMIC
	     5: [0x5582f02f5254; memsz:     44] flags: 0x4; PT_NOTE
	     6: [0x5582f02f5a8c; memsz:     44] flags: 0x4; PT_GNU_EH_FRAME
	     7: [0x5582f02f5000; memsz:      0] flags: 0x6; PT_GNU_STACK
	     8: [0x5582f04f5da0; memsz:    260] flags: 0x4; PT_GNU_RELRO
	Name: "linux-vdso.so.1" (4 segments)
	     0: [0x7ffc899fe000; memsz:   100b] flags: 0x5; PT_LOAD
	     1: [0x7ffc899fe348; memsz:    120] flags: 0x4; PT_DYNAMIC
	     2: [0x7ffc899fe7a8; memsz:     3c] flags: 0x4; PT_NOTE
	     3: [0x7ffc899fe7e4; memsz:     3c] flags: 0x4; PT_GNU_EH_FRAME
	Name: "/lib/x86_64-linux-gnu/libc.so.6" (10 segments)
	     0: [0x7f552960a040; memsz:    230] flags: 0x4; PT_PHDR
	     1: [0x7f55297c7d50; memsz:     1c] flags: 0x4; PT_INTERP
	     2: [0x7f552960a000; memsz: 1e6aa0] flags: 0x5; PT_LOAD
	     3: [0x7f55299f1620; memsz:   94c0] flags: 0x6; PT_LOAD
	     4: [0x7f55299f4b80; memsz:    1e0] flags: 0x6; PT_DYNAMIC
	     5: [0x7f552960a270; memsz:     44] flags: 0x4; PT_NOTE
	     6: [0x7f55299f1620; memsz:     90] flags: 0x4; PT_TLS
	     7: [0x7f55297c7d6c; memsz:   59dc] flags: 0x4; PT_GNU_EH_FRAME
	     8: [0x7f552960a000; memsz:      0] flags: 0x6; PT_GNU_STACK
	     9: [0x7f55299f1620; memsz:   39e0] flags: 0x4; PT_GNU_RELRO
	Name: "/lib64/ld-linux-x86-64.so.2" (7 segments)
	     0: [0x7f55299fb000; memsz:  26c24] flags: 0x5; PT_LOAD
	     1: [0x7f5529c22680; memsz:   1af0] flags: 0x6; PT_LOAD
	     2: [0x7f5529c22e68; memsz:    170] flags: 0x6; PT_DYNAMIC
	     3: [0x7f55299fb1c8; memsz:     24] flags: 0x4; PT_NOTE
	     4: [0x7f5529a1f144; memsz:    66c] flags: 0x4; PT_GNU_EH_FRAME
	     5: [0x7f55299fb000; memsz:      0] flags: 0x6; PT_GNU_STACK
	     6: [0x7f5529c22680; memsz:    980] flags: 0x4; PT_GNU_RELRO

The same manpage can be found at:
	http://man7.org/linux/man-pages/man3/dl_iterate_phdr.3.html

-- Luc

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Jameson Nash]

[-- Attachment #1: Type: text/plain, Size: 1097 bytes --]

Cool, my motivation for creating `libwhich` in part was to be able to
inspect how different systems handle dynamic libraries and this reflection
method.

So seems like the main difference is just that on Ubuntu the first entry
doesn't have a name, and those are explicitly skipped in my code (
https://github.com/vtjnash/libwhich/blob/a11efad57d90fdb95790312d90c8f66d803125cf/libwhich.c#L200
)

Indeed, the example in the Ubuntu manpage also clarifies that: "The first
shared object for which output is displayed (where the name is an empty
string) is the main program"

Which seems to suggest that users should just unconditionally skip the
first entry if they aren't interested in the program name. Does that sound
right?

-Jameson


On Mon, May 28, 2018 at 6:32 PM Bob B. <bob66@gmx.com> wrote:

> > Rather than special-casing knowledge of particular systems I think the
> > code should just be fixed so that it doesn't break if the main program
> > is included.
>
> I'm CC'ing the libwhich people so let them know and discuss.
> For context - http://www.openwall.com/lists/musl/2018/05/28/1
>

[-- Attachment #2: Type: text/html, Size: 1714 bytes --]

Re: dl_iterate_phdr() behaves differently on musl and glibc

[Bob B.]

> So seems like the main difference is just that on Ubuntu the first entry doesn't have a name
Yes, I mistakenly thought it wasn't reported for some reason, but they just don't have a name.

> Which seems to suggest that users should just unconditionally skip the first entry if they aren't interested in the program name.
> Does that sound right?
It sounds right. But the question now is what should libwhich do.
Ignore first entry on musl too or start to report first entry on glibc. (first entry being the program itself).
As libwhich is used to determine what are the dependencies of a program I don't think it's really necessary to point out that it depends on itself. But it doesn't do any harm either... And could simplify code (also taking away the special handle for Apple systems).
Either way it should be consistent across systems.
Nice work! Let us know what you choose.