perl native musl, ldd

perl native musl, ldd

[Jean-Marc Pigeon]

[-- Attachment #1: Type: text/plain, Size: 2305 bytes --]

Hello,


I compiled perl (native, under gcc-musl) and I have
a non expected result with ldd.

Example: (with a musl system tree chroot)
bash=4/3# cd /usr/lib/perl5/5.20.2/auto/IPC/SysV/
bash-4.3# ldd SysV.so
	ldd (0x7f5d60dd2000)
	libc.so => ldd (0x7f5d60dd2000)
Error relocating SysV.so: Perl_croak: symbol not found
Error relocating SysV.so: Perl_sv_setiv: symbol not found
Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
Error relocating SysV.so: Perl_av_store: symbol not found
etc.

While with glibc (with a glibc system tree chroot)
bash-4.3# cd /usr/lib/perl5/5.20.2/auto/IPC/SysV
bash-4.3# ldd SysV.so
	linux-vdso.so.1 (0x00007fff09556000)
	libc.so.6 => /lib64/libc.so.6 (0x00007f25be856000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f25bee00000)

I am puzzled about those "Error relocating"
native (within musl) perl make didn't return error,
so I am assuming everything went well.

Advices...? hints? do we have a real problem
with musl ldd or I goofed somehow?

;------------------------
Note:
bash-4.3# readelf -l /usr/bin/perl | grep program
There are 7 program headers, starting at offset 64
      [Requesting program interpreter: /lib/ld-musl-x86_64.so.1]

bash-4.3# /usr/bin/perl -v

This is perl 5, version 20, subversion 2 (v5.20.2) built for x86_64-linux

Copyright 1987-2015, Larry Wall

Perl may be copied only under the terms of either the Artistic License
or the
GNU General Public License, which may be found in the Perl 5 source kit.

Complete documentation for Perl, including FAQ lists, should be found on
this system using "man perl" or "perldoc perl".  If you have access to the
Internet, point your browser at http://www.perl.org/, the Perl Home Page.

bash-4.3# ls -ails /usr/bin/ldd
4325457 0 lrwxrwxrwx 1 root root 29 Apr  2 15:11 /usr/bin/ldd ->
../../lib/ld-musl-x86_64.so.1
;-------------------------

Many Thanks for help.
-- 

A bientôt
===========================================================
Jean-Marc Pigeon                        E-Mail: jmp@safe.ca
SAFE Inc.                             Phone: (514) 493-4280
  Clement, 'a kiss solution' to get rid of SPAM (at last)
     Clement' Home base <"http://www.clement.safe.ca">
===========================================================


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3903 bytes --]

Re: perl native musl, ldd

[Rich Felker]

On Thu, Apr 02, 2015 at 11:53:19AM -0400, Jean-Marc Pigeon wrote:
> Hello,
> 
> 
> I compiled perl (native, under gcc-musl) and I have
> a non expected result with ldd.
> 
> Example: (with a musl system tree chroot)
> bash=4/3# cd /usr/lib/perl5/5.20.2/auto/IPC/SysV/
> bash-4.3# ldd SysV.so
> 	ldd (0x7f5d60dd2000)
> 	libc.so => ldd (0x7f5d60dd2000)
> Error relocating SysV.so: Perl_croak: symbol not found
> Error relocating SysV.so: Perl_sv_setiv: symbol not found
> Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
> Error relocating SysV.so: Perl_av_store: symbol not found
> etc.
> 
> While with glibc (with a glibc system tree chroot)
> bash-4.3# cd /usr/lib/perl5/5.20.2/auto/IPC/SysV
> bash-4.3# ldd SysV.so
> 	linux-vdso.so.1 (0x00007fff09556000)
> 	libc.so.6 => /lib64/libc.so.6 (0x00007f25be856000)
> 	/lib64/ld-linux-x86-64.so.2 (0x00007f25bee00000)
> 
> I am puzzled about those "Error relocating"
> native (within musl) perl make didn't return error,
> so I am assuming everything went well.
> 
> Advices...? hints? do we have a real problem
> with musl ldd or I goofed somehow?

Hmm, it seems like glibc ldd does not (or at least does not always)
report symbols that can't be resolved. What seems to be happening is
that the above symbols are intended to be provided by the main program
(the perl interpreter) loading SysV.so rather than resolved via
library dependencies.

I hadn't really considered using ldd on .so files, only on main
programs where this would not be an issue.

If someone can help determine exactly what glibc ldd's behavior is
here (does it always ignore missing symbols? or only for .so files,
and if so, how does it distinguish .so file from main program?) it
should be easy to make musl's behavior match.

Rich

Re: perl native musl, ldd

[Jean-Marc Pigeon]

[-- Attachment #1: Type: text/plain, Size: 2359 bytes --]

On 04/02/2015 12:18 PM, Rich Felker wrote:
> On Thu, Apr 02, 2015 at 11:53:19AM -0400, Jean-Marc Pigeon wrote:
>> Hello,
>>
>>
>> I compiled perl (native, under gcc-musl) and I have
>> a non expected result with ldd.
>>
>> Example: (with a musl system tree chroot)
>> bash=4/3# cd /usr/lib/perl5/5.20.2/auto/IPC/SysV/
>> bash-4.3# ldd SysV.so
>> 	ldd (0x7f5d60dd2000)
>> 	libc.so => ldd (0x7f5d60dd2000)
>> Error relocating SysV.so: Perl_croak: symbol not found
>> Error relocating SysV.so: Perl_sv_setiv: symbol not found
>> Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
>> Error relocating SysV.so: Perl_av_store: symbol not found
>> etc.
>>
>> While with glibc (with a glibc system tree chroot)
>> bash-4.3# cd /usr/lib/perl5/5.20.2/auto/IPC/SysV
>> bash-4.3# ldd SysV.so
>> 	linux-vdso.so.1 (0x00007fff09556000)
>> 	libc.so.6 => /lib64/libc.so.6 (0x00007f25be856000)
>> 	/lib64/ld-linux-x86-64.so.2 (0x00007f25bee00000)
>>
>> I am puzzled about those "Error relocating"
>> native (within musl) perl make didn't return error,
>> so I am assuming everything went well.
>>
>> Advices...? hints? do we have a real problem
>> with musl ldd or I goofed somehow?
> 
> Hmm, it seems like glibc ldd does not (or at least does not always)
> report symbols that can't be resolved. What seems to be happening is
> that the above symbols are intended to be provided by the main program
> (the perl interpreter) loading SysV.so rather than resolved via
> library dependencies.
> 
> I hadn't really considered using ldd on .so files, only on main
> programs where this would not be an issue.
> 
> If someone can help determine exactly what glibc ldd's behavior is
> here (does it always ignore missing symbols? or only for .so files,
> and if so, how does it distinguish .so file from main program?) it
> should be easy to make musl's behavior match.
> 

I am using a packager using ldd to establish
dependencies list.



-- 

A bientôt
===========================================================
Jean-Marc Pigeon                        E-Mail: jmp@safe.ca
SAFE Inc.                             Phone: (514) 493-4280
  Clement, 'a kiss solution' to get rid of SPAM (at last)
     Clement' Home base <"http://www.clement.safe.ca">
===========================================================


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3903 bytes --]

Re: perl native musl, ldd

[u-wsnj]

On Thu, Apr 02, 2015 at 02:17:36PM -0400, Jean-Marc Pigeon wrote:
> >> bash-4.3# ldd SysV.so
> >> 	ldd (0x7f5d60dd2000)
> >> 	libc.so => ldd (0x7f5d60dd2000)
> >> Error relocating SysV.so: Perl_croak: symbol not found
> >> Error relocating SysV.so: Perl_sv_setiv: symbol not found
> >> Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
> >> Error relocating SysV.so: Perl_av_store: symbol not found
> >> etc.

> I am using a packager using ldd to establish
> dependencies list.

I would not say reporting the symbols is a bug, rather that the packager
is kind of relying on UB. Is there a specification of how a program called
"ldd" shall format its output and which data shall be present?

Rune

Re: perl native musl, ldd

[Jean-Marc Pigeon]

[-- Attachment #1: Type: text/plain, Size: 1466 bytes --]

On 04/02/2015 02:48 PM, u-wsnj@aetey.se wrote:
> On Thu, Apr 02, 2015 at 02:17:36PM -0400, Jean-Marc Pigeon wrote:
>>>> bash-4.3# ldd SysV.so
>>>> 	ldd (0x7f5d60dd2000)
>>>> 	libc.so => ldd (0x7f5d60dd2000)
>>>> Error relocating SysV.so: Perl_croak: symbol not found
>>>> Error relocating SysV.so: Perl_sv_setiv: symbol not found
>>>> Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
>>>> Error relocating SysV.so: Perl_av_store: symbol not found
>>>> etc.
> 
>> I am using a packager using ldd to establish
>> dependencies list.
> 
> I would not say reporting the symbols is a bug, rather that the packager
> is kind of relying on UB. Is there a specification of how a program called
> "ldd" shall format its output and which data shall be present?
> 
> Rune
> 
Packager Relying on ldd UB, sure!.
Using ldd was the best way I found to list one package all
dependencies (looking at ELF file type ans searching for
required external components).
If you have a better way (more standard) to propose not using
ldd that will be a good thing. idea?


-- 

A bientôt
===========================================================
Jean-Marc Pigeon                        E-Mail: jmp@safe.ca
SAFE Inc.                             Phone: (514) 493-4280
  Clement, 'a kiss solution' to get rid of SPAM (at last)
     Clement' Home base <"http://www.clement.safe.ca">
===========================================================


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3903 bytes --]

Re: perl native musl, ldd

[Laurent Bercot]

On 02/04/2015 22:40, Jean-Marc Pigeon wrote:
> Using ldd was the best way I found to list one package all
> dependencies (looking at ELF file type ans searching for
> required external components).

  Yeah, there's no way this is gonna work. Dependencies are not
only about dynamic libraries, they could be about executable files,
data files, or something else entirely. ldd would only give you a
very partial idea of a package's dependencies.


> If you have a better way (more standard) to propose not using
> ldd that will be a good thing. idea?

  I'm afraid there's no easy way - you cannot automate dependency
tracking. If you're going to package software, you will have to
manually list all the dependencies for every single one of your
packages.

-- 
  Laurent

Re: perl native musl, ldd

[Nathan McSween]

[-- Attachment #1: Type: text/plain, Size: 1952 bytes --]

Analyse the source, if you can't do that analyse the binary. This is a big
project though, assuming apis are the same (no runtime tests) a
sufficiently useful package manager would be able to find packages that
could say work with a partial compat api and those that could not. The
alternative is to have the gentoo style 1000 monkeys.
On Apr 2, 2015 1:41 PM, "Jean-Marc Pigeon" <jmp@safe.ca> wrote:

> On 04/02/2015 02:48 PM, u-wsnj@aetey.se wrote:
> > On Thu, Apr 02, 2015 at 02:17:36PM -0400, Jean-Marc Pigeon wrote:
> >>>> bash-4.3# ldd SysV.so
> >>>>    ldd (0x7f5d60dd2000)
> >>>>    libc.so => ldd (0x7f5d60dd2000)
> >>>> Error relocating SysV.so: Perl_croak: symbol not found
> >>>> Error relocating SysV.so: Perl_sv_setiv: symbol not found
> >>>> Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
> >>>> Error relocating SysV.so: Perl_av_store: symbol not found
> >>>> etc.
> >
> >> I am using a packager using ldd to establish
> >> dependencies list.
> >
> > I would not say reporting the symbols is a bug, rather that the packager
> > is kind of relying on UB. Is there a specification of how a program
> called
> > "ldd" shall format its output and which data shall be present?
> >
> > Rune
> >
> Packager Relying on ldd UB, sure!.
> Using ldd was the best way I found to list one package all
> dependencies (looking at ELF file type ans searching for
> required external components).
> If you have a better way (more standard) to propose not using
> ldd that will be a good thing. idea?
>
>
> --
>
> A bientôt
> ===========================================================
> Jean-Marc Pigeon                        E-Mail: jmp@safe.ca
> SAFE Inc.                             Phone: (514) 493-4280
>   Clement, 'a kiss solution' to get rid of SPAM (at last)
>      Clement' Home base <"http://www.clement.safe.ca">
> ===========================================================
>
>

[-- Attachment #2: Type: text/html, Size: 2657 bytes --]

Re: perl native musl, ldd

[u-wsnj]

On Thu, Apr 02, 2015 at 04:40:22PM -0400, Jean-Marc Pigeon wrote:
> > I would not say reporting the symbols is a bug, rather that the packager
> > is kind of relying on UB. Is there a specification of how a program called
> > "ldd" shall format its output and which data shall be present?

> Packager Relying on ldd UB, sure!.
> Using ldd was the best way I found to list one package all
> dependencies (looking at ELF file type ans searching for
> required external components).

I am using ldd to find the libraries necessary for binaries and had to
adjust my "one-liner" scripts when I moved on from glibc to musl ldd.

I would suggest that it is your scripts which are to be (easily) adjusted,
not the ldd which from my perspective works just fine, for a purpose very
similar to yours.

> If you have a better way (more standard) to propose not using
> ldd that will be a good thing. idea?

There is no such standard because the concept of a "dependency
on a component" exists only in a context of a certain packaging
system. Software in general does not belong to a single such context,
so there is no clear notion of components, different parties do have
different views.

Rune

Re: perl native musl, ldd

[Rich Felker]

On Thu, Apr 02, 2015 at 04:40:22PM -0400, Jean-Marc Pigeon wrote:
> On 04/02/2015 02:48 PM, u-wsnj@aetey.se wrote:
> > On Thu, Apr 02, 2015 at 02:17:36PM -0400, Jean-Marc Pigeon wrote:
> >>>> bash-4.3# ldd SysV.so
> >>>> 	ldd (0x7f5d60dd2000)
> >>>> 	libc.so => ldd (0x7f5d60dd2000)
> >>>> Error relocating SysV.so: Perl_croak: symbol not found
> >>>> Error relocating SysV.so: Perl_sv_setiv: symbol not found
> >>>> Error relocating SysV.so: Perl_sv_2pv_flags: symbol not found
> >>>> Error relocating SysV.so: Perl_av_store: symbol not found
> >>>> etc.
> > 
> >> I am using a packager using ldd to establish
> >> dependencies list.
> > 
> > I would not say reporting the symbols is a bug, rather that the packager
> > is kind of relying on UB. Is there a specification of how a program called
> > "ldd" shall format its output and which data shall be present?
> > 
> > Rune
> > 
> Packager Relying on ldd UB, sure!.

I think calling this "UB" is a misappropriation of an unrelated term
just to criticize what you're doing. You are relying on a complex
utility that might have OS/libc-specific behavior that doesn't match
your expectations, or that might not even exist on some systems, but
that's not automatically a horrible thing as long as you know your
system(s).

> Using ldd was the best way I found to list one package all
> dependencies (looking at ELF file type ans searching for
> required external components).
> If you have a better way (more standard) to propose not using
> ldd that will be a good thing. idea?

Using readelf (or your own ELF parsing) and looking for the DT_NEEDED
entries would perhaps be a cleaner approach. It would only list direct
dependencies rather than recursive ones and it would not depend on
running target code (i.e. it would work equally well with
cross-compiled packages).

Of course this does not cover all dependencies (non-library things)
but it at least automates a large part of the dependency tracking
process.

Rich

Re: perl native musl, ldd

[Szabolcs Nagy]

* Jean-Marc Pigeon <jmp@safe.ca> [2015-04-02 16:40:22 -0400]:
> Using ldd was the best way I found to list one package all
> dependencies (looking at ELF file type ans searching for
> required external components).
> If you have a better way (more standard) to propose not using
> ldd that will be a good thing. idea?

if you care about the elf dependencies then use the elf standard

an elf binary will have its dependencies in the dynamic section
(DT_NEEDED entries, you can get them by readelf -d or objdump -p)

(of course the names there are mapped to paths in an implementation
defined way and will depend on the runtime environment)

Re: perl native musl, ldd

[Jean-Marc Pigeon]

[-- Attachment #1: Type: text/plain, Size: 1332 bytes --]

On 04/02/2015 05:34 PM, Szabolcs Nagy wrote:
> * Jean-Marc Pigeon <jmp@safe.ca> [2015-04-02 16:40:22 -0400]:
>> Using ldd was the best way I found to list one package all
>> dependencies (looking at ELF file type ans searching for
>> required external components).
>> If you have a better way (more standard) to propose not using
>> ldd that will be a good thing. idea?
> 
> if you care about the elf dependencies then use the elf standard
> 
> an elf binary will have its dependencies in the dynamic section
> (DT_NEEDED entries, you can get them by readelf -d or objdump -p)
> 
> (of course the names there are mapped to paths in an implementation
> defined way and will depend on the runtime environment)
> 
Good idea, I tried to use readelf to do this (few month ago)
but it was not providing the full path. That's why I
decided to use ldd (was in hurry).
I need to redo the function using readelf in a smarter
way. Thanks.


-- 

A bientôt
===========================================================
Jean-Marc Pigeon                        E-Mail: jmp@safe.ca
SAFE Inc.                             Phone: (514) 493-4280
  Clement, 'a kiss solution' to get rid of SPAM (at last)
     Clement' Home base <"http://www.clement.safe.ca">
===========================================================


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3903 bytes --]

bug? in musl libc failed to preserve ownership for

[Jean-Marc Pigeon]

[-- Attachment #1: Type: text/plain, Size: 2132 bytes --]

Hello.

Using cp (from native musl coreutils compilation, not busybox) I am
getting a strange result when doing a copy
(using musls-1.1.8)

test (user:group, no privilege 301:300, doing this as root is no problem):
;-------------------------------------------------------------
mkdir d1
cd d1
touch p1
ln -s p1 lp1
cd ..
cp -a d1 d2
cp: failed to preserve ownership for d2/lp1: Not supported
;--------------------------------------------------------------
Note: same cp from coreutils glibc working fine.

strace extract

getdents64(3, /* 4 entries */, 2048)    = 96
getdents64(3, /* 0 entries */, 2048)    = 0
close(3)                                = 0
lstat("d1/lp1", {st_mode=S_IFLNK|0777, st_size=2, ...}) = 0
readlink("d1/lp1", "p1", 3)             = 2
symlink("p1", "d2/lp1")                 = 0
lstat("d2/lp1", {st_mode=S_IFLNK|0777, st_size=2, ...}) = 0
lchown("d2/lp1", 301, 300)              = 0
newfstatat(AT_FDCWD, "d2/lp1", {st_mode=S_IFLNK|0777, st_size=2, ...},
AT_SYMLINK_NOFOLLOW) = 0
fcntl(1, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
writev(2, [{"cp: ", 4}, {NULL, 0}], 2cp: )  = 4
writev(2, [{"", 0}, {"failed to preserve ownership for"..., 39}],
2failed to preserve ownership for d2/lp1) = 39
writev(2, [{": Not supported", 15}, {NULL, 0}], 2: Not supported) = 15
writev(2, [{"", 0}, {"\n", 1}], 2
)      = 1
lstat("d1/p1", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
open("d1/p1", O_RDONLY|O_NOFOLLOW)      = 3
;----------------------------------------------------------------------

My understanding/feeling, could be within fcntl not
reading symlink file status properly. possible?

however, even if 'cp' is complaining, the duplication
is properly done.

-- 

A bientôt
===========================================================
Jean-Marc Pigeon                        E-Mail: jmp@safe.ca
SAFE Inc.                             Phone: (514) 493-4280
  Clement, 'a kiss solution' to get rid of SPAM (at last)
     Clement' Home base <"http://www.clement.safe.ca">
===========================================================


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 3903 bytes --]

Re: bug? in musl libc failed to preserve ownership for

[Rich Felker]

On Thu, Apr 02, 2015 at 09:18:49PM -0400, Jean-Marc Pigeon wrote:
> Hello.
> 
> Using cp (from native musl coreutils compilation, not busybox) I am
> getting a strange result when doing a copy
> (using musls-1.1.8)
> 
> test (user:group, no privilege 301:300, doing this as root is no problem):
> ;-------------------------------------------------------------
> mkdir d1
> cd d1
> touch p1
> ln -s p1 lp1
> cd ..
> cp -a d1 d2
> cp: failed to preserve ownership for d2/lp1: Not supported
> ;--------------------------------------------------------------
> Note: same cp from coreutils glibc working fine.
> 
> strace extract
> 
> getdents64(3, /* 4 entries */, 2048)    = 96
> getdents64(3, /* 0 entries */, 2048)    = 0
> close(3)                                = 0
> lstat("d1/lp1", {st_mode=S_IFLNK|0777, st_size=2, ...}) = 0
> readlink("d1/lp1", "p1", 3)             = 2
> symlink("p1", "d2/lp1")                 = 0
> lstat("d2/lp1", {st_mode=S_IFLNK|0777, st_size=2, ...}) = 0
> lchown("d2/lp1", 301, 300)              = 0
> newfstatat(AT_FDCWD, "d2/lp1", {st_mode=S_IFLNK|0777, st_size=2, ...},
> AT_SYMLINK_NOFOLLOW) = 0
> fcntl(1, F_GETFL)                       = 0x8002 (flags O_RDWR|O_LARGEFILE)
> writev(2, [{"cp: ", 4}, {NULL, 0}], 2cp: )  = 4
> writev(2, [{"", 0}, {"failed to preserve ownership for"..., 39}],
> 2failed to preserve ownership for d2/lp1) = 39
> writev(2, [{": Not supported", 15}, {NULL, 0}], 2: Not supported) = 15
> writev(2, [{"", 0}, {"\n", 1}], 2
> )      = 1
> lstat("d1/p1", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
> open("d1/p1", O_RDONLY|O_NOFOLLOW)      = 3
> ;----------------------------------------------------------------------
> 
> My understanding/feeling, could be within fcntl not
> reading symlink file status properly. possible?

fcntl is unrelated; it's not acting on the symlink.

I've tried to make sense of the above strace but I don't see anything
in the source that would be causing the newfstatat syscall. The only
code paths I can find in the source that print that message just check
for failure of lchown, and lchown is not failing. I checked current
git master and 8.19 which I had lying around. What specific version
are you using?

Rich