musl setup attempt

musl setup attempt

[LM]

I tried following the instructions at
http://git.musl-libc.org/cgit/musl/tree/INSTALL for Option 1.  Built
musl on a Debian Squeeze system and while configuring, set the prefix,
exec-prefix and syslibdir to a subdirectory beneath my home directory
and set target to i386.  A few programs seem to build and run fine
with this setup.  For instance, I was able to get bzip2 and diffh to
build and they run right on my Debian system.  However, several other
programs build without any complaints and when I try to run them, I
get input/output error.  I see that when I try to run programs like
bzip2 or some of the utilities with wavpack for instance.  I assumed
that possibly the program couldn't find the loader to load relevant
libraries.  I have /etc/ld-musl-i386.path set to the home directory
where I installed the musl lib files and I have LD_LIBRARY_PATH
environment variable set to that lib directory as well.  When that
didn't work, I even went in and tried to hardcode dynlink.c and
reloc.h to point to my locations.  Still getting the same errors.
Anyone have any ideas what for work-arounds or fixes?

According to the README (
http://git.musl-libc.org/cgit/musl/tree/README ):  "The musl project
is actively seeking contributors, mostly in the areas
of porting, testing, and application compatibility improvement."
What type of help do you need with porting and application
compatibility?  I'm currently working with several Open Source sdl,
fltk, pdcurses and command line based applications and building them
on Windows and FreeBSD.  If I can get musl working the way I hope,
would be interested in getting those Open Source applications working
with it as well.

Thanks.

Sincerely,
Laura
http://www.distasis.com/cpp

Re: musl setup attempt

[Isaac Dunham]

On Wed, 13 Mar 2013 07:54:18 -0400
LM <lmemsm@gmail.com> wrote:

> 
> I tried following the instructions at
> http://git.musl-libc.org/cgit/musl/tree/INSTALL for Option 1.  Built

I presume that means you used current git?

> musl on a Debian Squeeze system and while configuring, set the prefix,
> exec-prefix and syslibdir to a subdirectory beneath my home directory
> and set target to i386.  A few programs seem to build and run fine
> with this setup.  For instance, I was able to get bzip2 and diffh to
> build and they run right on my Debian system.  However, several other
> programs build without any complaints and when I try to run them, I
> get input/output error.  I see that when I try to run programs like
> bzip2 or some of the utilities with wavpack for instance.  I assumed
> that possibly the program couldn't find the loader to load relevant

If that were the issue, you could not run anything. The loader is specified by the specfile musl-gcc uses, and it's what make the program run.  If libraries are missing, the program will generally fail to run (depending on your settings).
So it's probably something else.

> libraries.  I have /etc/ld-musl-i386.path set to the home directory
> where I installed the musl lib files and I have LD_LIBRARY_PATH
> environment variable set to that lib directory as well.  When that
> didn't work, I even went in and tried to hardcode dynlink.c and
> reloc.h to point to my locations.  Still getting the same errors.
> Anyone have any ideas what for work-arounds or fixes?

No ideas just yet, but it would probably be most helpful if you installed strace, then used it on bzip2:
(this assumes prefix=~/; replace everything with appropriate paths)

apt-get install strace
#If I read correctly, bzip2 was affected-
#if not, use a program that is affected.
strace ~/bin/bzip2 -k ~/lib/libc.so 2>musl.strace
ls ~/lib/  #You should see libc.so, libc.so.bz2, and others 

Then either look through musl.strace and find what's just before the io errors, or just attach the file (you'll likely need to compress it, unless the io error cuts it short).

grep -B 7 -i input.output musl-bz2.strace

> According to the README (
> http://git.musl-libc.org/cgit/musl/tree/README ):  "The musl project
> is actively seeking contributors, mostly in the areas
> of porting, testing, and application compatibility improvement."
> What type of help do you need with porting and application
> compatibility?  I'm currently working with several Open Source sdl,
> fltk, pdcurses and command line based applications and building them
> on Windows and FreeBSD.  If I can get musl working the way I hope,
> would be interested in getting those Open Source applications working
> with it as well.
> 
> Thanks.
> 
> Sincerely,
> Laura
> http://www.distasis.com/cpp


-- 
Isaac Dunham <idunham@lavabit.com>

Re: musl setup attempt

[John Spencer]

On 03/13/2013 12:54 PM, LM wrote:
> I tried following the instructions at
> http://git.musl-libc.org/cgit/musl/tree/INSTALL for Option 1.  Built
> musl on a Debian Squeeze system and while configuring, set the prefix,
> exec-prefix and syslibdir to a subdirectory beneath my home directory
> and set target to i386.  A few programs seem to build and run fine
> with this setup.  For instance, I was able to get bzip2 and diffh to
> build and they run right on my Debian system.  However, several other
> programs build without any complaints and when I try to run them, I
> get input/output error.  I see that when I try to run programs like
> bzip2 or some of the utilities with wavpack for instance.  I assumed
> that possibly the program couldn't find the loader to load relevant
> libraries.  I have /etc/ld-musl-i386.path set to the home directory
> where I installed the musl lib files and I have LD_LIBRARY_PATH
> environment variable set to that lib directory as well.  When that
> didn't work, I even went in and tried to hardcode dynlink.c and
> reloc.h to point to my locations.  Still getting the same errors.
> Anyone have any ideas what for work-arounds or fixes?

please see http://wiki.musl-libc.org/Getting_started .
it has detailed instructions about how to use musl from /home.
after installation, the key point is to use "CC=musl-gcc" when running 
configure scripts.

if you still get that error, please provide the exact command you issue 
and the complete error message.
in order to debug dynlinker issues, you can always use
readelf -a mybinary | grep -i interp
that will show you the dynamic linker baked into an executable.
readelf -a mybinary | grep -i needed
will show you which other dynamic libraries are used.

also note that many programs need patches because they're using 
non-portable constructs.
here you can find build instructions for some programs:
https://github.com/rofl0r/sabotage/tree/master/pkg
if patches are referenced from the build script using $K, you can find 
them here
https://github.com/rofl0r/sabotage/tree/master/KEEP
> According to the README (
> http://git.musl-libc.org/cgit/musl/tree/README ):  "The musl project
> is actively seeking contributors, mostly in the areas
> of porting, testing, and application compatibility improvement."
> What type of help do you need with porting and application
> compatibility?
mostly finding portability bugs in programs, like missing include 
directives, and then reporting the issue upstream
(and keep nagging them until they finally fix it).
> I'm currently working with several Open Source sdl,
> fltk, pdcurses and command line based applications and building them
> on Windows and FreeBSD.  If I can get musl working the way I hope,
> would be interested in getting those Open Source applications working
> with it as well.
>
> Thanks.
>
> Sincerely,
> Laura
> http://www.distasis.com/cpp
>

Re: musl setup attempt

[LM]

On Wed, Mar 13, 2013 at 2:49 PM, John Spencer <maillist-musl@barfooze.de> wrote:
> mostly finding portability bugs in programs, like missing include
> directives, and then reporting the issue upstream
> (and keep nagging them until they finally fix it).

I have really mixed results with reporting portability bugs.  More
often than not, projects refuse to accept the bugs unless they're for
platforms they officially support.  I typically work with
cross-platform software, but many of the cross-platform projects still
only officially support a limited number of systems.  Some of them
have even been down-right nasty when I submit a patch to fix an issue
for my platform.  (Of course, I've run across some projects where the
developers have been very nice too and fix things extremely quickly.)
Am very curious if anyone else has had problems with this sort of
thing and how you handle the situation.

I'm currently looking into whether I can find enough Open Source
implementations for the types of applications I need that avoid any
projects where the developers are unfriendly.  I further limited
things down to applications that will run reasonably well on older
systems.  That cuts out a lot of Open Source applications (and
libraries).  Will see if I can still put together a coherent and
useful system under those conditions.

Re: musl setup attempt

[Isaac Dunham]

On Fri, 15 Mar 2013 07:54:39 -0400
LM <lmemsm@gmail.com> wrote:

> I have really mixed results with reporting portability bugs.  More
> often than not, projects refuse to accept the bugs unless they're for
> platforms they officially support.  I typically work with
> cross-platform software, but many of the cross-platform projects still
> only officially support a limited number of systems.  Some of them
> have even been down-right nasty when I submit a patch to fix an issue
> for my platform.  (Of course, I've run across some projects where the
> developers have been very nice too and fix things extremely quickly.)
> Am very curious if anyone else has had problems with this sort of
> thing and how you handle the situation.

A few points:
1) Patches beat bug reports.  Make sure that you note upstream policy about copyright assignments and so on, though.
Also follow the code style upstream uses.
2) Make sure it's not going to break upstream policy.
Examples: don't change -std=c89
3) Make sure it doesn't disable something for other platforms (eg, breaking tests for uclibc)
4) Make it as little change as appropriate
5) If at all possible, test on other platforms.

The best response I had was a trivial patch for libnl (adding a couple headers) which I prepared, tested on musl and glibc, then sent with a comment that it fixed build on musl and worked on glibc. It was applied almost immediately.



-- 
Isaac Dunham <idunham@lavabit.com>

Re: musl setup attempt

[LM]

On 3/15/13, Isaac Dunham <idunham@lavabit.com> wrote:
> A few points:
> 1) Patches beat bug reports.  Make sure that you note upstream policy about
> copyright assignments and so on, though.
> Also follow the code style upstream uses.
> 2) Make sure it's not going to break upstream policy.
> Examples: don't change -std=c89
> 3) Make sure it doesn't disable something for other platforms (eg, breaking
> tests for uclibc)
> 4) Make it as little change as appropriate
> 5) If at all possible, test on other platforms.

Thanks for the points.  It's good to know I'm on the right track.  I
do all of those things.  I do typically provide patches with my bug
reports.  To me, there's usually no point in saying it's broken if I
can't give a fix for it.

> The best response I had was a trivial patch for libnl (adding a couple
> headers) which I prepared, tested on musl and glibc, then sent with a
> comment that it fixed build on musl and worked on glibc. It was applied
> almost immediately.

I've had lots of patches added to software.  I've also had lots of
uncomfortable results where the developers were not very polite (and
this seems to be happening a lot more often in recent years than it
did years ago when I first started doing this).  If you know the
phrase, one bad apples spoils the whole bunch, well that certainly
applies for me.  It's made me rather uncomfortable every time I send
in a patch now and I often think twice before doing so.  I'm not sure
if someone's going to want it or if they're going to be nasty about
it.  I even preface some of my messages now by saying in case you'd
like to support this platform, here's what I had to do to get your
software to compile successfully on this system.

Sincerely,
Laura

Re: musl setup attempt

[Rich Felker]

On Fri, Mar 15, 2013 at 02:11:41PM -0700, Isaac Dunham wrote:
> On Fri, 15 Mar 2013 07:54:39 -0400
> LM <lmemsm@gmail.com> wrote:
> 
> > I have really mixed results with reporting portability bugs.  More
> > often than not, projects refuse to accept the bugs unless they're for
> > platforms they officially support.  I typically work with
> > cross-platform software, but many of the cross-platform projects still
> > only officially support a limited number of systems.  Some of them
> > have even been down-right nasty when I submit a patch to fix an issue
> > for my platform.  (Of course, I've run across some projects where the
> > developers have been very nice too and fix things extremely quickly.)
> > Am very curious if anyone else has had problems with this sort of
> > thing and how you handle the situation.
> 
> A few points:
> 1) Patches beat bug reports. Make sure that you note upstream policy
> about copyright assignments and so on, though.
> Also follow the code style upstream uses.
> 2) Make sure it's not going to break upstream policy.
> Examples: don't change -std=c89
> 3) Make sure it doesn't disable something for other platforms (eg,
> breaking tests for uclibc)
> 4) Make it as little change as appropriate
> 5) If at all possible, test on other platforms.
> 
> The best response I had was a trivial patch for libnl (adding a
> couple headers) which I prepared, tested on musl and glibc, then
> sent with a comment that it fixed build on musl and worked on glibc.
> It was applied almost immediately.

Thank you. This email belongs on the wiki.

Rich

re: musl setup attempt

[LM]

John Spencer wrote:
>please see http://wiki.musl-libc.org/Getting_started .
>it has detailed instructions about how to use musl from /home.
>after installation, the key point is to use "CC=musl-gcc" when running configure scripts.

I am using a custom, portable build system that sets up the
environment, automates the
builds and handles install and uninstall of packages to a specified
location (in this case
within my home directory).  The build system settings include
CC=musl-gcc, CPP=musl-gcc and
CFLAGS and LDFLAGS are set to only point to the paths where the musl
include and library files
are located.  I'm also use -static.

Some of the applications, like diffh, diction, bzip2 are building and
running fine with musl.
If I check objdump -p, I see a different library when built with musl
than I do when I build
them with gcc.

>also note that many programs need patches because they're using non-portable constructs.
>here you can find build instructions for some programs:
>https://github.com/rofl0r/sabotage/tree/master/pkg

Not seeing any patches referenced for xz:
https://github.com/rofl0r/sabotage/blob/master/pkg/xz

Isaac Dunham wrote:
>If that were the issue, you could not run anything. The loader is specified by the specfile
>musl-gcc uses, and it's what make the program run.  If libraries are missing, the program will >generally fail to run (depending on your settings). So it's probably something else.

The applications that are showing errors on my system include programs from xz,
wavpack, libpaper and speex.

Some programs do appear to be running, so likely it is something else.

>...but it would probably be most helpful if you installed strace...

Am not very familiar with strace, but I ran it on some of the programs
that were displaying errors.
The stderr results from using strace on wavpack were:

execvc ("./wavpack, ["./wavpack"],[/* 30 vars */]) = -1  EIO
(Input/output error)
dup(2)                                             = 3
fcntl64 (3,F_GETFL)                                = 0x8001 (flags
O_WRONLY|O_LARGEFILE)
close(3)                                           = 0
write (2, "strace: exec: Input/output error"..., 33 strace: exec:
Input/output error
) = 33
exit_group(1)                                      = ?

The other programs that displayed the same error had very similar
output using strace.

Thanks.

Sincerely,
Laura