* Hi and a few questions @ 2012-05-20 17:03 Richard Pennington 2012-05-20 17:21 ` Rich Felker ` (2 more replies) 0 siblings, 3 replies; 12+ messages in thread From: Richard Pennington @ 2012-05-20 17:03 UTC (permalink / raw) To: musl Hi, I'm putting together a set of tools and libraries for doing embedded C and C++ development using clang/LLVM. The web site describing the project is http://ellcc.org I just discovered musl (from a post on the minux mailing list) and it looks very cool. I'm currently using a partial Linux port of the NetBSD libraries, but it looks like musl might be a better choice. I'm thinking of switching, and I have a few questions, but first a little background. I want to target several processors, including i386, x86_64, arm, mips, microblaze, ppc, and ppc64 so it looks like musl support will have to be added for the currently unsupported processors. I've done some preliminary testing by compiling the Open POSIX Test Suite (http://posixtest.sourceforge.net) three ways: 1. with gcc/glibc, x86_64 2. with clang/LLVM/glibc, x86_64 3. with clang/LLVM/musl, x86_64 The results have been good enough that I'm pretty sure I want to switch: [~/ellcc/posixtestsuite] main% grep PASS logfile.musl | wc 5074 15286 275399 [~/ellcc/posixtestsuite] main% grep PASS logfile.ecc | wc 5381 16143 294510 [~/ellcc/posixtestsuite] main% grep PASS logfile.gcc | wc 5380 16140 294458 Now for my questions: 1. Can musl be built out of the source tree? I'd like to be able to build for different processors in different directories. 2. Are the include/bits files the only include files that differ between processors? 3. Are people actively working on other musl ports? I'd wouldn't want to duplicate their efforts. Sorry for the basic questions, but I just started looking at musl this morning. ;-) -Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:03 Hi and a few questions Richard Pennington @ 2012-05-20 17:21 ` Rich Felker 2012-05-20 19:57 ` Richard Pennington 2012-05-20 20:49 ` Isaac Dunham 2012-05-20 17:53 ` Szabolcs Nagy 2012-05-20 20:28 ` Szabolcs Nagy 2 siblings, 2 replies; 12+ messages in thread From: Rich Felker @ 2012-05-20 17:21 UTC (permalink / raw) To: musl On Sun, May 20, 2012 at 12:03:20PM -0500, Richard Pennington wrote: > I want to target several processors, including i386, x86_64, arm, > mips, microblaze, ppc, and ppc64 so it looks like musl support will > have to be added for the currently unsupported processors. Yes, and I'd be very happy to get support added. The reason for lack of ports is not lack of portability but lack of knowledge about these targets. I read up on ARM and did the ARM port using qemu / Aboriginal Linux boot images just because I found it a bit shameful to only support x86[_64], but I haven't gotten around to doing this with any others. > I've done some preliminary testing by compiling the Open POSIX Test Suite > (http://posixtest.sourceforge.net) three ways: > 1. with gcc/glibc, x86_64 > 2. with clang/LLVM/glibc, x86_64 > 3. with clang/LLVM/musl, x86_64 > > The results have been good enough that I'm pretty sure I want to switch: > > [~/ellcc/posixtestsuite] main% grep PASS logfile.musl | wc > 5074 15286 275399 > [~/ellcc/posixtestsuite] main% grep PASS logfile.ecc | wc > 5381 16143 294510 > [~/ellcc/posixtestsuite] main% grep PASS logfile.gcc | wc > 5380 16140 294458 I suspect the situation is somewhat better than these results indicate. Open POSIX Test Suite, while useful, has a number of tests that are incorrect/invalid (mostly, by virtue of invoking undefined behavior then testing the behavior) that actually test for glibc behavior rather than POSIX. As far as I know, all of the tests musl fails to PASS are either (1) of this form, or (2) testing features (like priority scheduling) that musl does not yet support. > Now for my questions: > 1. Can musl be built out of the source tree? I'd like to be able to build > for different processors in different directories. At present, no. Even if the trivial changes were made to put the .o files somewhere else, there's also the issue of the include/bits symlink (which could actually be removed since arch/$(ARCH) is also in the -I path, but doing so would complicate the install rules and preclude using musl "in-place" without "make install" which is presently possible and a useful setup. I'd welcome ideas (though I'd have to weigh whether to adopt them) for making this possible, but the source is small enough that I wonder if it's really necessary.. > 2. Are the include/bits files the only include files that differ between > processors? No, include/bits is just the _public_ differences to the interfaces. There are also some internal headers in the arch/$(ARCH) directories, but more importantly, musl's build system implicitly replaces any source file foo.c with $(ARCH)/foo.s if the latter exists. Some of these replacements (e.g. all the ones in math) are purely size/speed optimizations, but most are essential, functions that cannot be implemented except in assembly: setjmp/longjmp, clone, dlsym, and some internal stuff: - cancellation-point syscalls: needs to store stack/instruction pointer values so a thread can tell if it is blocked at a cancellation point when a cancellation request arrives. - thread exiting: needs to call munmap on its own stack, obviously without touching the stack after the syscall returns. - setting up the thread pointer register: this is arch-specific. - startup code (crt/*): must convert the initial register/stack contents into arguments for __libc_start_main > 3. Are people actively working on other musl ports? I'd wouldn't want to > duplicate their efforts. No. There's been some talk in the past, but no code. > Sorry for the basic questions, but I just started looking at musl this > morning. ;-) No problem. I don't mind answering at all; these are good questions and it's useful to have the answers on the mailing list archive. Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:21 ` Rich Felker @ 2012-05-20 19:57 ` Richard Pennington 2012-05-20 20:49 ` Isaac Dunham 1 sibling, 0 replies; 12+ messages in thread From: Richard Pennington @ 2012-05-20 19:57 UTC (permalink / raw) To: musl On Sunday, May 20, 2012 01:21:16 PM Rich Felker wrote: > On Sun, May 20, 2012 at 12:03:20PM -0500, Richard Pennington wrote: > > I want to target several processors, including i386, x86_64, arm, > > mips, microblaze, ppc, and ppc64 so it looks like musl support will > > have to be added for the currently unsupported processors. > > Yes, and I'd be very happy to get support added. The reason for lack > of ports is not lack of portability but lack of knowledge about these > targets. I read up on ARM and did the ARM port using qemu / Aboriginal > Linux boot images just because I found it a bit shameful to only > support x86[_64], but I haven't gotten around to doing this with any > others. Good. I'll start working on the ports, then. I also use QEMU for testing, generally using QEMU Linux user space emulation. > I suspect the situation is somewhat better than these results > indicate. Open POSIX Test Suite, while useful, has a number of tests > that are incorrect/invalid (mostly, by virtue of invoking undefined > behavior then testing the behavior) that actually test for glibc > behavior rather than POSIX. As far as I know, all of the tests musl > fails to PASS are either (1) of this form, or (2) testing features > (like priority scheduling) that musl does not yet support. I agree, I haven't done a complete analysis yet, but many of the errors are because -Werror is set and also because of the missing pthread_setschedparam and friends. > > > Now for my questions: > > 1. Can musl be built out of the source tree? I'd like to be able to build > > > > for different processors in different directories. > > At present, no. Even if the trivial changes were made to put the .o > files somewhere else, there's also the issue of the include/bits > symlink (which could actually be removed since arch/$(ARCH) is also in > the -I path, but doing so would complicate the install rules and > preclude using musl "in-place" without "make install" which is > presently possible and a useful setup. > > I'd welcome ideas (though I'd have to weigh whether to adopt them) for > making this possible, but the source is small enough that I wonder if > it's really necessary.. Thinking about it, I agree that changing the build/install doesn't add much value. I think I'll use your normal build/install. > > > 2. Are the include/bits files the only include files that differ between > > > > processors? > > No, include/bits is just the _public_ differences to the interfaces. > There are also some internal headers in the arch/$(ARCH) directories, > but more importantly, musl's build system implicitly replaces any > source file foo.c with $(ARCH)/foo.s if the latter exists. Some of > these replacements (e.g. all the ones in math) are purely size/speed > optimizations, but most are essential, functions that cannot be > implemented except in assembly: setjmp/longjmp, clone, dlsym, and some > internal stuff: > > - cancellation-point syscalls: needs to store stack/instruction > pointer values so a thread can tell if it is blocked at a > cancellation point when a cancellation request arrives. > > - thread exiting: needs to call munmap on its own stack, obviously > without touching the stack after the syscall returns. > > - setting up the thread pointer register: this is arch-specific. > > - startup code (crt/*): must convert the initial register/stack > contents into arguments for __libc_start_main > Understood. I should be able to reuse some of the assembly language support stuff (startup, setjmp/longjmp, etc.) that I've already written for the other targets, I'll just model them after yours. > > 3. Are people actively working on other musl ports? I'd wouldn't want to > > > > duplicate their efforts. > > No. There's been some talk in the past, but no code. Good. I'll get to work. -Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:21 ` Rich Felker 2012-05-20 19:57 ` Richard Pennington @ 2012-05-20 20:49 ` Isaac Dunham 2012-05-20 21:25 ` Rich Felker 1 sibling, 1 reply; 12+ messages in thread From: Isaac Dunham @ 2012-05-20 20:49 UTC (permalink / raw) To: musl On Sun, 20 May 2012 13:21:16 -0400 Rich Felker <dalias@aerifal.cx> wrote: > On Sun, May 20, 2012 at 12:03:20PM -0500, Richard Pennington wrote: > > I want to target several processors, including i386, x86_64, arm, > > mips, microblaze, ppc, and ppc64 so it looks like musl support will > > have to be added for the currently unsupported processors. > > Yes, and I'd be very happy to get support added. The reason for lack > of ports is not lack of portability but lack of knowledge about these > targets. I read up on ARM and did the ARM port using qemu / Aboriginal > Linux boot images just because I found it a bit shameful to only > support x86[_64], but I haven't gotten around to doing this with any > others. There was someone who was asking about portability previously; he has a project that will use arm and mips cpus (this is the project that needs libuv, hence the discussion on IRC) and-if musl works with libuv-he thinks he could convince his boss to fund a port, if one isn't ready ahead of time. Microblaze is one of the oddball CPUs that you can configure without an MMU. Would this project target MMU configurations only? > > Now for my questions: > > 1. Can musl be built out of the source tree? I'd like to be > > able to build for different processors in different directories. > > At present, no. Even if the trivial changes were made to put the .o > files somewhere else, there's also the issue of the include/bits > symlink (which could actually be removed since arch/$(ARCH) is also in > the -I path, but doing so would complicate the install rules and > preclude using musl "in-place" without "make install" which is > presently possible and a useful setup. > > I'd welcome ideas (though I'd have to weigh whether to adopt them) for > making this possible, but the source is small enough that I wonder if > it's really necessary.. For what it's worth, a shadow tree (see lndir(1)) would probably do all that's really needed, if you 1 Get musl source code 2 lndir $MUSL_SOURCE ${MUSL_SOURCE}-${ARCH} 3 Configure and build in ${MUSL_SOURCE}-${ARCH} You might have issues with shadowing after you have built musl (ie, do 3 in-tree, 2, repeat 3 in shadow tree), I wouldn't know for sure. Isaac Dunham ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 20:49 ` Isaac Dunham @ 2012-05-20 21:25 ` Rich Felker 0 siblings, 0 replies; 12+ messages in thread From: Rich Felker @ 2012-05-20 21:25 UTC (permalink / raw) To: musl On Sun, May 20, 2012 at 01:49:36PM -0700, Isaac Dunham wrote: > There was someone who was asking about portability previously; he has a > project that will use arm and mips cpus (this is the project that needs > libuv, hence the discussion on IRC) and-if musl works with libuv-he > thinks he could convince his boss to fund a port, if one isn't ready > ahead of time. Good to know! > Microblaze is one of the oddball CPUs that you can configure without an > MMU. Would this project target MMU configurations only? Well POSIX requires an MMU (or a completely virtualized system that has one), and as far as I know Linux does too (although there's UCLinux or whatever too). I suspect the situation would be the same as how we handle 2.4 kernels: "If some functionality works, great, but let's not do all kinds of crazy things to make stuff work 'better' when it's impossible to work 100% right anyway due to platform limitations." > For what it's worth, a shadow tree (see lndir(1)) would probably do all > that's really needed, if you > 1 Get musl source code > 2 lndir $MUSL_SOURCE ${MUSL_SOURCE}-${ARCH} > 3 Configure and build in ${MUSL_SOURCE}-${ARCH} Yes, this should work very well. > You might have issues with shadowing after you have built musl (ie, do > 3 in-tree, 2, repeat 3 in shadow tree), I wouldn't know for sure. The only major problem I can think of is include/bits getting shadowed, but "make clean" removes it anyway. So even this might work. Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:03 Hi and a few questions Richard Pennington 2012-05-20 17:21 ` Rich Felker @ 2012-05-20 17:53 ` Szabolcs Nagy 2012-05-20 20:01 ` Richard Pennington 2012-05-20 20:44 ` Richard Pennington 2012-05-20 20:28 ` Szabolcs Nagy 2 siblings, 2 replies; 12+ messages in thread From: Szabolcs Nagy @ 2012-05-20 17:53 UTC (permalink / raw) To: musl * Richard Pennington <rich@pennware.com> [2012-05-20 12:03:20 -0500]: > I want to target several processors, including i386, x86_64, arm, mips, > microblaze, ppc, and ppc64 so it looks like musl support will have to be added > for the currently unsupported processors. yes > I've done some preliminary testing by compiling the Open POSIX Test Suite > (http://posixtest.sourceforge.net) three ways: > 1. with gcc/glibc, x86_64 > 2. with clang/LLVM/glibc, x86_64 > 3. with clang/LLVM/musl, x86_64 nice, unfortunately it does not seem to be maintained and it's not updated to posix 2008: "... error: implicit declaration of function 'usleep'" > The results have been good enough that I'm pretty sure I want to switch: > > [~/ellcc/posixtestsuite] main% grep PASS logfile.musl | wc > 5074 15286 275399 > [~/ellcc/posixtestsuite] main% grep PASS logfile.ecc | wc > 5381 16143 294510 > [~/ellcc/posixtestsuite] main% grep PASS logfile.gcc | wc > 5380 16140 294458 > many tests are broken and thus the build fails eg "functional/threads/schedule/1-1.c:22:1: error: "_XOPEN_SOURCE" redefined" they shouldn't define that in the source file (without undefing it first) "conformance/interfaces/pthread_key_create/1-2.c:44: error: control reaches end of non-void function" there are a couple of similar bad tests "conformance/interfaces/aio_read/9-1.c: In function 'main': conformance/interfaces/aio_read/9-1.c:59: error: implicit declaration of function 'open'" they use open without including fcntl.h etc > Now for my questions: > 1. Can musl be built out of the source tree? I'd like to be able to build > for different processors in different directories. what do you mean by out of the source tree? you can set an install prefix for make install if you want to have the .o files for each target in a separate build directory then you can have several git clones or hack the Makefile > 2. Are the include/bits files the only include files that differ between > processors? no, see eg src/thread/$ARCH or src/math/$ARCH > 3. Are people actively working on other musl ports? I'd wouldn't want to > duplicate their efforts. i don't think so mips port was mentioned on irc at some point but i don't think anyone took it up ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:53 ` Szabolcs Nagy @ 2012-05-20 20:01 ` Richard Pennington 2012-05-20 20:44 ` Richard Pennington 1 sibling, 0 replies; 12+ messages in thread From: Richard Pennington @ 2012-05-20 20:01 UTC (permalink / raw) To: musl On Sunday, May 20, 2012 07:53:16 PM Szabolcs Nagy wrote: [snip] > > (http://posixtest.sourceforge.net) three ways: > > 1. with gcc/glibc, x86_64 > > 2. with clang/LLVM/glibc, x86_64 > > 3. with clang/LLVM/musl, x86_64 > > nice, unfortunately it does not seem to be maintained > and it's not updated to posix 2008: > "... error: implicit declaration of function 'usleep'" > Yeah, I noticed that right away. I've made a lot of changes to the test suite already to modernice it for gcc/glibc and clang/LLVM/glibc. [snip] > many tests are broken and thus the build fails eg > > "functional/threads/schedule/1-1.c:22:1: error: "_XOPEN_SOURCE" redefined" > they shouldn't define that in the source file (without undefing it first) > > "conformance/interfaces/pthread_key_create/1-2.c:44: error: control reaches > end of non-void function" there are a couple of similar bad tests > > "conformance/interfaces/aio_read/9-1.c: In function 'main': > conformance/interfaces/aio_read/9-1.c:59: error: implicit declaration of > function 'open'" they use open without including fcntl.h > > etc Many of these problems have already been fixed in my copy. > > > Now for my questions: > > 1. Can musl be built out of the source tree? I'd like to be able to build > > > > for different processors in different directories. > > what do you mean by out of the source tree? > you can set an install prefix for make install > > if you want to have the .o files for each target in a separate build > directory then you can have several git clones or hack the Makefile I've decided not to change the build rules much, if any. It doesn't take long to build the library anyway, so building several times after cleaning is not a big deal. > > > 2. Are the include/bits files the only include files that differ between > > > > processors? > > no, see eg src/thread/$ARCH or src/math/$ARCH Will do. > > > 3. Are people actively working on other musl ports? I'd wouldn't want to > > > > duplicate their efforts. > > i don't think so > mips port was mentioned on irc at some point but i don't think anyone took > it up Thanks for the info. -Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:53 ` Szabolcs Nagy 2012-05-20 20:01 ` Richard Pennington @ 2012-05-20 20:44 ` Richard Pennington 2012-05-20 21:20 ` Rich Felker 1 sibling, 1 reply; 12+ messages in thread From: Richard Pennington @ 2012-05-20 20:44 UTC (permalink / raw) To: musl On Sunday, May 20, 2012 07:53:16 PM Szabolcs Nagy wrote: [snip] > "conformance/interfaces/pthread_key_create/1-2.c:44: error: control reaches > end of non-void function" there are a couple of similar bad tests This one might be real. I think the declaration of void pthread_exit(void *); should be void pthread_exit(void *) __attribute__ ((__noreturn__)); for gcc and clang. Does musl allow this type of attribute usage? I don't see many attributes used in the header files. -Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 20:44 ` Richard Pennington @ 2012-05-20 21:20 ` Rich Felker 0 siblings, 0 replies; 12+ messages in thread From: Rich Felker @ 2012-05-20 21:20 UTC (permalink / raw) To: musl On Sun, May 20, 2012 at 03:44:24PM -0500, Richard Pennington wrote: > On Sunday, May 20, 2012 07:53:16 PM Szabolcs Nagy wrote: > [snip] > > "conformance/interfaces/pthread_key_create/1-2.c:44: error: control reaches > > end of non-void function" there are a couple of similar bad tests > > This one might be real. I think the declaration of > > void pthread_exit(void *); > > should be > > void pthread_exit(void *) __attribute__ ((__noreturn__)); > > for gcc and clang. Does musl allow this type of attribute usage? I don't see > many attributes used in the header files. In principle I'm not opposed to adding attributes under #ifdef __GNUC__ when they have clear benefits to the generated code. If exit functions are such a case, I would consider it. However, on the uClibc list it was noted that the noreturn attribute makes WORSE code from a debugging standpoint by making it impossible to backtrace an exit in the debugger (which can be useful when there's a breakpoint or crash in an atexit or pthread cleanup function). Also, while POSIX does not permit conforming applications to do this, musl allows longjmp out of cleanup handlers (which might have been invoked via pthread_exit or cancellation) and uses this allowance internally in the POSIX timer code (to allow reuse of the timer thread for realtime robustness, i.e. no danger that a thread cannot be created). Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 17:03 Hi and a few questions Richard Pennington 2012-05-20 17:21 ` Rich Felker 2012-05-20 17:53 ` Szabolcs Nagy @ 2012-05-20 20:28 ` Szabolcs Nagy 2012-05-20 20:36 ` Richard Pennington 2 siblings, 1 reply; 12+ messages in thread From: Szabolcs Nagy @ 2012-05-20 20:28 UTC (permalink / raw) To: musl * Richard Pennington <rich@pennware.com> [2012-05-20 12:03:20 -0500]: > I'm putting together a set of tools and libraries for doing embedded C and C++ > development using clang/LLVM. > > The web site describing the project is http://ellcc.org > btw have you tried musl with c++? i wonder how well that works iirc libc++ had some issues with musl ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 20:28 ` Szabolcs Nagy @ 2012-05-20 20:36 ` Richard Pennington 2012-05-21 1:54 ` Isaac Dunham 0 siblings, 1 reply; 12+ messages in thread From: Richard Pennington @ 2012-05-20 20:36 UTC (permalink / raw) To: musl On Sunday, May 20, 2012 10:28:59 PM Szabolcs Nagy wrote: > * Richard Pennington <rich@pennware.com> [2012-05-20 12:03:20 -0500]: > > I'm putting together a set of tools and libraries for doing embedded C and > > C++ development using clang/LLVM. > > > > The web site describing the project is http://ellcc.org > > btw have you tried musl with c++? > > i wonder how well that works > iirc libc++ had some issues with musl I haven't gotten that far yet. Thanks for the warning. -Rich ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Hi and a few questions 2012-05-20 20:36 ` Richard Pennington @ 2012-05-21 1:54 ` Isaac Dunham 0 siblings, 0 replies; 12+ messages in thread From: Isaac Dunham @ 2012-05-21 1:54 UTC (permalink / raw) To: musl On Sun, 20 May 2012 15:36 -0500 Richard Pennington <rich@pennware.com> wrote: > On Sunday, May 20, 2012 10:28:59 PM Szabolcs Nagy wrote: > > * Richard Pennington <rich@pennware.com> [2012-05-20 12:03:20 > > -0500]: > > > I'm putting together a set of tools and libraries for doing > > > embedded C and C++ development using clang/LLVM. > > > > btw have you tried musl with c++? > > > > i wonder how well that works > > iirc libc++ had some issues with musl > > I haven't gotten that far yet. Thanks for the warning. AFAIK they were addressed last month. (There are probably some residual issues though...) Someone had a hacked clang + musl working several months ago, also. ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-05-21 1:54 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-05-20 17:03 Hi and a few questions Richard Pennington 2012-05-20 17:21 ` Rich Felker 2012-05-20 19:57 ` Richard Pennington 2012-05-20 20:49 ` Isaac Dunham 2012-05-20 21:25 ` Rich Felker 2012-05-20 17:53 ` Szabolcs Nagy 2012-05-20 20:01 ` Richard Pennington 2012-05-20 20:44 ` Richard Pennington 2012-05-20 21:20 ` Rich Felker 2012-05-20 20:28 ` Szabolcs Nagy 2012-05-20 20:36 ` Richard Pennington 2012-05-21 1:54 ` Isaac Dunham
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).