Hello

Hello

[orc]

Hi all.
Finally frustrated with gnulibc, I found your project.
I want to thank you for developing this. It's really a good and simple
replacement for this pile of crap that is called 'glibc' today. Of
course there still issues and missing features, but I am impressed. I
have already a custom root musl-enabled cross-compiled image that is
only 40M long on i686 (with gcc+binutils+busybox+flex/bison/m4/bash)
and it can produce itself natively without problems, can build a
working kernel. It can build g++! (but painfully slow in QEMU)
So I get a system, but not without any issues of course. I can report
issues about building various packages, and submit patches if needed. I
have a set of patches for various packages (most of them are about
missing/invalid #includes).
I'm not a developer, sorry (I have a basic-to-middle C knowledge
enough to understand why build fails), but still can help in areas of
application compatibility, testing and cross-compiling. Anyone
interested?

Re: Hello

[John Spencer]

On 06/07/2012 02:01 PM, orc wrote:
> I'm not a developer, sorry (I have a basic-to-middle C knowledge
> enough to understand why build fails), but still can help in areas of
> application compatibility, testing and cross-compiling. Anyone
> interested?
>
hello,
as described on the webpage, the most helpful work you can do 
additionally to what you suggested is
- spread the word
- file bug reports against upstream software bugtrackers/maillists (such 
as the missing includes you encountered)
- convince people not to use gnulib / gnu autocrap / cmake
- you could help sabotage linux to get a proper X11 setup so that one 
could start to use musl on the desktop
see https://github.com/rofl0r/sabotage
if you want to look into that, i think its most helpful to look at the 
buildscripts alpine linux uses.
-- JS

Re: Hello

[orc]

On Thu, 07 Jun 2012 15:13:38 +0200
John Spencer <maillist-musl@barfooze.de> wrote:

> On 06/07/2012 02:01 PM, orc wrote:
> > I'm not a developer, sorry (I have a basic-to-middle C knowledge
> > enough to understand why build fails), but still can help in areas
> > of application compatibility, testing and cross-compiling. Anyone
> > interested?
> >
> hello,
> as described on the webpage, the most helpful work you can do 
> additionally to what you suggested is
> - spread the word
> - file bug reports against upstream software bugtrackers/maillists
> (such as the missing includes you encountered)
> - convince people not to use gnulib / gnu autocrap / cmake
> - you could help sabotage linux to get a proper X11 setup so that one 
> could start to use musl on the desktop
> see https://github.com/rofl0r/sabotage
> if you want to look into that, i think its most helpful to look at
> the buildscripts alpine linux uses.
> -- JS

Okay, thanks for pointing the direction.
I already have X11 stuff in my TODO list for the musl-enabled system,
but now I feel it will just fail without any results with huge error
logs.
And I already see the X11 stuff in the sabotage tree. Does it builds
correctly?
At this time I think to take an older X release without some stuff I
don't need. X11R7.6 build requires python and weird xml libs for
example. Maybe even XFree86 (to test it in qemu for the micro desktop
system project).

Re: Hello

[Rich Felker]

On Thu, Jun 07, 2012 at 06:29:13PM +0200, John Spencer wrote:
> >At this time I think to take an older X release without some stuff I
> >don't need. X11R7.6 build requires python and weird xml libs for
> >example. Maybe even XFree86 (to test it in qemu for the micro desktop
> >system project).
> >
> what kept me from configuring X11 to work properly is its huge
> freaking complexity.
> if an older version is simpler and can do the job as well, i'm all for it.
> i fear though that you won't get any up-to-date software compiled
> against it.
> probably not worth the effort.

As far as the server goes, even the oldest X server versions should
work fine with modern software (aside from possibly being really slow
when the software wants to do 3D, etc.). That's how X was designed.

With that said, they're also hell in terms of wanting to do low-level
hardware access themselves, not playing nice with the kernel, and not
supporting any modern hardware except maybe in unaccelerated vga,
vesa, or plain fbdev mode. And they're probably full of bad
nonportable legacy unix code and #ifdef hell..

Rich

Re: Hello

[John Spencer]

On 06/07/2012 05:18 PM, orc wrote:
>
> Okay, thanks for pointing the direction.
> I already have X11 stuff in my TODO list for the musl-enabled system,
> but now I feel it will just fail without any results with huge error
> logs.
> And I already see the X11 stuff in the sabotage tree. Does it builds
> correctly?

yes, it builds and works. you have to enable a vga driver mode at the 
kernel prompt though.
you'll get a completely barebone X desktop with nothing else on it as 
the ugly xclock and 2 xterms (after ca 20 seconds of pointless waiting...)
not something anybody except nortti wants to use...

> At this time I think to take an older X release without some stuff I
> don't need. X11R7.6 build requires python and weird xml libs for
> example. Maybe even XFree86 (to test it in qemu for the micro desktop
> system project).
>
what kept me from configuring X11 to work properly is its huge freaking 
complexity.
if an older version is simpler and can do the job as well, i'm all for it.
i fear though that you won't get any up-to-date software compiled 
against it.
probably not worth the effort.

--JS

Re: Hello

[orc]

On Thu, 7 Jun 2012 12:19:28 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> As far as the server goes, even the oldest X server versions should
> work fine with modern software (aside from possibly being really slow
> when the software wants to do 3D, etc.). That's how X was designed.
> 
Of course we need modern X11libs for that. But yes, server expects only
Xau, Xdmcp, crypto for ssl (modern X links to stuff like udev, pixman
and others, but this can be avoided).
When building and running X I got only one thing: for 3D, games and so
on I need a dedicated machine with videoboard vendor's drivers (no
matter opensource it or proprietary). For anything else, I would use
vesa. So there is a question: will musl support this configs? Or I will
need patchelf and 'libc6-legacy' for them?

Re: Hello

[Jens Staal]

[-- Attachment #1: Type: text/plain, Size: 409 bytes --]

> And I already see the X11 stuff in the sabotage tree. Does it builds
> correctly?
> At this time I think to take an older X release without some stuff I
> don't need. X11R7.6 build requires python and weird xml libs for
> example. Maybe even XFree86 (to test it in qemu for the micro desktop
> system project).

What about the "tinyX" from tiny core linux? No idea if this could be built
statically even...

[-- Attachment #2: Type: text/html, Size: 479 bytes --]

Re: Hello

[Christian Neukirchen]

Rich Felker <dalias@aerifal.cx> writes:

> On Thu, Jun 07, 2012 at 06:29:13PM +0200, John Spencer wrote:
>> >At this time I think to take an older X release without some stuff I
>> >don't need. X11R7.6 build requires python and weird xml libs for
>> >example. Maybe even XFree86 (to test it in qemu for the micro desktop
>> >system project).
>> >
>> what kept me from configuring X11 to work properly is its huge
>> freaking complexity.
>> if an older version is simpler and can do the job as well, i'm all for it.
>> i fear though that you won't get any up-to-date software compiled
>> against it.
>> probably not worth the effort.
>
> As far as the server goes, even the oldest X server versions should
> work fine with modern software (aside from possibly being really slow
> when the software wants to do 3D, etc.). That's how X was designed.

Reality is, applications simply won't work without XRENDER, XKB and other
2000'ish things.  E.g. GTK generally needs 8bpp or more, or it will not
even start.

-- 
Christian Neukirchen  <chneukirchen@gmail.com>  http://chneukirchen.org

Re: Hello

[orc]

On Thu, 7 Jun 2012 19:33:24 +0200
Jens Staal <staal1978@gmail.com> wrote:

> > And I already see the X11 stuff in the sabotage tree. Does it builds
> > correctly?
> > At this time I think to take an older X release without some stuff I
> > don't need. X11R7.6 build requires python and weird xml libs for
> > example. Maybe even XFree86 (to test it in qemu for the micro
> > desktop system project).
> 
> What about the "tinyX" from tiny core linux? No idea if this could be
> built statically even...

Yes, I'll try that too.

Re: Hello

[Jens Staal]

2012/6/7 John Spencer <maillist-musl@barfooze.de>:
> On 06/07/2012 05:18 PM, orc wrote:
>>
> what kept me from configuring X11 to work properly is its huge freaking
> complexity.
> if an older version is simpler and can do the job as well, i'm all for it.
> i fear though that you won't get any up-to-date software compiled against
> it.
> probably not worth the effort.
>
> --JS

What about Tiny Core Linux's "tinyX", and perhaps one could even
compile that statically? I have no idea how compatible that is to
Xorg's X11 though... Another appealing alternative could be if there
is a small X11 layer to run on top of Wayland, if that would be easier
to build/configure.

Re: Hello

[John Spencer]

On 06/07/2012 08:03 PM, Jens Staal wrote:
> 2012/6/7 John Spencer<maillist-musl@barfooze.de>:
>> On 06/07/2012 05:18 PM, orc wrote:
>> what kept me from configuring X11 to work properly is its huge freaking
>> complexity.
>> if an older version is simpler and can do the job as well, i'm all for it.
>> i fear though that you won't get any up-to-date software compiled against
>> it.
>> probably not worth the effort.
>>
>> --JS
> What about Tiny Core Linux's "tinyX", and perhaps one could even
> compile that statically? I have no idea how compatible that is to
> Xorg's X11 though... Another appealing alternative could be if there
> is a small X11 layer to run on top of Wayland, if that would be easier
> to build/configure.
>

that's what sabotage currently uses

http://www.x-oz.com/Xfbdev.1.html

"Xfbdev - Linux framebuffer device tiny X server"

Re: Hello

[Rich Felker]

On Fri, Jun 08, 2012 at 01:15:16AM +0800, orc wrote:
> On Thu, 7 Jun 2012 12:19:28 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > As far as the server goes, even the oldest X server versions should
> > work fine with modern software (aside from possibly being really slow
> > when the software wants to do 3D, etc.). That's how X was designed.
> > 
> Of course we need modern X11libs for that. But yes, server expects only
> Xau, Xdmcp, crypto for ssl (modern X links to stuff like udev, pixman
> and others, but this can be avoided).
> When building and running X I got only one thing: for 3D, games and so
> on I need a dedicated machine with videoboard vendor's drivers (no
> matter opensource it or proprietary). For anything else, I would use
> vesa.
Unless reading video memory has gotten substantially faster (i.e. the
same speed as main memory) I don't think vesa suffices for ANY use.
The way the X server is written, almost all common operations require
reading back from video memory if implemented in software. This could
be avoided by changing the implementation to do everything on a
mirrored buffer in main memory and only copy the final version to
video memory, but as far as I know that was never done.

If I'm wrong and video memory access issues have changed all this much
in the past 4 or 5 years, please ignore this message...

> So there is a question: will musl support this configs? Or I will
> need patchelf and 'libc6-legacy' for them?

It's intended to work, but I don't know whether it does yet.

Rich

Re: Hello

[Isaac Dunham]

On Thu, 7 Jun 2012 23:31:41 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Jun 08, 2012 at 01:15:16AM +0800, orc wrote:
> > On Thu, 7 Jun 2012 12:19:28 -0400
> > Rich Felker <dalias@aerifal.cx> wrote:

> > When building and running X I got only one thing: for 3D, games and
> > so on I need a dedicated machine with videoboard vendor's drivers
> > (no matter opensource it or proprietary). For anything else, I
> > would use vesa.
> Unless reading video memory has gotten substantially faster (i.e. the
> same speed as main memory) I don't think vesa suffices for ANY use.
> The way the X server is written, almost all common operations require
> reading back from video memory if implemented in software. This could
> be avoided by changing the implementation to do everything on a
> mirrored buffer in main memory and only copy the final version to
> video memory, but as far as I know that was never done.
Works/usable on a 2000 Dell Dimension/PIII@800MHz/128 to 256 MB RAM/i810
graphics running NetBSD 5.1.
(with an i810, there are no usable drivers at present...though kdrive
i810 might work if I can dig it up...)

However, I can't compile vesa drivers with musl.
(vesa needs sys/vm86.h to use the BIOS drivers, and sys/io.h for
hardware)
Also, uvesafb (the universal framebuffer driver) may not work with
musl: it relies on a userspace daemon, which uses vm86 or libx86 (which
in turn wants vm86, though it can emulate it...)

Isaac Dunham

Re: Hello

[aep]

> Unless reading video memory has gotten substantially faster

Not in hardware, but the new DRI interface in the kernel made graphics 
in linux much much less shitty.

> The way the X server is written, almost all common operations require
> reading back from video memory if implemented in software

This is because X is a network protocol, not a graphics stack.
The fix is to emulate X11 on top of a real graphics architecture, 
namely DRI.
The only implementation i know is wayland.

Either way, the 1960 way of accessing GPU commandbuffers in userspace 
is a dead end.

> By changing the implementation to do everything on a
> mirrored buffer in main memory

Correct. Afaik this is what XComposite in vesa already does, and 
llvmpipe can operate very effiently on DRM shares in main memory.
You can even play quake3 on an arm cortex A9 with llvmpipe, it's right 
on my desk.
But those things are 2012 technology, and they come at a price of a 
pile of dirt below them, with horribly broken build systems and 
misinterpreted "portability" issues.
Getting a real GFX stack on top of any musl-only distro is going to be 
a long walk.


On Thu, 7 Jun 2012 23:31:41 -0400, Rich Felker wrote:
> On Fri, Jun 08, 2012 at 01:15:16AM +0800, orc wrote:
>> On Thu, 7 Jun 2012 12:19:28 -0400
>> Rich Felker <dalias@aerifal.cx> wrote:
>>
>> > As far as the server goes, even the oldest X server versions 
>> should
>> > work fine with modern software (aside from possibly being really 
>> slow
>> > when the software wants to do 3D, etc.). That's how X was 
>> designed.
>> >
>> Of course we need modern X11libs for that. But yes, server expects 
>> only
>> Xau, Xdmcp, crypto for ssl (modern X links to stuff like udev, 
>> pixman
>> and others, but this can be avoided).
>> When building and running X I got only one thing: for 3D, games and 
>> so
>> on I need a dedicated machine with videoboard vendor's drivers (no
>> matter opensource it or proprietary). For anything else, I would use
>> vesa.
> Unless reading video memory has gotten substantially faster (i.e. the
> same speed as main memory) I don't think vesa suffices for ANY use.
> The way the X server is written, almost all common operations require
> reading back from video memory if implemented in software. This could
> be avoided by changing the implementation to do everything on a
> mirrored buffer in main memory and only copy the final version to
> video memory, but as far as I know that was never done.
>
> If I'm wrong and video memory access issues have changed all this 
> much
> in the past 4 or 5 years, please ignore this message...
>
>> So there is a question: will musl support this configs? Or I will
>> need patchelf and 'libc6-legacy' for them?
>
> It's intended to work, but I don't know whether it does yet.
>
> Rich

Re: Hello

[Rich Felker]

On Fri, Jun 08, 2012 at 02:28:29PM +0200, aep wrote:
> >Unless reading video memory has gotten substantially faster
> 
> Not in hardware, but the new DRI interface in the kernel made
> graphics in linux much much less shitty.

Actually, it might have. Especially with integrated Intel graphics,
if I'm not mistaken there is no "video memory"; the graphics
controller just uses part of main memory for video purposes, right?

Rich

Re: Hello

[Rich Felker]

On Fri, Jun 08, 2012 at 06:17:04PM +0200, aep wrote:
> >Actually, it might have. Especially with integrated Intel graphics,
> >if I'm not mistaken there is no "video memory"; the graphics
> >controller just uses part of main memory for video purposes, right?
> 
> sandy bridge and stuff? I am slighly behind current x86 tech, so i
> don't know, but it's just an implementation detail anyway.
> The hardware part of copying CPU to GPU memory is barely relevant
> anymore.
> GFX nowadays is all about shoving the whole program into the gpu, so
> the cpu can do other things, i.e. GLSL.

The whole conversation was about purely unaccelerated graphics, where
the X server is (or at least historicaly was) constantly memcpy'ing
huge amounts of data out of video memory (e.g. for dragging a window).
GPU is irrelevant to this usage.

Rich

Re: Hello

[aep]

> Actually, it might have. Especially with integrated Intel graphics,
> if I'm not mistaken there is no "video memory"; the graphics
> controller just uses part of main memory for video purposes, right?

sandy bridge and stuff? I am slighly behind current x86 tech, so i 
don't know, but it's just an implementation detail anyway.
The hardware part of copying CPU to GPU memory is barely relevant 
anymore.
GFX nowadays is all about shoving the whole program into the gpu, so 
the cpu can do other things, i.e. GLSL.
In 1980 you had to load stuff back and forth to manipulate pixels, 
nowadays you just tell the gpu to manipulate the pixels in a bulk 
without ever loading the frame in main memory.
Xorg is quite backwards in that perpective, as it is designed for 
drawing UI's on any arbitrary device. You probably could draw on a 
printer.
Fortunately we have DRI now which allows us to just bypass all of this 
mess and go to the gpu directly.

Re: Hello

[Isaac Dunham]

On Fri, 08 Jun 2012 18:17:04 +0200
aep <aep@exys.org> wrote:

> > Actually, it might have. Especially with integrated Intel graphics,
> > if I'm not mistaken there is no "video memory"; the graphics
> > controller just uses part of main memory for video purposes, right?
> sandy bridge and stuff? I am slighly behind current x86 tech,
ISTR even the i8xx did that, and let you change the amount allocated.
..
> Xorg is quite backwards in that perpective, as it is designed for 
> drawing UI's on any arbitrary device. You probably could draw on a 
> printer.
*cough* Xprint did that...

Isaac Dunham

Re: Hello

[orc]

On Thu, 7 Jun 2012 23:18:31 +0800
orc <orc@sibserver.ru> wrote:

> On Thu, 07 Jun 2012 15:13:38 +0200
> John Spencer <maillist-musl@barfooze.de> wrote:
> 
> > On 06/07/2012 02:01 PM, orc wrote:
> > > I'm not a developer, sorry (I have a basic-to-middle C knowledge
> > > enough to understand why build fails), but still can help in areas
> > > of application compatibility, testing and cross-compiling. Anyone
> > > interested?
> > >
> > hello,
> > as described on the webpage, the most helpful work you can do 
> > additionally to what you suggested is
> > - spread the word
> > - file bug reports against upstream software bugtrackers/maillists
> > (such as the missing includes you encountered)
> > - convince people not to use gnulib / gnu autocrap / cmake
> > - you could help sabotage linux to get a proper X11 setup so that
> > one could start to use musl on the desktop
> > see https://github.com/rofl0r/sabotage
> > if you want to look into that, i think its most helpful to look at
> > the buildscripts alpine linux uses.
> > -- JS
> 
> Okay, thanks for pointing the direction.
> I already have X11 stuff in my TODO list for the musl-enabled system,
> but now I feel it will just fail without any results with huge error
> logs.
> And I already see the X11 stuff in the sabotage tree. Does it builds
> correctly?
> At this time I think to take an older X release without some stuff I
> don't need. X11R7.6 build requires python and weird xml libs for
> example. Maybe even XFree86 (to test it in qemu for the micro desktop
> system project).

Some news from that point.

I spent some time building the same X11 tree as my host system uses
(X11R76) and now I can say that it works almost unmodified with musl
0.9.1. Some notes about it:
- all libs compiled normally except xcb and Mesa. XCB deals with XML
  and python that I did not installed, Mesa depends on g++.
  (Unfortunately all X libs have now rpath hardcoded, thanks to
  libtool's idiotic behavior. Oh.)
- apps compiled normally (some failed due to unset CFLAGS, was too lazy
  to fix the build.sh)
- xorg-server-1.11.2:
    - did not linked with musl 0.9.1 (missing ioperm() and iopl()
      syscall wrappers, added manually).
    - It doesn't wants to build DRI extensions just because dri.pc
      does not exist (silly).
    - Needs to fix certain glibc-only assumptions, like selecting
      fgetln() instead of getline() and fixing nonexistent __uid_t
      and __gid_t defines.

Finally, Xfbdev works, Xorg server (vesa) works (but needs some help
with module loading in xorg.conf). There was minor bug in libpciaccess,
it used sscanf() with format string that relied on glibc-specific
behavior.
My goal was not to run Xfbdev server, but (unmodified) original Xorg
server, for use on desktop systems. Now I can confirm that this is
possible with musl.

Requirements:
All the X11 stuff from bloated X11R76 release requires only pkg-config,
fontconfig and it's dependencies, libpng, pixman and intltool (that
required perl for XML-Parser and gettext).
Resulting X11 tree is about 200M in size (after 'strip -pg'), huge as a
whole my test rootfs.

I'll test the rootfs on some real x86_64 machine later, to see if Xorg
server is slow due to missing features or it is just QEMU/kvm issue.
After that I will proceed to see how some gui libraries and apps are
compatible with musl :-)

(If anyone already has X11 with musl, ok, just ignore this message)

Re: Hello

[idunham]

> On Thu, 7 Jun 2012 23:18:31 +0800
> orc <orc@sibserver.ru> wrote:
>> Okay, thanks for pointing the direction.
>> I already have X11 stuff in my TODO list for the musl-enabled system,
>> but now I feel it will just fail without any results with huge error
>> logs.
>> And I already see the X11 stuff in the sabotage tree. Does it builds
>> correctly?
>> At this time I think to take an older X release without some stuff I
>> don't need. X11R7.6 build requires python and weird xml libs for
>> example. Maybe even XFree86 (to test it in qemu for the micro desktop
>> system project).
>
> Some news from that point.
>
> I spent some time building the same X11 tree as my host system uses
> (X11R76) and now I can say that it works almost unmodified with musl
> 0.9.1. Some notes about it:
> - all libs compiled normally except xcb and Mesa. XCB deals with XML
>   and python that I did not installed, Mesa depends on g++.
>   (Unfortunately all X libs have now rpath hardcoded, thanks to
>   libtool's idiotic behavior. Oh.)
> - apps compiled normally (some failed due to unset CFLAGS, was too lazy
>   to fix the build.sh)
> - xorg-server-1.11.2:
>     - did not linked with musl 0.9.1 (missing ioperm() and iopl()
>       syscall wrappers, added manually).
Are these wrappers available somewhere online (in a git tree or something)?
>     - It doesn't wants to build DRI extensions just because dri.pc
>       does not exist (silly).
DRI is for HW acceleration, and is provided by Mesa.
>     - Needs to fix certain glibc-only assumptions, like selecting
>       fgetln() instead of getline() and fixing nonexistent __uid_t
>       and __gid_t defines.
I've been working (occasionally) on an "uncdef" script to fix this sort of
problem. What I'd like to do longterm is provide something that can
automatically generate a patch (copy the tree, change, save diff).
> Finally, Xfbdev works, Xorg server (vesa) works (but needs some help
> with module loading in xorg.conf). There was minor bug in libpciaccess,
> it used sscanf() with format string that relied on glibc-specific
> behavior.
> My goal was not to run Xfbdev server, but (unmodified) original Xorg
> server, for use on desktop systems. Now I can confirm that this is
> possible with musl.
>
> Requirements:
> All the X11 stuff from bloated X11R76 release requires only pkg-config,
> fontconfig and it's dependencies, libpng, pixman and intltool (that
> required perl for XML-Parser and gettext).
> Resulting X11 tree is about 200M in size (after 'strip -pg'), huge as a
> whole my test rootfs.
>
> I'll test the rootfs on some real x86_64 machine later, to see if Xorg
> server is slow due to missing features or it is just QEMU/kvm issue.
Anything X is slow wth Qemu, between the graphics layer and the
emulation--I've seen 20-second lag times with twm on netbsd (on kvm, AMD
Neo @1.6GHz host), while a PIII @800 MHz is reasonably usable. Try using
the VNC interface, it's much faster (but still slow).
> After that I will proceed to see how some gui libraries and apps are
> compatible with musl :-)
Snowflake Linux has GTK2 + Xfce, I've heard of GTK1.2, I know meh, jwm,
mupdf, mtpaint, etc. build (not sure about patches, just saw sizes
listed).
> (If anyone already has X11 with musl, ok, just ignore this message)
Only Xfbdev/Xnest yet AFAICT.
___________________________________________________________________________

Re: Hello

[Rich Felker]

On Fri, Jun 22, 2012 at 09:43:13PM -0400, idunham@lavabit.com wrote:
> DRI is for HW acceleration, and is provided by Mesa.
> >     - Needs to fix certain glibc-only assumptions, like selecting
> >       fgetln() instead of getline() and fixing nonexistent __uid_t

Hmm, I've been thinking about adding fgetln. It's a really nice BSD
interface that's something like a more-efficient version of getline
(it can return a pointer into the buffer when possible) and the junk I
just added to make gnulib happy (which is slightly more powerful but
more of a pain for apps to use). I'm surprised glibc has it though; I
thought it was really BSD-only.

> >       and __gid_t defines.
> I've been working (occasionally) on an "uncdef" script to fix this sort of
> problem. What I'd like to do longterm is provide something that can
> automatically generate a patch (copy the tree, change, save diff).

What about using the existing git repo if it's there, and if not,
running git init and and initial checkin then git diff? That seems
easier than scripting your own copy of the whole tree to make diffs,
but maybe it's too heavy-weight for your taste.

Rich

Re: Hello

[orc]

[-- Attachment #1: Type: text/plain, Size: 1943 bytes --]

On Fri, 22 Jun 2012 21:43:13 -0400 (EDT)
idunham@lavabit.com wrote:

> > On Thu, 7 Jun 2012 23:18:31 +0800
> > orc <orc@sibserver.ru> wrote:
> >> Okay, thanks for pointing the direction.
> >> I already have X11 stuff in my TODO list for the musl-enabled
> >> system, but now I feel it will just fail without any results with
> >> huge error logs.
> >> And I already see the X11 stuff in the sabotage tree. Does it
> >> builds correctly?
> >> At this time I think to take an older X release without some stuff
> >> I don't need. X11R7.6 build requires python and weird xml libs for
> >> example. Maybe even XFree86 (to test it in qemu for the micro
> >> desktop system project).
> >
> > Some news from that point.
> >
> > I spent some time building the same X11 tree as my host system uses
> > (X11R76) and now I can say that it works almost unmodified with musl
> > 0.9.1. Some notes about it:
> > - all libs compiled normally except xcb and Mesa. XCB deals with XML
> >   and python that I did not installed, Mesa depends on g++.
> >   (Unfortunately all X libs have now rpath hardcoded, thanks to
> >   libtool's idiotic behavior. Oh.)
> > - apps compiled normally (some failed due to unset CFLAGS, was too
> > lazy to fix the build.sh)
> > - xorg-server-1.11.2:
> >     - did not linked with musl 0.9.1 (missing ioperm() and iopl()
> >       syscall wrappers, added manually).
> Are these wrappers available somewhere online (in a git tree or
> something)?

I added them manually, they are just one-line system call wrappers.
Attached patch just adds them as an additional files, without declaring
them in headers (I declared them in unistd.h, but not sure, maybe
wrong place for linux-specific calls).

> Resulting X11 tree is about 200M in size (after 'strip -pg'), huge as
> a whole my test rootfs.

There is my mistake, not 200M but about 106M. I did wrong computation
(actually just did 'df -h' on mount point instead of 'du -xsh /X')

[-- Attachment #2: musl-0.9.2-ioperm.patch --]
[-- Type: application/octet-stream, Size: 728 bytes --]

diff -Naur musl-0.9.2.o/src/linux/ioperm.c musl-0.9.2/src/linux/ioperm.c
--- musl-0.9.2.o/src/linux/ioperm.c	1970-01-01 00:00:00.000000000 +0000
+++ musl-0.9.2/src/linux/ioperm.c	2012-06-25 11:56:40.305613400 +0000
@@ -0,0 +1,7 @@
+#include <unistd.h>
+#include "syscall.h"
+
+int ioperm(unsigned long from, unsigned long num, int turn_on)
+{
+	return syscall(SYS_ioperm, from, num, turn_on);
+}
diff -Naur musl-0.9.2.o/src/linux/iopl.c musl-0.9.2/src/linux/iopl.c
--- musl-0.9.2.o/src/linux/iopl.c	1970-01-01 00:00:00.000000000 +0000
+++ musl-0.9.2/src/linux/iopl.c	2012-06-25 11:56:23.122613400 +0000
@@ -0,0 +1,8 @@
+#include <unistd.h>
+#include "syscall.h"
+
+int iopl(int level)
+{
+	return syscall(SYS_iopl, level);
+}
+

Re: Hello

[orc]

On Fri, 22 Jun 2012 21:51:07 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Jun 22, 2012 at 09:43:13PM -0400, idunham@lavabit.com wrote:
> > DRI is for HW acceleration, and is provided by Mesa.
> > >     - Needs to fix certain glibc-only assumptions, like selecting
> > >       fgetln() instead of getline() and fixing nonexistent __uid_t
> 
> I'm surprised glibc has it though; I thought it was really BSD-only.

glibc has no fgetln(), and Xorg uses getline if building with glibc:

#ifdef __GLIBC__
            while ((read = getline(&line, &len, fp)) != -1) {
#else
            while ((line = fgetln(fp, &len)) != (char *)NULL) {
#endif /* __GLIBC __ */

(in xorg-server-1.11.2/hw/xfree86/common/xf86pciBus.c)

Re: Hello

[idunham]

> On Fri, 22 Jun 2012 21:43:13 -0400 (EDT)
> idunham@lavabit.com wrote:
>
>> > Some news from that point.
>> >
>> > I spent some time building the same X11 tree as my host system uses
>> > (X11R76) and now I can say that it works almost unmodified with musl
>> > 0.9.1. Some notes about it:
>> > - all libs compiled normally except xcb and Mesa. XCB deals with XML
>> >   and python that I did not installed, Mesa depends on g++.
>> >   (Unfortunately all X libs have now rpath hardcoded, thanks to
>> >   libtool's idiotic behavior. Oh.)
>> > - apps compiled normally (some failed due to unset CFLAGS, was too
>> > lazy to fix the build.sh)
>> > - xorg-server-1.11.2:
>> >     - did not linked with musl 0.9.1 (missing ioperm() and iopl()
>> >       syscall wrappers, added manually).
>> Are these wrappers available somewhere online (in a git tree or
>> something)?
>
> I added them manually, they are just one-line system call wrappers.
> Attached patch just adds them as an additional files, without declaring
> them in headers (I declared them in unistd.h, but not sure, maybe
> wrong place for linux-specific calls).
>
Wrong place: sys/io.h (not yet provided by musl) is where those go. _

Re: Hello

[orc]

On Thu, 7 Jun 2012 23:31:41 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> > So there is a question: will musl support this configs? Or I will
> > need patchelf and 'libc6-legacy' for them?
> 
> It's intended to work, but I don't know whether it does yet.

Did some research there last days: for, example, one that proprietary
drivers that nvidia ships it was required about 30 missing symbols,
half of them are one-liners system calls, 5 were glibc-specific
functions that were easy to add (one of them is gnu_get_libc_version()
that is designed to return a plain string), 4 were missing math
functions that already defined as a macros in math.h, rest is a forest
of weak aliases around already existed functions (plus two aliases to
objects). That allowed me to run plain unmodified X11 applications (not
even gtk2 ones) and accelerated glxgears without errors (The gtk2 or
qt or other such libraries compiled against glibc is not my target,
just to prove that userspace nvidia could be run with musl).
If you interested, I can put a patch that adds such forest of
weak_alias'es to improve (partly) glibc compatibility. And separate
patch for missing syscall wrappers.

> 
> Rich

Re: Hello

[Rich Felker]

On Fri, Jul 06, 2012 at 01:24:17AM +0800, orc wrote:
> On Thu, 7 Jun 2012 23:31:41 -0400
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > > So there is a question: will musl support this configs? Or I will
> > > need patchelf and 'libc6-legacy' for them?
> > 
> > It's intended to work, but I don't know whether it does yet.
> 
> Did some research there last days: for, example, one that proprietary
> drivers that nvidia ships it was required about 30 missing symbols,
> half of them are one-liners system calls, 5 were glibc-specific
> functions that were easy to add (one of them is gnu_get_libc_version()
> that is designed to return a plain string), 4 were missing math
> functions that already defined as a macros in math.h, rest is a forest
> of weak aliases around already existed functions (plus two aliases to
> objects). That allowed me to run plain unmodified X11 applications (not
> even gtk2 ones) and accelerated glxgears without errors (The gtk2 or
> qt or other such libraries compiled against glibc is not my target,
> just to prove that userspace nvidia could be run with musl).
> If you interested, I can put a patch that adds such forest of
> weak_alias'es to improve (partly) glibc compatibility. And separate
> patch for missing syscall wrappers.

I'm very interested in this. I'm surprised it was that easy to make it
work, and just curious about all the aliases that were involved and
whether they make since or whether they're hacks. Post patches or a
report in whatever form you prefer; I'll review it and hopefully it
can be committed without much additional fuss.

Rich

Re: Hello

[orc]

[-- Attachment #1: Type: text/plain, Size: 3542 bytes --]

On Thu, 5 Jul 2012 19:34:57 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Jul 06, 2012 at 01:24:17AM +0800, orc wrote:
> > On Thu, 7 Jun 2012 23:31:41 -0400
> > Rich Felker <dalias@aerifal.cx> wrote:
> > 
> > > > So there is a question: will musl support this configs? Or I
> > > > will need patchelf and 'libc6-legacy' for them?
> > > 
> > > It's intended to work, but I don't know whether it does yet.
> > 
> > Did some research there last days: for, example, one that
> > proprietary drivers that nvidia ships it was required about 30
> > missing symbols, half of them are one-liners system calls, 5 were
> > glibc-specific functions that were easy to add (one of them is
> > gnu_get_libc_version() that is designed to return a plain string),
> > 4 were missing math functions that already defined as a macros in
> > math.h, rest is a forest of weak aliases around already existed
> > functions (plus two aliases to objects). That allowed me to run
> > plain unmodified X11 applications (not even gtk2 ones) and
> > accelerated glxgears without errors (The gtk2 or qt or other such
> > libraries compiled against glibc is not my target, just to prove
> > that userspace nvidia could be run with musl). If you interested, I
> > can put a patch that adds such forest of weak_alias'es to improve
> > (partly) glibc compatibility. And separate patch for missing
> > syscall wrappers.
> 
> I'm very interested in this. I'm surprised it was that easy to make it
> work, and just curious about all the aliases that were involved and
> whether they make since or whether they're hacks. Post patches or a
> report in whatever form you prefer; I'll review it and hopefully it
> can be committed without much additional fuss.

Here a patch, attached. It contains both missing syscalls and weak
aliases. It does not contain glibc-specific stuff (if you want, I can
send it too, but it looks ugly, intended only for 'run it
successfully'). Some notes about:
- rawmemchr() was taken from uClibc
- ioperm() and iopl() were not necessary to make glxgears work, just
  added them because Xorg will want them
- I don't think libc even needs xattr stuff, but glibc includes them.
  On many systems they are usually duplicated, libattr provides same
  interface
- It seems that every function in src/locale needs it's __underscore
  alias, to match glibc api
- there some ugly __funcname_internal aliases, don't know why glibc
  defines them in that way

Probably you will want to add:
- weak_aliases for __underscores
- weak_aliases __funcname_internal
- rawmemchr() (probably your own implementation, not uClibc one)
- some missing syscalls (I really misguessed number of needed syscalls,
  seems that this was a number of aliases, not syscalls)

glibc-specific functions and objects required to make glxgears work:
- dlvsym() (wrapper around dlsym(), we don't need ugly symvers)
- gnu_get_libc_*()
- malloc_usable_size() (returns 0 always, probably there is no code in
  musl to make it work. Definition in eglibc was cryptic for me, but it
  clearly seems to be the glibc/ptmalloc feature)
- 4 function-wrappers in math code: __isnan(), __isnanf(), __isinf(),
  __isinff()
- __xmknod()
- _IO_2_1_stdout_ -> stdout

gtk2 will not work that way, I checked. One library in chain requires
libstdc++, libstdc++ defines 'unique' symbols (see manual page of
binutils nm) which musl linker cannot handle. Additionally, there is
much more missing symbols including missing functions. But plain X11
apps worked (I checked xfontsel and xlogo).

> 
> Rich


[-- Attachment #2: musl-0.9.2-misc-symbols.patch --]
[-- Type: application/octet-stream, Size: 15832 bytes --]

diff -Naur musl-0.9.2.o/include/fcntl.h musl-0.9.2/include/fcntl.h
--- musl-0.9.2.o/include/fcntl.h	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/include/fcntl.h	2012-07-06 12:49:23.950799602 +0800
@@ -8,6 +8,8 @@
 #define __NEED_off_t
 #define __NEED_pid_t
 #define __NEED_mode_t
+#define __NEED_size_t
+#define __NEED_ssize_t
 
 #include <bits/alltypes.h>
 
@@ -97,6 +99,7 @@
 #define F_TEST  3
 
 int lockf(int, int, off_t);
+ssize_t splice(int, off_t, int, off_t, size_t, unsigned int);
 #endif
 
 #if defined(_GNU_SOURCE)
diff -Naur musl-0.9.2.o/include/string.h musl-0.9.2/include/string.h
--- musl-0.9.2.o/include/string.h	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/include/string.h	2012-07-06 12:50:20.256799602 +0800
@@ -92,6 +92,7 @@
 char *strsep(char **, const char *);
 void *memrchr(const void *, int, size_t);
 void *mempcpy(void *, const void *, size_t);
+void *rawmemchr(const void *, int);
 #ifndef __cplusplus
 char *basename();
 #endif
diff -Naur musl-0.9.2.o/include/sys/io.h musl-0.9.2/include/sys/io.h
--- musl-0.9.2.o/include/sys/io.h	1970-01-01 07:00:00.000000000 +0700
+++ musl-0.9.2/include/sys/io.h	2012-07-06 12:49:23.950799602 +0800
@@ -0,0 +1,2 @@
+int ioperm(unsigned long, unsigned long, int);
+int iopl(int);
diff -Naur musl-0.9.2.o/include/unistd.h musl-0.9.2/include/unistd.h
--- musl-0.9.2.o/include/unistd.h	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/include/unistd.h	2012-07-06 12:49:23.951799602 +0800
@@ -31,6 +31,7 @@
 #include <bits/alltypes.h>
 
 int pipe(int [2]);
+int pipe2(int [2], int);
 int close(int);
 int dup(int);
 int dup2(int, int);
diff -Naur musl-0.9.2.o/src/fcntl/fcntl.c musl-0.9.2/src/fcntl/fcntl.c
--- musl-0.9.2.o/src/fcntl/fcntl.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/fcntl/fcntl.c	2012-07-06 12:49:23.951799602 +0800
@@ -24,3 +24,5 @@
 	}
 	return syscall(SYS_fcntl, fd, cmd, arg);
 }
+
+weak_alias(fcntl, __fcntl);
diff -Naur musl-0.9.2.o/src/linux/ioperm.c musl-0.9.2/src/linux/ioperm.c
--- musl-0.9.2.o/src/linux/ioperm.c	1970-01-01 07:00:00.000000000 +0700
+++ musl-0.9.2/src/linux/ioperm.c	2012-07-06 12:49:23.951799602 +0800
@@ -0,0 +1,6 @@
+#include "syscall.h"
+
+int ioperm(unsigned long from, unsigned long num, int turn_on)
+{
+	return syscall(SYS_ioperm, from, num, turn_on);
+}
diff -Naur musl-0.9.2.o/src/linux/iopl.c musl-0.9.2/src/linux/iopl.c
--- musl-0.9.2.o/src/linux/iopl.c	1970-01-01 07:00:00.000000000 +0700
+++ musl-0.9.2/src/linux/iopl.c	2012-07-06 12:49:23.951799602 +0800
@@ -0,0 +1,6 @@
+#include "syscall.h"
+
+int iopl(int level)
+{
+	return syscall(SYS_iopl, level);
+}
diff -Naur musl-0.9.2.o/src/linux/splice.c musl-0.9.2/src/linux/splice.c
--- musl-0.9.2.o/src/linux/splice.c	1970-01-01 07:00:00.000000000 +0700
+++ musl-0.9.2/src/linux/splice.c	2012-07-06 12:49:23.951799602 +0800
@@ -0,0 +1,7 @@
+#include <sys/types.h>
+#include "syscall.h"
+
+ssize_t splice(int fd_in, off_t *off_in, int fd_out, off_t *off_out, size_t len, unsigned int flags)
+{
+	return syscall(SYS_splice, fd_in, off_in, fd_out, off_out, len, flags);
+}
diff -Naur musl-0.9.2.o/src/linux/xattr.c musl-0.9.2/src/linux/xattr.c
--- musl-0.9.2.o/src/linux/xattr.c	1970-01-01 07:00:00.000000000 +0700
+++ musl-0.9.2/src/linux/xattr.c	2012-07-06 12:49:23.951799602 +0800
@@ -0,0 +1,47 @@
+#include <sys/types.h>
+#include "syscall.h"
+
+ssize_t getxattr(const char *path, const char *name, void *value, size_t size)
+{
+	return syscall(SYS_getxattr, path, name, value, size);
+}
+
+ssize_t lgetxattr(const char *path, const char *name, void *value, size_t size)
+{
+	return syscall(SYS_lgetxattr, path, name, value, size);
+}
+
+ssize_t fgetxattr(int filedes, const char *name, void *value, size_t size)
+{
+	return syscall(SYS_fgetxattr, filedes, name, value, size);
+}
+
+ssize_t listxattr(const char *path, char *list, size_t size)
+{
+	return syscall(SYS_listxattr, path, list, size);
+}
+
+ssize_t llistxattr(const char *path, char *list, size_t size)
+{
+	return syscall(SYS_llistxattr, path, list, size);
+}
+
+ssize_t flistxattr(int filedes, char *list, size_t size)
+{
+	return syscall(SYS_flistxattr, filedes, list, size);
+}
+
+int setxattr(const char *path, const char *name, const void *value, size_t size, int flags)
+{
+	return syscall(SYS_setxattr, path, name, value, size, flags);
+}
+
+int lsetxattr(const char *path, const char *name, const void *value, size_t size, int flags)
+{
+	return syscall(SYS_lsetxattr, path, name, value, size, flags);
+}
+
+int fsetxattr(int filedes, const char *name, const void *value, size_t size, int flags)
+{
+	return syscall(SYS_fsetxattr, filedes, name, value, size, flags);
+}
diff -Naur musl-0.9.2.o/src/locale/duplocale.c musl-0.9.2/src/locale/duplocale.c
--- musl-0.9.2.o/src/locale/duplocale.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/duplocale.c	2012-07-06 12:49:23.951799602 +0800
@@ -1,6 +1,7 @@
 #include <stdlib.h>
 #include <string.h>
 #include "locale_impl.h"
+#include "libc.h"
 
 locale_t duplocale(locale_t old)
 {
@@ -9,3 +10,5 @@
 	if (new && old != LC_GLOBAL_LOCALE) memcpy(new, old, sizeof *new);
 	return new;
 }
+
+weak_alias(duplocale, __duplocale);
diff -Naur musl-0.9.2.o/src/locale/freelocale.c musl-0.9.2/src/locale/freelocale.c
--- musl-0.9.2.o/src/locale/freelocale.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/freelocale.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,7 +1,10 @@
 #include <stdlib.h>
 #include "locale_impl.h"
+#include "libc.h"
 
 void freelocale(locale_t l)
 {
 	free(l);
 }
+
+weak_alias(freelocale, __freelocale);
diff -Naur musl-0.9.2.o/src/locale/iswctype_l.c musl-0.9.2/src/locale/iswctype_l.c
--- musl-0.9.2.o/src/locale/iswctype_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/iswctype_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,6 +1,9 @@
 #include <wctype.h>
+#include "libc.h"
 
 int iswctype_l(wint_t c, wctype_t t, locale_t l)
 {
 	return iswctype(c, t);
 }
+
+weak_alias(iswctype_l, __iswctype_l);
diff -Naur musl-0.9.2.o/src/locale/newlocale.c musl-0.9.2/src/locale/newlocale.c
--- musl-0.9.2.o/src/locale/newlocale.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/newlocale.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,6 +1,7 @@
 #include <stdlib.h>
 #include <string.h>
 #include "locale_impl.h"
+#include "libc.h"
 
 locale_t newlocale(int mask, const char *name, locale_t base)
 {
@@ -9,3 +10,5 @@
 	if (!base) base = calloc(1, sizeof *base);
 	return base;
 }
+
+weak_alias(newlocale, __newlocale);
diff -Naur musl-0.9.2.o/src/locale/nl_langinfo_l.c musl-0.9.2/src/locale/nl_langinfo_l.c
--- musl-0.9.2.o/src/locale/nl_langinfo_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/nl_langinfo_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,7 +1,10 @@
 #include <locale.h>
 #include <langinfo.h>
+#include "libc.h"
 
 char *nl_langinfo_l(nl_item item, locale_t l)
 {
 	return nl_langinfo(item);
 }
+
+weak_alias(nl_langinfo_l, __nl_langinfo_l);
diff -Naur musl-0.9.2.o/src/locale/strcoll_l.c musl-0.9.2/src/locale/strcoll_l.c
--- musl-0.9.2.o/src/locale/strcoll_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/strcoll_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,7 +1,10 @@
 #include <string.h>
 #include <locale.h>
+#include "libc.h"
 
 int strcoll_l(const char *l, const char *r, locale_t loc)
 {
 	return strcoll(l, r);
 }
+
+weak_alias(strcoll_l, __strcoll_l);
diff -Naur musl-0.9.2.o/src/locale/strftime_l.c musl-0.9.2/src/locale/strftime_l.c
--- musl-0.9.2.o/src/locale/strftime_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/strftime_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,7 +1,10 @@
 #include <locale.h>
 #include <time.h>
+#include "libc.h"
 
 size_t strftime_l(char *s, size_t n, const char *f, const struct tm *tm, locale_t l)
 {
 	return strftime(s, n, f, tm);
 }
+
+weak_alias(strftime_l, __strftime_l);
diff -Naur musl-0.9.2.o/src/locale/strxfrm_l.c musl-0.9.2/src/locale/strxfrm_l.c
--- musl-0.9.2.o/src/locale/strxfrm_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/strxfrm_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,6 +1,9 @@
 #include <string.h>
+#include "libc.h"
 
 size_t strxfrm_l(char *dest, const char *src, size_t n, locale_t l)
 {
 	return strxfrm(dest, src, n);
 }
+
+weak_alias(strxfrm_l, __strxfrm_l);
diff -Naur musl-0.9.2.o/src/locale/towlower_l.c musl-0.9.2/src/locale/towlower_l.c
--- musl-0.9.2.o/src/locale/towlower_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/towlower_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,6 +1,9 @@
 #include <wctype.h>
+#include "libc.h"
 
 wint_t towlower_l(wint_t c, locale_t l)
 {
 	return towlower(c);
 }
+
+weak_alias(towlower_l, __towlower_l);
diff -Naur musl-0.9.2.o/src/locale/towupper_l.c musl-0.9.2/src/locale/towupper_l.c
--- musl-0.9.2.o/src/locale/towupper_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/towupper_l.c	2012-07-06 12:49:23.952799602 +0800
@@ -1,6 +1,9 @@
 #include <wctype.h>
+#include "libc.h"
 
 wint_t towupper_l(wint_t c, locale_t l)
 {
 	return towupper(c);
 }
+
+weak_alias(towupper_l, __towupper_l);
diff -Naur musl-0.9.2.o/src/locale/uselocale.c musl-0.9.2/src/locale/uselocale.c
--- musl-0.9.2.o/src/locale/uselocale.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/uselocale.c	2012-07-06 12:49:23.953799602 +0800
@@ -1,5 +1,6 @@
 #include "locale_impl.h"
 #include "pthread_impl.h"
+#include "libc.h"
 
 locale_t uselocale(locale_t l)
 {
@@ -8,3 +9,5 @@
 	if (l) self->locale = l;
 	return old;
 }
+
+weak_alias(uselocale, __uselocale);
diff -Naur musl-0.9.2.o/src/locale/wcscoll_l.c musl-0.9.2/src/locale/wcscoll_l.c
--- musl-0.9.2.o/src/locale/wcscoll_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/wcscoll_l.c	2012-07-06 12:49:23.953799602 +0800
@@ -1,6 +1,9 @@
 #include <wchar.h>
+#include "libc.h"
 
 int wcscoll_l(const wchar_t *l, const wchar_t *r, locale_t locale)
 {
 	return wcscoll(l, r);
 }
+
+weak_alias(wcscoll_l, __wcscoll_l);
diff -Naur musl-0.9.2.o/src/locale/wcsxfrm_l.c musl-0.9.2/src/locale/wcsxfrm_l.c
--- musl-0.9.2.o/src/locale/wcsxfrm_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/wcsxfrm_l.c	2012-07-06 12:49:23.953799602 +0800
@@ -1,6 +1,9 @@
 #include <wchar.h>
+#include "libc.h"
 
 size_t wcsxfrm_l(wchar_t *dest, const wchar_t *src, size_t n, locale_t locale)
 {
 	return wcsxfrm(dest, src, n);
 }
+
+weak_alias(wcsxfrm_l, __wcsxfrm_l);
diff -Naur musl-0.9.2.o/src/locale/wctype_l.c musl-0.9.2/src/locale/wctype_l.c
--- musl-0.9.2.o/src/locale/wctype_l.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/locale/wctype_l.c	2012-07-06 12:49:23.953799602 +0800
@@ -1,6 +1,9 @@
 #include <wctype.h>
+#include "libc.h"
 
 wctype_t wctype_l(const char *s, locale_t l)
 {
 	return wctype(s);
 }
+
+weak_alias(wctype_l, __wctype_l);
diff -Naur musl-0.9.2.o/src/math/__fpclassify.c musl-0.9.2/src/math/__fpclassify.c
--- musl-0.9.2.o/src/math/__fpclassify.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/math/__fpclassify.c	2012-07-06 13:05:11.344799602 +0800
@@ -8,3 +8,10 @@
 	if (e==0x7ff) return u.bits<<12 ? FP_NAN : FP_INFINITE;
 	return FP_NORMAL;
 }
+
+int finite(double x)
+{
+	return isfinite(x);
+}
+
+weak_alias(finite, __finite);
diff -Naur musl-0.9.2.o/src/select/poll.c musl-0.9.2/src/select/poll.c
--- musl-0.9.2.o/src/select/poll.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/select/poll.c	2012-07-06 12:49:23.953799602 +0800
@@ -6,3 +6,5 @@
 {
 	return syscall_cp(SYS_poll, fds, n, timeout);
 }
+
+weak_alias(poll, __poll);
diff -Naur musl-0.9.2.o/src/signal/i386/sigsetjmp.s musl-0.9.2/src/signal/i386/sigsetjmp.s
--- musl-0.9.2.o/src/signal/i386/sigsetjmp.s	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/signal/i386/sigsetjmp.s	2012-07-06 12:49:23.953799602 +0800
@@ -1,6 +1,9 @@
 .global sigsetjmp
 .type sigsetjmp,@function
+.weak __sigsetjmp
+.type __sigsetjmp,@function
 sigsetjmp:
+__sigsetjmp:
 	mov 4(%esp),%eax
 	mov 8(%esp),%ecx
 	mov %ecx,24(%eax)
diff -Naur musl-0.9.2.o/src/signal/x86_64/sigsetjmp.s musl-0.9.2/src/signal/x86_64/sigsetjmp.s
--- musl-0.9.2.o/src/signal/x86_64/sigsetjmp.s	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/signal/x86_64/sigsetjmp.s	2012-07-06 12:49:23.955799602 +0800
@@ -1,7 +1,10 @@
 /* Copyright 2011-2012 Nicholas J. Kain, licensed under standard MIT license */
 .global sigsetjmp
 .type sigsetjmp,@function
+.weak __sigsetjmp
+.type __sigsetjmp,@function
 sigsetjmp:
+__sigsetjmp:
 	andl %esi,%esi
 	movq %rsi,64(%rdi)
 	jz 1f
diff -Naur musl-0.9.2.o/src/stdio/fscanf.c musl-0.9.2/src/stdio/fscanf.c
--- musl-0.9.2.o/src/stdio/fscanf.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/stdio/fscanf.c	2012-07-06 12:49:23.955799602 +0800
@@ -1,5 +1,6 @@
 #include <stdio.h>
 #include <stdarg.h>
+#include "libc.h"
 
 int fscanf(FILE *f, const char *fmt, ...)
 {
@@ -10,3 +11,5 @@
 	va_end(ap);
 	return ret;
 }
+
+weak_alias(fscanf, __isoc99_fscanf);
diff -Naur musl-0.9.2.o/src/stdio/sscanf.c musl-0.9.2/src/stdio/sscanf.c
--- musl-0.9.2.o/src/stdio/sscanf.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/stdio/sscanf.c	2012-07-06 12:49:23.955799602 +0800
@@ -1,5 +1,6 @@
 #include <stdio.h>
 #include <stdarg.h>
+#include "libc.h"
 
 int sscanf(const char *s, const char *fmt, ...)
 {
@@ -10,3 +11,5 @@
 	va_end(ap);
 	return ret;
 }
+
+weak_alias(sscanf, __isoc99_sscanf);
diff -Naur musl-0.9.2.o/src/stdlib/strtod.c musl-0.9.2/src/stdlib/strtod.c
--- musl-0.9.2.o/src/stdlib/strtod.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/stdlib/strtod.c	2012-07-06 12:49:23.955799602 +0800
@@ -2,6 +2,7 @@
 #include "shgetc.h"
 #include "floatscan.h"
 #include "stdio_impl.h"
+#include "libc.h"
 
 static long double strtox(const char *s, char **p, int prec)
 {
@@ -30,3 +31,5 @@
 {
 	return strtox(s, p, 2);
 }
+
+weak_alias(strtod, __strtod_internal);
diff -Naur musl-0.9.2.o/src/stdlib/strtol.c musl-0.9.2/src/stdlib/strtol.c
--- musl-0.9.2.o/src/stdlib/strtol.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/stdlib/strtol.c	2012-07-06 12:49:23.955799602 +0800
@@ -1,6 +1,7 @@
 #include "stdio_impl.h"
 #include "intscan.h"
 #include "shgetc.h"
+#include "libc.h"
 
 static unsigned long long strtox(const char *s, char **p, int base, unsigned long long lim)
 {
@@ -51,3 +52,8 @@
 {
 	return strtoull(s, p, base);
 }
+
+weak_alias(strtoll, __strtoll_internal);
+weak_alias(strtol, __strtol_internal);
+weak_alias(strtoul, __strtoul_internal);
+weak_alias(strtoull, __strtoull_internal);
diff -Naur musl-0.9.2.o/src/string/rawmemchr.c musl-0.9.2/src/string/rawmemchr.c
--- musl-0.9.2.o/src/string/rawmemchr.c	1970-01-01 07:00:00.000000000 +0700
+++ musl-0.9.2/src/string/rawmemchr.c	2012-07-06 12:49:23.956799602 +0800
@@ -0,0 +1,14 @@
+// Taken from uClibc libc/string/rawmemchr.c
+#include <string.h>
+#include "libc.h"
+
+void *rawmemchr(const void *s, int c)
+{
+	register const unsigned char *r = s;
+
+	while (*r != ((unsigned char)c)) ++r;
+
+	return (void *) r;	/* silence the warning */
+}
+
+weak_alias(rawmemchr, __rawmemchr);
diff -Naur musl-0.9.2.o/src/string/strndup.c musl-0.9.2/src/string/strndup.c
--- musl-0.9.2.o/src/string/strndup.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/string/strndup.c	2012-07-06 12:49:23.956799602 +0800
@@ -1,5 +1,6 @@
 #include <stdlib.h>
 #include <string.h>
+#include "libc.h"
 
 char *strndup(const char *s, size_t n)
 {
@@ -10,3 +11,5 @@
 	d[l] = 0;
 	return d;
 }
+
+weak_alias(strndup, __strndup);
diff -Naur musl-0.9.2.o/src/unistd/pipe.c musl-0.9.2/src/unistd/pipe.c
--- musl-0.9.2.o/src/unistd/pipe.c	2012-06-25 09:02:55.000000000 +0800
+++ musl-0.9.2/src/unistd/pipe.c	2012-07-06 12:49:23.956799602 +0800
@@ -5,3 +5,8 @@
 {
 	return syscall(SYS_pipe, fd);
 }
+
+int pipe2(int fd[2], int flg)
+{
+	return syscall(SYS_pipe, fd, flg);
+}

Re: Hello

[Rich Felker]

On Fri, Jul 06, 2012 at 02:06:01PM +0800, orc wrote:
> Here a patch, attached. It contains both missing syscalls and weak
> aliases. It does not contain glibc-specific stuff (if you want, I can
> send it too, but it looks ugly, intended only for 'run it
> successfully'). Some notes about:
> - rawmemchr() was taken from uClibc
> - ioperm() and iopl() were not necessary to make glxgears work, just
>   added them because Xorg will want them
> - I don't think libc even needs xattr stuff, but glibc includes them.
>   On many systems they are usually duplicated, libattr provides same
>   interface
> - It seems that every function in src/locale needs it's __underscore
>   alias, to match glibc api
> - there some ugly __funcname_internal aliases, don't know why glibc
>   defines them in that way

I think most of this is an artifact of some hacks they do in the glibc
headers for inlining/optimization and/or _FORTIFY_SOURCE support.
The functions it creates references to are not (AFAIK) in the LSB ABI
and thus binaries using them are non-LSB-conforming...

> Probably you will want to add:
> - weak_aliases for __underscores

Except most of them should be in the opposite direction. Especially
for functions like strxfrm_l where we'll eventually want the ISO C
"foo" function to depend on the POSIX "foo_l" function, the latter
will need its real name to be the __-prefixed version.

> - weak_aliases __funcname_internal

These are rather ugly and stupid, but seem harmless.

> - rawmemchr() (probably your own implementation, not uClibc one)

Indeed, I'll want to copy the existing fast code from strchr.

> - some missing syscalls (I really misguessed number of needed syscalls,
>   seems that this was a number of aliases, not syscalls)

These look mostly fine, but the header changes are all wrong (the
declarations are not under _GNU_SOURCE control).

> glibc-specific functions and objects required to make glxgears work:
> - dlvsym() (wrapper around dlsym(), we don't need ugly symvers)

Ugly because it needs to be written in asm and adds a useless per-arch
asm function. Perhaps we could just have dlvsym be a weak alias to
dlsym in the existing asm files...

> - gnu_get_libc_*()

return "not glibc";

> - malloc_usable_size() (returns 0 always, probably there is no code in
>   musl to make it work. Definition in eglibc was cryptic for me, but it
>   clearly seems to be the glibc/ptmalloc feature)

return (*((size_t *)ptr - 1) & -2) - 2*sizeof(size_t);


> - 4 function-wrappers in math code: __isnan(), __isnanf(), __isinf(),
>   __isinff()

Reasonable to add; perhaps even more efficient than the current
approach.

There's also that function named "finite" whose name violates the
namespace and thus it cannot be in a common source file with other
stuff that could be needed by conformant code.

> - __xmknod()

Ugly but this should be added; it's analogous to the __xstat junk.

> - _IO_2_1_stdout_ -> stdout

Very ugly, but shouldn't break anything...

> gtk2 will not work that way, I checked. One library in chain requires
> libstdc++, libstdc++ defines 'unique' symbols (see manual page of
> binutils nm) which musl linker cannot handle. Additionally, there is
> much more missing symbols including missing functions. But plain X11
> apps worked (I checked xfontsel and xlogo).

Have you looked into building the apps/libs natively against musl
except for the nvidia binary blob, to see if the blob works under that
usage? I think that's a usage case that's a lot more applicable to
real-world usage of musl, and in fact it's probably the first real
reason anybody would be interested in having musl work with code that
was built against glibc...

Rich

Re: Hello

[orc]

On Fri, 6 Jul 2012 02:26:14 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> On Fri, Jul 06, 2012 at 02:06:01PM +0800, orc wrote:
> > Here a patch, attached. It contains both missing syscalls and weak
> > aliases. It does not contain glibc-specific stuff (if you want, I
> > can send it too, but it looks ugly, intended only for 'run it
> > successfully'). Some notes about:
> > - rawmemchr() was taken from uClibc
> > - ioperm() and iopl() were not necessary to make glxgears work, just
> >   added them because Xorg will want them
> > - I don't think libc even needs xattr stuff, but glibc includes
> > them. On many systems they are usually duplicated, libattr provides
> > same interface
> > - It seems that every function in src/locale needs it's __underscore
> >   alias, to match glibc api
> > - there some ugly __funcname_internal aliases, don't know why glibc
> >   defines them in that way
> 
> I think most of this is an artifact of some hacks they do in the glibc
> headers for inlining/optimization and/or _FORTIFY_SOURCE support.
> The functions it creates references to are not (AFAIK) in the LSB ABI
> and thus binaries using them are non-LSB-conforming...
> 
> > Probably you will want to add:
> > - weak_aliases for __underscores
> 
> Except most of them should be in the opposite direction. Especially
> for functions like strxfrm_l where we'll eventually want the ISO C
> "foo" function to depend on the POSIX "foo_l" function, the latter
> will need its real name to be the __-prefixed version.
> 
> > - weak_aliases __funcname_internal
> 
> These are rather ugly and stupid, but seem harmless.
> 
> > - rawmemchr() (probably your own implementation, not uClibc one)
> 
> Indeed, I'll want to copy the existing fast code from strchr.
> 
> > - some missing syscalls (I really misguessed number of needed
> > syscalls, seems that this was a number of aliases, not syscalls)
> 
> These look mostly fine, but the header changes are all wrong (the
> declarations are not under _GNU_SOURCE control).

Of course adapt the patch to be correct here. My local manpages did not
say much about feature test macros.

> 
> > glibc-specific functions and objects required to make glxgears work:
> > - dlvsym() (wrapper around dlsym(), we don't need ugly symvers)
> 
> Ugly because it needs to be written in asm and adds a useless per-arch
> asm function. Perhaps we could just have dlvsym be a weak alias to
> dlsym in the existing asm files...
> 
> > - gnu_get_libc_*()
> 
> return "not glibc";
> 
> > - malloc_usable_size() (returns 0 always, probably there is no code
> > in musl to make it work. Definition in eglibc was cryptic for me,
> > but it clearly seems to be the glibc/ptmalloc feature)
> 
> return (*((size_t *)ptr - 1) & -2) - 2*sizeof(size_t);

It will be nice to see correct implementation instead of 'always-fail',
cool.

> 
> 
> > - 4 function-wrappers in math code: __isnan(), __isnanf(),
> > __isinf(), __isinff()
> 
> Reasonable to add; perhaps even more efficient than the current
> approach.
> 
> There's also that function named "finite" whose name violates the
> namespace and thus it cannot be in a common source file with other
> stuff that could be needed by conformant code.

Yes, I also did not sure about it.

> 
> > - __xmknod()
> 
> Ugly but this should be added; it's analogous to the __xstat junk.
> 
> > - _IO_2_1_stdout_ -> stdout
> 
> Very ugly, but shouldn't break anything...


*looking in glibc-compat patch*
I forgot this one:
- A _*_chk versions of common functions like memcpy, *printf, etc. and
  longjmp. They are nearly same as their non-chk versions (additional
  arg) except that they do additional buffer overflow checks. I did
  quick weak_alias to non-chk versions to test that it will work. It
  worked, but what do you think about them?

> 
> > gtk2 will not work that way, I checked. One library in chain
> > requires libstdc++, libstdc++ defines 'unique' symbols (see manual
> > page of binutils nm) which musl linker cannot handle. Additionally,
> > there is much more missing symbols including missing functions. But
> > plain X11 apps worked (I checked xfontsel and xlogo).
> 
> Have you looked into building the apps/libs natively against musl
> except for the nvidia binary blob, to see if the blob works under that
> usage? I think that's a usage case that's a lot more applicable to
> real-world usage of musl, and in fact it's probably the first real
> reason anybody would be interested in having musl work with code that
> was built against glibc...

I mostly do building in KVM now to test how applications are portable
and collect the patches that will be needed when transiting to
musl-enabled system. I was busy with patching old gcc and binutils these
days, since I found that musl systems by default are built with
executable stack (GNU_STACK thing) enabled (binutils issue). I will try
that on host of course, but I generally dislike cross-compiling even to
same arch because of autotools and pkg-config (will be required by
gtk2 stuff).

> 
> Rich

Re: Hello

[idunham]

>> > gtk2 will not work that way, I checked. One library in chain
>> > requires libstdc++, libstdc++ defines 'unique' symbols (see manual
>> > page of binutils nm) which musl linker cannot handle. Additionally,
>> > there is much more missing symbols including missing functions. But
>> > plain X11 apps worked (I checked xfontsel and xlogo).
>>
>> Have you looked into building the apps/libs natively against musl
>> except for the nvidia binary blob, to see if the blob works under that
>> usage? I think that's a usage case that's a lot more applicable to
>> real-world usage of musl, and in fact it's probably the first real
>> reason anybody would be interested in having musl work with code that
>> was built against glibc...
The first step would probably be dropping a musl-compatible (generic, not
gnu-linux) libstdc++ into the test environment.
> I mostly do building in KVM now to test how applications are portable
> and collect the patches that will be needed when transiting to
> musl-enabled system. I was busy with patching old gcc and binutils these
> days, since I found that musl systems by default are built with
> executable stack (GNU_STACK thing) enabled (binutils issue). I will try
> that on host of course, but I generally dislike cross-compiling even to
> same arch because of autotools and pkg-config (will be required by
> gtk2 stuff
I presume you know to use PKG_CONFIG_LIBDIR ? With that set,
cross-compiling seemed fairly easy to me.
Also, I often set up a chroot with musl and build under that.  If you'd
rather not build the whole chroot yourself, you can prepare a chroot and
follow the installation notes for converting a glibc-based system, using
the chroot environment instead of a full install (this works well with
Debian).
If you do this, make sure not to do a plain cp of the system libraries;
this would mess up the sonames, since lib*.so must be symlinks to
lib*.so.<version>.  It also really messes with ldconfig.

Re: Hello

[Rich Felker]

On Fri, Jul 06, 2012 at 04:14:17PM -0700, idunham@lavabit.com wrote:
> >> Have you looked into building the apps/libs natively against musl
> >> except for the nvidia binary blob, to see if the blob works under that
> >> usage? I think that's a usage case that's a lot more applicable to
> >> real-world usage of musl, and in fact it's probably the first real
> >> reason anybody would be interested in having musl work with code that
> >> was built against glibc...
> The first step would probably be dropping a musl-compatible (generic, not
> gnu-linux) libstdc++ into the test environment.

Actually, generic vs gnu-linux locale crap is not the issue here. The
*ABI* support stuff in musl should make the gnu-linux code work on
musl if it was already compiled using glibc headers. It's the headers
that are incompatible with the gnu-linux os tree because we don't do
the ugly table-based stuff (which is much more bloated and probably
slower).

The issue that DOES matter with libstdc++ is name mangling. musl's
ABI-compat is limited to C. The type sizes and non-opaque type
representations match what's used on glibc/LSB, but the struct tags
and specific types for abstract opaque types like FILE and pthread_t
differ, meaning the name mangling will differ... Thus while you can in
theory use glibc-linked C++ programs with musl's LSB ABI support, you
can't mix-and-match glibc-linked C++ program/library code and
musl-linked C++ program/library code in a single program. The name
mangling will fail to match and the dynamic linker will fail to
resolve the C++ stuff.

Rich

Re: Hello

[orc]

On Fri, 6 Jul 2012 16:14:17 -0700 (PDT)
idunham@lavabit.com wrote:

> >> > gtk2 will not work that way, I checked. One library in chain
> >> > requires libstdc++, libstdc++ defines 'unique' symbols (see
> >> > manual page of binutils nm) which musl linker cannot handle.
> >> > Additionally, there is much more missing symbols including
> >> > missing functions. But plain X11 apps worked (I checked xfontsel
> >> > and xlogo).
> >>
> >> Have you looked into building the apps/libs natively against musl
> >> except for the nvidia binary blob, to see if the blob works under
> >> that usage? I think that's a usage case that's a lot more
> >> applicable to real-world usage of musl, and in fact it's probably
> >> the first real reason anybody would be interested in having musl
> >> work with code that was built against glibc...
> The first step would probably be dropping a musl-compatible (generic,
> not gnu-linux) libstdc++ into the test environment.

Bootstrapping libstdc++ I think even more nightmarish than dealing with
pkg-config or autohell, since this library is part of gcc. So I went in
another way: I've taken my package cache and relinked gtk2 and pango,
now they do not depend on libstdc++ (since they do not import things
from that library. Why they were linked to it - is another open
question.)
After some dancing around undefined symbols I've got not-so-big list:

__res_init(), __res_iclose(), inet_nsap_ntoa(), __res_maybe_init(),
initstate_r(), random_r() and stuff related to pthread scheduling:
pthread_setschedparam(), pthread_getschedparam().

Since that gtk2 and it's stuff do not require libstdc++ to work I'll
try to build them against musl.

> > I mostly do building in KVM now to test how applications are
> > portable and collect the patches that will be needed when
> > transiting to musl-enabled system. I was busy with patching old gcc
> > and binutils these days, since I found that musl systems by default
> > are built with executable stack (GNU_STACK thing) enabled (binutils
> > issue). I will try that on host of course, but I generally dislike
> > cross-compiling even to same arch because of autotools and
> > pkg-config (will be required by gtk2 stuff
> I presume you know to use PKG_CONFIG_LIBDIR ? With that set,
> cross-compiling seemed fairly easy to me.

Did know about PKG_CONFIG_PATH, so looks like replacing default
directory will solve that problems when looked into, not found stuff
and tried to link with host libs.

> Also, I often set up a chroot with musl and build under that.  If
> you'd rather not build the whole chroot yourself, you can prepare a
> chroot and follow the installation notes for converting a glibc-based
> system, using the chroot environment instead of a full install (this
> works well with Debian).
> If you do this, make sure not to do a plain cp of the system
> libraries; this would mess up the sonames, since lib*.so must be
> symlinks to lib*.so.<version>.  It also really messes with ldconfig.

Converting existing system maybe possible, but I will prefer to rebuild
all the stuff from scratch with musl and (possibly in future) with
different compiler(s) (I really tired from gcc/binutils, but there
seems no any alternatives written in C and able to build major
software like gtk2 and Linux kernel). Converting will be possible when
musl will be binary compatible with glibc, but this is not primary
goal. As a bonus, maybe. One issue here was that binary nvidia blob
that can be needed even after rebuild on desktop, so missing interfaces
were not big deal with it. Other things can be patched/recompiled.

> 
> 

Re: Hello

[idunham]

> On Fri, 6 Jul 2012 16:14:17 -0700 (PDT)
> idunham@lavabit.com wrote:
>
>> >> > gtk2 will not work that way, I checked. One library in chain
>> >> > requires libstdc++, libstdc++ defines 'unique' symbols (see
>> >> > manual page of binutils nm) which musl linker cannot handle.
>> >> > Additionally, there is much more missing symbols including
>> >> > missing functions. But plain X11 apps worked (I checked xfontsel
>> >> > and xlogo).
>> >>
>> >> Have you looked into building the apps/libs natively against musl
>> >> except for the nvidia binary blob, to see if the blob works under
>> >> that usage? I think that's a usage case that's a lot more
>> >> applicable to real-world usage of musl, and in fact it's probably
>> >> the first real reason anybody would be interested in having musl
>> >> work with code that was built against glibc...
>> The first step would probably be dropping a musl-compatible (generic,
>> not gnu-linux) libstdc++ into the test environment.
>
> Bootstrapping libstdc++ I think even more nightmarish than dealing with
> pkg-config or autohell, since this library is part of gcc. So I went in
> another way: I've taken my package cache and relinked gtk2 and pango,
> now they do not depend on libstdc++ (since they do not import things
> from that library. Why they were linked to it - is another open
> question.)
Apparently this wouldn't be ABI compatible (IE-don't bother trying), but
it isn't too hard:
apply the musl patches, then (if they don't already do this)
cd libstdc++-v3/os;  rm gnu-linux; cp generic gnu-linux; cd -

> After some dancing around undefined symbols I've got not-so-big list:
>
> __res_init(), __res_iclose(), inet_nsap_ntoa(), __res_maybe_init(),
> initstate_r(), random_r() and stuff related to pthread scheduling:
> pthread_setschedparam(), pthread_getschedparam().
>
> Since that gtk2 and it's stuff do not require libstdc++ to work I'll
> try to build them against musl.

>> Also, I often set up a chroot with musl and build under that.  If
>> you'd rather not build the whole chroot yourself, you can prepare a
>> chroot and follow the installation notes for converting a glibc-based
>> system, using the chroot environment instead of a full install (this
>> works well with Debian).
>> If you do this, make sure not to do a plain cp of the system
>> libraries; this would mess up the sonames, since lib*.so must be
>> symlinks to lib*.so.<version>.  It also really messes with ldconfig.
>
> Converting existing system maybe possible, but I will prefer to rebuild
> all the stuff from scratch with musl and (possibly in future) with
> different compiler(s) (I really tired from gcc/binutils, but there
> seems no any alternatives written in C and able to build major
> software like gtk2 and Linux kernel). Converting will be possible when
> musl will be binary compatible with glibc, but this is not primary
> goal. As a bonus, maybe. One issue here was that binary nvidia blob
> that can be needed even after rebuild on desktop, so missing interfaces
> were not big deal with it. Other things can be patched/recompiled.

Converting a system (as described in musl/INSTALL) does not mean replacing
libc; it means a parallel install of musl, move system libs out of the
default search path, then change gcc to use musl by default. It would work
 even with an ABI-incompatible libc, since you leave ld-linux alone.

Re: Hello

[orc]

On Sat, 7 Jul 2012 11:54:08 -0400 (EDT)
idunham@lavabit.com wrote:

> Converting a system (as described in musl/INSTALL) does not mean
> replacing libc; it means a parallel install of musl, move system libs
> out of the default search path, then change gcc to use musl by
> default. It would work even with an ABI-incompatible libc, since you
> leave ld-linux alone.

I still think that this is a generally bad idea - having dancing around
to just make it work with musl. I will do rebuild instead, with reduced
number of packages.
(Nvidia was as an exception here: it is binary blob designed to work
only with glibc)

Re: Hello

[Rich Felker]

On Sun, Jul 22, 2012 at 08:49:50PM -0700, idunham@lavabit.com wrote:
> > I need to look back at the archives but it seems a lot of this patch
> > was about getting glibc binaries to run with musl. Can we break the
> > efforts to integrate it down into 3 steps:
> 
> I broke it down into 10 patches, taking roughly the same approach.

Of these, 1-4 have been committed with a few fixes (pipe2 was wrongly
using SYS_pipe instead of SYS_pipe2, header issues, etc.) and
additions (vmsplice and macros needed for using splice). This covers
the syscall additions; the rest of the patches are duplicate symbols
for ABI purposes, and I'd like to head a better report of what's
really needed before going with them.

Rich

Re: Hello

[idunham]

> On Sun, Jul 22, 2012 at 07:09:25PM -0400, idunham@lavabit.com wrote:
>> I've been getting a little impatient waiting to see if anything happens,
>> so I started going through orc's patch and revising it.
>
> Sorry, I've had fairly little in the way of contiguous blocks of time
> to work on stuff over the past few weeks. Trying to catch up a bit...
Not directed at you in particular, and I understand that.


> I need to look back at the archives but it seems a lot of this patch
> was about getting glibc binaries to run with musl. Can we break the
> efforts to integrate it down into 3 steps:

I broke it down into 10 patches, taking roughly the same approach.

> 1. Linux API-level stuff musl should be supporting regardless of glibc
> compatibility.
See patches 1-4.  These are the missing syscalls.

According to what orc was saying, patch 1 was what he needed for Xorg.
Gregor also said he could use it.

> 2. glibc-compatibility symbols that need to be exported to get
> high-demand binary blobs (like video drivers) working.

> 3. Additional glibc-compatibility symbols, which may or may not be
> wanted/needed/desirable in the long term, and which we can at least
> defer addressing for a while.

orc would know better than I, but it *seemed* that he was saying that
everything in his patch fell in 1 or 2 ...

>> While looking at arm, I noticed that x86/mips gas uses @function while
>> arm uses %function...is there a reason for this?
>
> I think it's just a difference in the asm syntax rules for different
> targets. They're all very inconsistent...
OK, thanks.

>> >> > Probably you will want to add:
>> >> > - weak_aliases for __underscores

>> >> Except most of them should be in the opposite direction. Especially
>> >> for functions like strxfrm_l where we'll eventually want the ISO C
>> >> "foo" function to depend on the POSIX "foo_l" function, the latter
>> >> will need its real name to be the __-prefixed version.
>> Are there any of these that should not be the other way around?

> Need to review again...

See patches 5-10 (except the finite() patch)
I figured that any *isoc99* or __*_internal aliases were for ABI only and
left them as orc had it.
Everything in src/locale/ I assumed (!) was not ISO C99, and did them "in
the opposite direction".

Re: Hello

[Rich Felker]

On Sun, Jul 22, 2012 at 07:09:25PM -0400, idunham@lavabit.com wrote:
> I've been getting a little impatient waiting to see if anything happens,
> so I started going through orc's patch and revising it.

Sorry, I've had fairly little in the way of contiguous blocks of time
to work on stuff over the past few weeks. Trying to catch up a bit...

I need to look back at the archives but it seems a lot of this patch
was about getting glibc binaries to run with musl. Can we break the
efforts to integrate it down into 3 steps:

1. Linux API-level stuff musl should be supporting regardless of glibc
compatibility.

2. glibc-compatibility symbols that need to be exported to get
high-demand binary blobs (like video drivers) working.

3. Additional glibc-compatibility symbols, which may or may not be
wanted/needed/desirable in the long term, and which we can at least
defer addressing for a while.

> 1. It still applies cleanly.

Good. :-)

> 2. __sigsetjmp is only added on x86(_64).
> While looking at arm, I noticed that x86/mips gas uses @function while arm
> uses %function...is there a reason for this?

I think it's just a difference in the asm syntax rules for different
targets. They're all very inconsistent...

> 3. splice() is added to fcntl.h with _GNU_SOURCE, but needs (s)size_t;
> what I did was add
> #ifdef _GNU_SOURCE
> #define __NEED_size_t
> #define __NEED_ssize_t
> #endif
> 
> above where alltypes.h was included.

Looks correct.

> >> > - rawmemchr() was taken from uClibc
> (I'm temporarily dropping that part...)
> >> > - ioperm() and iopl() were not necessary to make glxgears work, just
> >> >   added them because Xorg will want them
> >> > Probably you will want to add:
> >> > - weak_aliases for __underscores
> >>
> >> Except most of them should be in the opposite direction. Especially
> >> for functions like strxfrm_l where we'll eventually want the ISO C
> >> "foo" function to depend on the POSIX "foo_l" function, the latter
> >> will need its real name to be the __-prefixed version.
> Are there any of these that should not be the other way around?

Need to review again...

Rich

Re: Hello

[idunham]

I've been getting a little impatient waiting to see if anything happens,
so I started going through orc's patch and revising it.

1. It still applies cleanly.

2. __sigsetjmp is only added on x86(_64).
While looking at arm, I noticed that x86/mips gas uses @function while arm
uses %function...is there a reason for this?

3. splice() is added to fcntl.h with _GNU_SOURCE, but needs (s)size_t;
what I did was add
#ifdef _GNU_SOURCE
#define __NEED_size_t
#define __NEED_ssize_t
#endif

above where alltypes.h was included.

>> > - rawmemchr() was taken from uClibc
(I'm temporarily dropping that part...)
>> > - ioperm() and iopl() were not necessary to make glxgears work, just
>> >   added them because Xorg will want them
>> > Probably you will want to add:
>> > - weak_aliases for __underscores
>>
>> Except most of them should be in the opposite direction. Especially
>> for functions like strxfrm_l where we'll eventually want the ISO C
>> "foo" function to depend on the POSIX "foo_l" function, the latter
>> will need its real name to be the __-prefixed version.
Are there any of these that should not be the other way around?
>> > - weak_aliases __funcname_internal
>>
>> These are rather ugly and stupid, but seem harmless.

>>
>> > glibc-specific functions and objects required to make glxgears work:
...
>> > - gnu_get_libc_*()
>> return "not glibc";