roadmap for 0.9.8 ?

roadmap for 0.9.8 ?

[John Spencer]

i was thinking about what will go into the next release...

open positions that were originally intended to go into 0.9.7 are, as 
taken from the latest "pending for release" mails:

- dl_iterate_phdr() - looks as if this is currently cooking, and 
possibly an important bit to improve libgcc support,

- explicit mips soft-float abi support (mainly for use with broken
    openwrt kernels) ,

- wordexp ?

- subarchs in build system (e.g. mips-softfloat, arm-hardfloat, etc.),

- nsz' suggested math improvements ?

additionally, i think it'd make sense to add stubs for the missing 
optional schedule functions (as most of them are already there), that 
way we could build some exotic programs like "fluidsynth" without 
patches, and further increase the pkgsrc success rate:

posix_spawnattr_setschedparam
posix_spawnattr_setschedpolicy
posix_spawnattr_getschedparam
posix_spawnattr_getschedpolicy
pthread_attr_getinheritsched
pthread_attr_getschedpolicy
pthread_attr_setinheritsched
pthread_attr_setschedpolicy
pthread_getschedparam
pthread_mutex_getprioceiling
pthread_mutex_setprioceiling
pthread_mutexattr_getprioceiling
pthread_mutexattr_getprotocol
pthread_mutexattr_setprioceiling
pthread_mutexattr_setprotocol
pthread_setschedparam
pthread_setschedprio
pthread_getschedprio
(i hope that's complete)

- ppc port ?

is there something else i missed, and what are your plans ?

besides of musl itself, i think it would be a nice achievement if we 
could get the microblaze toolchain patches integrated in the upstream 
repos (or at least patches against current gcc/binutils trees), so that 
we distro-builders dont need to special-case for microblaze.

Re: roadmap for 0.9.8 ?

[Rich Felker]

On Thu, Nov 01, 2012 at 12:37:27AM +0100, John Spencer wrote:
> i was thinking about what will go into the next release...
> 
> open positions that were originally intended to go into 0.9.7 are,
> as taken from the latest "pending for release" mails:
> 
> - dl_iterate_phdr() - looks as if this is currently cooking, and
> possibly an important bit to improve libgcc support,

I'm planning to get this done in the next 48 hours. Will libgcc
automatically use dl_iterate_phdr once it's added, or is there some
hard-coded glibc-specific logic we need to check in the -linux-musl
target patches for gcc?

> - explicit mips soft-float abi support (mainly for use with broken
>    openwrt kernels) ,

Mainly this requires a discussion of whether it should go in the build
system (sub-archs/abis) or just at the source level somewhere.

> - wordexp ?

It's no hurry since this is a crap interface nobody uses anyway, but I
would like to update it to use vfork, possibly by making it use
posix_spawn rather than duplicating the functionality. I would use
popen, which in many ways seems like the more natural choice, but that
requires transforming the argument into an appropriate escaped form
and allocating a temporary string for it.

Speaking of that, right now all uses of vfork are rather expensive
because we have to loop and check each signal to check that it doesn't
have a handler set. I'd like to make a global atomic bit array in
sigaction.c to store the set of signals which have ever had their
disposition set to a signal handling function. Then it would be easy
to loop over just that set when resetting, and in fact the resetting
code could be shared between different places that need it
(posix_spawn, system, popen, wordexp).

> - subarchs in build system (e.g. mips-softfloat, arm-hardfloat, etc.),

Yes, this mainly requires some thinking (or experimenting) to see what
the real requirements are.

> - nsz' suggested math improvements ?
> 
> additionally, i think it'd make sense to add stubs for the missing
> optional schedule functions (as most of them are already there),
> that way we could build some exotic programs like "fluidsynth"
> without patches, and further increase the pkgsrc success rate:

Hopefully we can just add real versions of them. A good half of the
effort of doing these is going to be just checking that we get all the
files added right, so from that standpoint we might as well just do it
all in one step rather than having to go back and make sure we get
everything right later when we fill them out. I don't think there are
any more spinlocks or other things that would make problems in
priority-enabled applications.

> - ppc port ?

Yes. Is anybody else up for doing the work integrating it, or should
I plan to do it myself? It'd be really nice to have somebody familiar
enough with the porting procedure to handle ports integration.

> is there something else i missed, and what are your plans ?

Yeah, I was pretty tired when I pushed the last release out; that's
why it wasn't accompanied by a development direction paragraph.

One thing that comes to mind is updating the wiki and keeping it
relevant. Most of the FAQ items are obsolete now.

Perhaps something to address soon is the missing 'm' modifier in scanf
(POSIX extension for malloc'ing strings rather than using
caller-provided buffers).

An internal task I want to work on is removing most/all of the
#include directives from stdio_impl.h. I think this would improve
compile time noticably. But it will not add any functionality, of
course.

We could also work on more C11 features, like <uchar.h> or C11
threads. C11 threads should just be a matter of making namespace-clean
aliases for the pthread functions and wrapping those with thrd_*
names. However, for the enums we might need to wait for glibc folks to
assign values.

I know you want some fixes/improvements in configure.

Aside from that, I'd like this release cycle to be much shorter. The
changes we're looking at are a lot less invasive (compared to TLS) so
I'll feel comfortable releasing sooner rather than sitting around
waiting for bugs to pop up like the last release cycle.

> besides of musl itself, i think it would be a nice achievement if we
> could get the microblaze toolchain patches integrated in the
> upstream repos (or at least patches against current gcc/binutils
> trees), so that we distro-builders dont need to special-case for
> microblaze.

Yes, I would like that very much. Have you looked into it at all?
According to Steve Bennett on the microblaze-linux list, David
Holsgrove is working on getting microblaze support upstreamed in
crosstool-ng. He's also employed by Xilinx. He would probably be the
person to contact about this, and I think it would be very nice for
him to be aware of musl as a development tool for microblaze
(especially since they currently seem to be lacking in tools that are
appropriately-sized for the sort of applications microblaze sounds
most useful in).

Another side project might be support for C11 Annex K. It's under
discussion on the glibc list right now. I'm not sure whether they'll
end up having it as an external library or built-in, but we might end
up needing to go the same way for binary-compat. My leaning is an
external library that's libc-agnostic, but it sounds like they want
tighter libc feature integration, so we might need to have a forked
(or rewritten) portable version that would work with musl even if it
is external. I consider this rather low priority since nobody is using
annex K (these are the ugly MS _s functions), but on the other hand
it's something somebody could do on the side as a learning exercise
(and useful task at the same time) without having to understand any
libc internals.

Rich

Re: roadmap for 0.9.8 ?

[Rich Felker]

Some updates; hoping to do a release again soon...

On Wed, Oct 31, 2012 at 08:05:19PM -0400, Rich Felker wrote:
> > - dl_iterate_phdr() - looks as if this is currently cooking, and
> > possibly an important bit to improve libgcc support,
> 
> I'm planning to get this done in the next 48 hours. Will libgcc

This goal was met; done.

> > - explicit mips soft-float abi support (mainly for use with broken
> >    openwrt kernels) ,
> 
> Mainly this requires a discussion of whether it should go in the build
> system (sub-archs/abis) or just at the source level somewhere.

Still on the table for discussion.

> Speaking of that, right now all uses of vfork are rather expensive
> because we have to loop and check each signal to check that it doesn't
> have a handler set. I'd like to make a global atomic bit array in
> sigaction.c to store the set of signals which have ever had their
> disposition set to a signal handling function. Then it would be easy
> to loop over just that set when resetting, and in fact the resetting
> code could be shared between different places that need it
> (posix_spawn, system, popen, wordexp).

This is another cleanup task which most people won't even see or
appreciate (except perhaps in benchmarks), but I think I'll tackle it
soon anyway.

> > additionally, i think it'd make sense to add stubs for the missing
> > optional schedule functions (as most of them are already there),
> > that way we could build some exotic programs like "fluidsynth"
> > without patches, and further increase the pkgsrc success rate:
> 
> Hopefully we can just add real versions of them. A good half of the

This is also on my agenda at high priority (pardon the pun).

> > - ppc port ?
> 
> Yes. Is anybody else up for doing the work integrating it, or should
> I plan to do it myself? It'd be really nice to have somebody familiar
> enough with the porting procedure to handle ports integration.

No progress yet. I'd still be interested in mentoring somebody
interested in doing this and/or other ports in hopes of getting to the
point where we have a co-maintainer for ports. This person would not
need to be an expert on musl internals, just familiar with asm, cross
or native compiling for non-x86 archs, and basic debugging (strace/gdb
usage), and would be the "go-to" person for arch-specific bug reports.

> One thing that comes to mind is updating the wiki and keeping it
> relevant. Most of the FAQ items are obsolete now.

Volunteers?

> Perhaps something to address soon is the missing 'm' modifier in scanf

Still pending. I'd like to do it before the next release.

> An internal task I want to work on is removing most/all of the
> #include directives from stdio_impl.h. I think this would improve

Done. Also did pthread_impl.h. Much cleaner now.

> We could also work on more C11 features, like <uchar.h> or C11
> threads. C11 threads should just be a matter of making namespace-clean

Pending. I'll probably hold off on this until the following release
cycle.

> I know you want some fixes/improvements in configure.

Done.

> Aside from that, I'd like this release cycle to be much shorter. The
> changes we're looking at are a lot less invasive (compared to TLS) so
> I'll feel comfortable releasing sooner rather than sitting around
> waiting for bugs to pop up like the last release cycle.

Being that we have some major improvements (dl_iterate_phdr and
several show-stopping MIPS-specific bugs fixed), I'd like to aim to
have the next release be made pretty soon. Please let me know if there
are other issues that should be addressed before release.

Rich

Re: roadmap for 0.9.8 ?

[John Spencer]

On 11/08/2012 11:24 PM, Rich Felker wrote:
> - explicit mips soft-float abi support (mainly for use with broken
>>>     openwrt kernels) ,
>> Mainly this requires a discussion of whether it should go in the build
>> system (sub-archs/abis) or just at the source level somewhere.
> Still on the table for discussion.

i don't have an opinion on this, however:
for path names that are using $ARCH, such as the ld-musl files, it would 
be nice if for example mips and mipsel had different filenames (for 
multi-lib installations).
same applies for armv4tl vs armv7l, etc

>> Speaking of that, right now all uses of vfork are rather expensive
>> because we have to loop and check each signal to check that it doesn't
>> have a handler set. I'd like to make a global atomic bit array in
>> sigaction.c to store the set of signals which have ever had their
>> disposition set to a signal handling function. Then it would be easy
>> to loop over just that set when resetting, and in fact the resetting
>> code could be shared between different places that need it
>> (posix_spawn, system, popen, wordexp).
> This is another cleanup task which most people won't even see or
> appreciate (except perhaps in benchmarks), but I think I'll tackle it
> soon anyway.

oh, i think everyone appreciates faster / less memory-hungry code.

>>> additionally, i think it'd make sense to add stubs for the missing
>>> optional schedule functions (as most of them are already there),
>>> that way we could build some exotic programs like "fluidsynth"
>>> without patches, and further increase the pkgsrc success rate:
>> Hopefully we can just add real versions of them. A good half of the
> This is also on my agenda at high priority (pardon the pun).
will it make it into the next release ?
i think, for the moment, having it as stubs would perfectly suffice.

>>> - ppc port ?
>> Yes. Is anybody else up for doing the work integrating it, or should
>> I plan to do it myself? It'd be really nice to have somebody familiar
>> enough with the porting procedure to handle ports integration.
> No progress yet. I'd still be interested in mentoring somebody
> interested in doing this and/or other ports in hopes of getting to the
> point where we have a co-maintainer for ports. This person would not
> need to be an expert on musl internals, just familiar with asm, cross
> or native compiling for non-x86 archs, and basic debugging (strace/gdb
> usage), and would be the "go-to" person for arch-specific bug reports.

i guess the ppc port can be postponed.

>> One thing that comes to mind is updating the wiki and keeping it
>> relevant. Most of the FAQ items are obsolete now.
> Volunteers?
i cleaned up the obsolete stuff.

>> Perhaps something to address soon is the missing 'm' modifier in scanf
> Still pending. I'd like to do it before the next release.
>
>> An internal task I want to work on is removing most/all of the
>> #include directives from stdio_impl.h. I think this would improve
> Done. Also did pthread_impl.h. Much cleaner now.
nice, i especially like the memset removal in leaf-functions.
which could lead to what we discussed on IRC a while ago (object files 
that are identical for PIC and non-PIC should only be built once)
>> We could also work on more C11 features, like<uchar.h>  or C11
>> threads. C11 threads should just be a matter of making namespace-clean
> Pending. I'll probably hold off on this until the following release
> cycle.
yeah, that doesn't seem especially urgent.
>> I know you want some fixes/improvements in configure.
> Done.
thanks
>> Aside from that, I'd like this release cycle to be much shorter. The
>> changes we're looking at are a lot less invasive (compared to TLS) so
>> I'll feel comfortable releasing sooner rather than sitting around
>> waiting for bugs to pop up like the last release cycle.
> Being that we have some major improvements (dl_iterate_phdr and
> several show-stopping MIPS-specific bugs fixed), I'd like to aim to
> have the next release be made pretty soon.
*nod*
>   Please let me know if there
> are other issues that should be addressed before release.
everything that came to mind is fixed by now.

Re: roadmap for 0.9.8 ?

[Rich Felker]

On Fri, Nov 09, 2012 at 04:36:47PM +0100, John Spencer wrote:
> On 11/08/2012 11:24 PM, Rich Felker wrote:
> >- explicit mips soft-float abi support (mainly for use with broken
> >>>    openwrt kernels) ,
> >>Mainly this requires a discussion of whether it should go in the build
> >>system (sub-archs/abis) or just at the source level somewhere.
> >Still on the table for discussion.
> 
> i don't have an opinion on this, however:
> for path names that are using $ARCH, such as the ld-musl files, it
> would be nice if for example mips and mipsel had different filenames
> (for multi-lib installations).
> same applies for armv4tl vs armv7l, etc

mips and mipsel are *different* archs/abis, so I agree it would make
sense for the dynamic linker to be named differently on mipsel.
However, armv4t and armv7 are the same arch. If the hardware is little
endian and supports armv7 ISA, there's no reason the same dynamic
linker can't be used both for apps built at armv7-native and older
armv4-compatible binaries. There's no reason the name of the dynamic
linker needs to indicate that the library is using v7-specific
features, because the ABI for linking to the library is the same
either way.

Of course, armeb and arm should be separate archs, for naming
purposes, just like mips and mipsel.

> >>>additionally, i think it'd make sense to add stubs for the missing
> >>>optional schedule functions (as most of them are already there),
> >>>that way we could build some exotic programs like "fluidsynth"
> >>>without patches, and further increase the pkgsrc success rate:
> >>Hopefully we can just add real versions of them. A good half of the
> >This is also on my agenda at high priority (pardon the pun).
> will it make it into the next release ?
> i think, for the moment, having it as stubs would perfectly suffice.

Yes, I think so. Like I said before, adding stubs and making sure
their behavior is conforming is almost as much work as making
operational versions of these functions.

> i guess the ppc port can be postponed.

My inclination would be to postpone it, but if there are any
volunteers to help, we could perhaps include it as an experimental
port.

> >>One thing that comes to mind is updating the wiki and keeping it
> >>relevant. Most of the FAQ items are obsolete now.
> >Volunteers?
> i cleaned up the obsolete stuff.

Thanks.

> >>An internal task I want to work on is removing most/all of the
> >>#include directives from stdio_impl.h. I think this would improve
> >Done. Also did pthread_impl.h. Much cleaner now.
> nice, i especially like the memset removal in leaf-functions.
> which could lead to what we discussed on IRC a while ago (object
> files that are identical for PIC and non-PIC should only be built
> once)

The biggest problem here is that I don't know any way to detect if a
file would be different with -fPIC without actually compiling it to
see (which would take just as long).

> >>I know you want some fixes/improvements in configure.
> >Done.
> thanks

Does it meet your needs?

> >  Please let me know if there
> >are other issues that should be addressed before release.
> everything that came to mind is fixed by now.

Great.

Rich