* Proposed roadmap to 1.0 @ 2013-06-29 23:50 Rich Felker 2013-06-30 2:53 ` Rob Landley ` (4 more replies) 0 siblings, 5 replies; 30+ messages in thread From: Rich Felker @ 2013-06-29 23:50 UTC (permalink / raw) To: musl Hi all, Here is a VERY tentative, proposed roadmap towards a 1.0 release of musl. Comments welcome! Rich 0.9.11 Projected release: ASAP No further goals at the moment except fixing additional bugs found. 0.9.12 Projected release: Mid to late July Key targets: - Overhaul of time handling, including zoneinfo support. - Overhaul resolver to better provide legacy APIs without code dup. - Hybrid automatic/manual audit for cruft and code smells. - Resolve symlink direction issue for dynamic linker. - Affinity/cpuset interfaces. 0.9.13 Projected release: Early August Key targets: - Full C++ ABI compatibility with glibc/LSB. - Support for all remaining iconv charsets of interest (KR/TW/HK). - Possible overhaul of iconv for performance and clarity/simplicity. - Possibly add stateful iconv support. - Establish formal procedure for regression testing. 0.9.14 Projected release: End of summer Key targets: - Complete documentation draft. - Performance testing on under-tested archs, fixing bottlenecks hit. - Review for gratuitous application breakage (anything that could be fixed with trivial changes that don't hurt musl's quality). 1.0.0 Projected release: Early fall Key targets: - Polished documentation. - Organized and coordinated publicity plan. - At least one new exciting addition to make the release noteworthy, but which has no chance of breaking things that work. Best candidate would be one or more new ports, labeled experimental. ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-29 23:50 Proposed roadmap to 1.0 Rich Felker @ 2013-06-30 2:53 ` Rob Landley 2013-06-30 3:01 ` Rich Felker 2013-06-30 5:20 ` Isaac ` (3 subsequent siblings) 4 siblings, 1 reply; 30+ messages in thread From: Rob Landley @ 2013-06-30 2:53 UTC (permalink / raw) To: musl; +Cc: musl On 06/29/2013 06:50:41 PM, Rich Felker wrote: > Hi all, > > Here is a VERY tentative, proposed roadmap towards a 1.0 release of > musl. Comments welcome! > > Rich > > 0.9.11 > Projected release: ASAP > No further goals at the moment except fixing additional bugs found. > > 0.9.12 > Projected release: Mid to late July > Key targets: > - Overhaul of time handling, including zoneinfo support. > - Overhaul resolver to better provide legacy APIs without code dup. > - Hybrid automatic/manual audit for cruft and code smells. > - Resolve symlink direction issue for dynamic linker. > - Affinity/cpuset interfaces. So this would be about the "feature complte for not-internationalized C99" level? I'll try to get support into aboriginal linux for that release. Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 2:53 ` Rob Landley @ 2013-06-30 3:01 ` Rich Felker 0 siblings, 0 replies; 30+ messages in thread From: Rich Felker @ 2013-06-30 3:01 UTC (permalink / raw) To: musl On Sat, Jun 29, 2013 at 09:53:29PM -0500, Rob Landley wrote: > On 06/29/2013 06:50:41 PM, Rich Felker wrote: > >Hi all, > > > >Here is a VERY tentative, proposed roadmap towards a 1.0 release of > >musl. Comments welcome! > > > >Rich > > > >0.9.11 > >Projected release: ASAP > >No further goals at the moment except fixing additional bugs found. > > > >0.9.12 > >Projected release: Mid to late July > >Key targets: > >- Overhaul of time handling, including zoneinfo support. > >- Overhaul resolver to better provide legacy APIs without code dup. > >- Hybrid automatic/manual audit for cruft and code smells. > >- Resolve symlink direction issue for dynamic linker. > >- Affinity/cpuset interfaces. > > So this would be about the "feature complte for > not-internationalized C99" level? It should already be feature-complete for C99, except possibly some bugs in the time stuff I want to overhaul. There are a few things missing for POSIX (some new strftime field width stuff comes to mind), but not much; probably a few more small details for XSI option in the way of interfaces that nobody uses, like encrypt(). And some things missing for C11, of course. > I'll try to get support into aboriginal linux for that release. Great! Yes, this sounds like a good target version for you, especially since I'm hoping to swat a lot more bugs by then (see the hybrid audit item above). It should also give us plenty of time to resolve any issues you encounter using musl with aboriginal between then and musl 1.0 so that everything is really polished when we get to 1.0. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-29 23:50 Proposed roadmap to 1.0 Rich Felker 2013-06-30 2:53 ` Rob Landley @ 2013-06-30 5:20 ` Isaac 2013-06-30 5:34 ` Rich Felker 2013-07-05 15:12 ` Rob Landley 2013-06-30 5:49 ` Strake ` (2 subsequent siblings) 4 siblings, 2 replies; 30+ messages in thread From: Isaac @ 2013-06-30 5:20 UTC (permalink / raw) To: musl On Sat, Jun 29, 2013 at 07:50:41PM -0400, Rich Felker wrote: > Hi all, > > Here is a VERY tentative, proposed roadmap towards a 1.0 release of > musl. Comments welcome! > > 0.9.11 > Projected release: ASAP > No further goals at the moment except fixing additional bugs found. +1 > 0.9.12 > Projected release: Mid to late July > Key targets: > - Overhaul of time handling, including zoneinfo support. > - Overhaul resolver to better provide legacy APIs without code dup. > - Hybrid automatic/manual audit for cruft and code smells. > - Resolve symlink direction issue for dynamic linker. > - Affinity/cpuset interfaces. > 0.9.13 > Projected release: Early August > Key targets: > - Full C++ ABI compatibility with glibc/LSB. > - Support for all remaining iconv charsets of interest (KR/TW/HK). > - Possible overhaul of iconv for performance and clarity/simplicity. > - Possibly add stateful iconv support. > - Establish formal procedure for regression testing. > 0.9.14 > Projected release: End of summer > Key targets: > - Complete documentation draft. > - Performance testing on under-tested archs, fixing bottlenecks hit. > - Review for gratuitous application breakage (anything that could be > fixed with trivial changes that don't hurt musl's quality). > 1.0.0 > Projected release: Early fall > Key targets: > - Polished documentation. > - Organized and coordinated publicity plan. > - At least one new exciting addition to make the release noteworthy, > but which has no chance of breaking things that work. Best candidate > would be one or more new ports, labeled experimental. How about s390 and ia64? ;-) All joking aside, I'd say +1. And for ports, arm64, mips64 or mips n32, x32, and/or sh seem like interesting targets. While sparc is not "dead", basically leon is the only sparc cpu that is alive and likely to provide an interested audience. And that's sparc32. m68k/coldfire are 32-bit only, slow, and largely obsolete with little prospect of new development (Freescale is working on ppc and arm systems), but there is some use of them in the embedded market, so I could imagine a port being useful to someone. Do we currently support 64-bit ppc? ia64 appears to be limited in use/dying, besides not being the ideal target. (big iron, and you'd pretty much need to interest Oracle and similar companies before you get much use). hppa and alpha are most interesting for a computer historian. m32r is live, but I'm not aware of much interest. tilera and epiphany (the Parallela coprocessor) sound interesting, but are likely to be limited in availability. Isaac Dunham ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 5:20 ` Isaac @ 2013-06-30 5:34 ` Rich Felker 2013-06-30 6:42 ` Isaac ` (2 more replies) 2013-07-05 15:12 ` Rob Landley 1 sibling, 3 replies; 30+ messages in thread From: Rich Felker @ 2013-06-30 5:34 UTC (permalink / raw) To: musl On Sat, Jun 29, 2013 at 10:20:45PM -0700, Isaac wrote: > > 1.0.0 > > Projected release: Early fall > > Key targets: > > - Polished documentation. > > - Organized and coordinated publicity plan. > > - At least one new exciting addition to make the release noteworthy, > > but which has no chance of breaking things that work. Best candidate > > would be one or more new ports, labeled experimental. > > How about s390 and ia64? ;-) s390 looks like a maybe. I'm not sufficiently familiar with it to call it a no, and Rob seemed interested in supporting it at one time. ia64 is nothing but gratuitous incompatibility and arch-specific code where it doesn't belong, all for an arch that was dead before it was launched. I think it's officially dead now even, or maybe that's just wishful thinking. > All joking aside, I'd say +1. > And for ports, arm64, mips64 or mips n32, x32, and/or sh seem like > interesting targets. Agreed, all of those look interesting. Super H might be another candidate; IIRC it was used on some game consoles and automotive control computers. > While sparc is not "dead", basically leon is the only sparc cpu that is > alive and likely to provide an interested audience. > And that's sparc32. I don't really know much about sparc except that the register windows system looks ugly. > m68k/coldfire are 32-bit only, slow, and largely obsolete with little > prospect of new development (Freescale is working on ppc and arm systems), > but there is some use of them in the embedded market, so I could imagine a > port being useful to someone. m68k is in some ways an arch I'd like to avoid, but if it's interesting to people we could do it. > Do we currently support 64-bit ppc? No, but 32-bit apps can run on 64-bit kernel as far as I know. I was just looking at the 64-bit ABI earlier today and it's rather gratuitously ugly, but probably not too hard to support. > ia64 appears to be limited in use/dying, besides not being the ideal target. > (big iron, and you'd pretty much need to interest Oracle and similar companies > before you get much use). > hppa and alpha are most interesting for a computer historian. Is hppa the same as pa-risc? If so, it's one I'd definitely like to omit. It's the only machine with a stack that grows upward, so it's a good deal of added generality required for just one arch. And of course like you say it's of interest only to historians. Alpha was kind of interesting, but it's just as dead I think... If I remember right, the kernel support was broken for years and nobody realized... > m32r is live, but I'm not aware of much interest. > tilera and epiphany (the Parallela coprocessor) sound interesting, > but are likely to be limited in availability. Not familiar with them, but my guess would be they're interesting. In embedded, everything has niche uses. On the high-end server side, on the other hand, anything but x86_64 (for straight power) or ARM (for cutting the primary cost of a data center: electricity) is madness. In other words, I think there's a lot more value in supporting diversity on the embedded side than on the enterprise side. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 5:34 ` Rich Felker @ 2013-06-30 6:42 ` Isaac 2013-06-30 7:21 ` Justin Cormack 2013-07-04 18:10 ` Rob Landley 2013-06-30 9:24 ` Rob Landley 2013-06-30 10:45 ` Szabolcs Nagy 2 siblings, 2 replies; 30+ messages in thread From: Isaac @ 2013-06-30 6:42 UTC (permalink / raw) To: musl On Sun, Jun 30, 2013 at 01:34:09AM -0400, Rich Felker wrote: > On Sat, Jun 29, 2013 at 10:20:45PM -0700, Isaac wrote: > > > 1.0.0 > > > Projected release: Early fall > > > Key targets: > > > - Polished documentation. > > > - Organized and coordinated publicity plan. > > > - At least one new exciting addition to make the release noteworthy, > > > but which has no chance of breaking things that work. Best candidate > > > would be one or more new ports, labeled experimental. > > > > How about s390 and ia64? ;-) > > s390 looks like a maybe. I'm not sufficiently familiar with it to call > it a no, and Rob seemed interested in supporting it at one time. > > ia64 is nothing but gratuitous incompatibility and arch-specific code > where it doesn't belong, all for an arch that was dead before it was > launched. I think it's officially dead now even, or maybe that's just > wishful thinking. > > All joking aside, I'd say +1. I mentioned those as the two worst candidates I could think of (hence the "all joking aside"). s390 is a 31-bit (yes, really) architecture; to be precise, it's the discontinued IBM System/390 architecture, which has been replaced with a 64-bit cpu. All along it has been a mainframe platform. > > And for ports, arm64, mips64 or mips n32, x32, and/or sh seem like > > interesting targets. > > Agreed, all of those look interesting. Super H might be another > candidate; IIRC it was used on some game consoles and automotive > control computers. Super H == sh for short. I'm using Debian's port names. > > While sparc is not "dead", basically leon is the only sparc cpu that is > > alive and likely to provide an interested audience. > > And that's sparc32. > > I don't really know much about sparc except that the register windows > system looks ugly. > > > m68k/coldfire are 32-bit only, slow, and largely obsolete with little > > prospect of new development (Freescale is working on ppc and arm systems), > > but there is some use of them in the embedded market, so I could imagine a > > port being useful to someone. > > m68k is in some ways an arch I'd like to avoid, but if it's > interesting to people we could do it. I'm mentioning it as potentially making sense in terms of the usage. > > Do we currently support 64-bit ppc? > > No, but 32-bit apps can run on 64-bit kernel as far as I know. I was > just looking at the 64-bit ABI earlier today and it's rather > gratuitously ugly, but probably not too hard to support. Apparently, it's also slower on some CPUs: http://www.yellowdog-board.com/viewtopic.php?p=23037#p23037 > Is hppa the same as pa-risc? If so, it's one I'd definitely like to > omit. It's the only machine with a stack that grows upward, so it's a Yes. > > m32r is live, but I'm not aware of much interest. > > tilera and epiphany (the Parallela coprocessor) sound interesting, > > but are likely to be limited in availability. > > Not familiar with them, but my guess would be they're interesting. In Tilera Tile: http://www.tilera.com/products/processors In brief, it's a 64-bit processor that comes with up to 100 cores per cpu (last I checked), topping out around 1.6 GHz. Linux is the only OS. Epiphany: http://www.adapteva.com/introduction/ Used in this project: http://www.kickstarter.com/projects/adapteva/parallella-a-supercomputer-for-everyone/ In short, a multicore 32-bit risc cpu currently only used as a coprocessor. Not really a candidate for a port, but if it ever does get a full Linux, it might be interesting. > embedded, everything has niche uses. On the high-end server side, on > the other hand, anything but x86_64 (for straight power) or ARM (for > cutting the primary cost of a data center: electricity) is madness. In > other words, I think there's a lot more value in supporting diversity > on the embedded side than on the enterprise side. Power has a bit of the enterprise, too; it's got lower power usage (vs x86, no comparisons with ARM I'm aware of), and currently holds the highest clock speed of any stock cpu. But I'd somehow expect embedded to be more open to a new libc than enterprise; for the latter, we'd need either a big vendor using musl or a distro (Debian/RHEL/SLES) using it...most likely in place of klibc / newlib / dietlibc, for static rescue tools or for the initrd. Which reminds me... I've been in contact with someone working on a musl package for Debian: https://github.com/wermut/musl/ He's currently looking for a sponsor/mentor, and has packaging that is co-installable with libc6. Isaac Dunham ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 6:42 ` Isaac @ 2013-06-30 7:21 ` Justin Cormack 2013-06-30 12:02 ` Rich Felker 2013-07-04 18:13 ` Rob Landley 2013-07-04 18:10 ` Rob Landley 1 sibling, 2 replies; 30+ messages in thread From: Justin Cormack @ 2013-06-30 7:21 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 5092 bytes --] On 30 Jun 2013 07:43, "Isaac" <idunham@lavabit.com> wrote: > > On Sun, Jun 30, 2013 at 01:34:09AM -0400, Rich Felker wrote: > > On Sat, Jun 29, 2013 at 10:20:45PM -0700, Isaac wrote: > > > > 1.0.0 > > > > Projected release: Early fall > > > > Key targets: > > > > - Polished documentation. > > > > - Organized and coordinated publicity plan. > > > > - At least one new exciting addition to make the release noteworthy, > > > > but which has no chance of breaking things that work. Best candidate > > > > would be one or more new ports, labeled experimental. > > > > > > How about s390 and ia64? ;-) > > > > s390 looks like a maybe. I'm not sufficiently familiar with it to call > > it a no, and Rob seemed interested in supporting it at one time. > > > > ia64 is nothing but gratuitous incompatibility and arch-specific code > > where it doesn't belong, all for an arch that was dead before it was > > launched. I think it's officially dead now even, or maybe that's just > > wishful thinking. > > > > All joking aside, I'd say +1. > > I mentioned those as the two worst candidates I could think of (hence the > "all joking aside"). s390 is a 31-bit (yes, really) architecture; to be > precise, it's the discontinued IBM System/390 architecture, which has been > replaced with a 64-bit cpu. > All along it has been a mainframe platform. > > > > And for ports, arm64, mips64 or mips n32, x32, and/or sh seem like > > > interesting targets. > > > > Agreed, all of those look interesting. Super H might be another > > candidate; IIRC it was used on some game consoles and automotive > > control computers. > > Super H == sh for short. I'm using Debian's port names. > > > > While sparc is not "dead", basically leon is the only sparc cpu that is > > > alive and likely to provide an interested audience. > > > And that's sparc32. > > > > I don't really know much about sparc except that the register windows > > system looks ugly. > > > > > m68k/coldfire are 32-bit only, slow, and largely obsolete with little > > > prospect of new development (Freescale is working on ppc and arm systems), > > > but there is some use of them in the embedded market, so I could imagine a > > > port being useful to someone. > > > > m68k is in some ways an arch I'd like to avoid, but if it's > > interesting to people we could do it. > > I'm mentioning it as potentially making sense in terms of the usage. > > > > Do we currently support 64-bit ppc? > > > > No, but 32-bit apps can run on 64-bit kernel as far as I know. I was > > just looking at the 64-bit ABI earlier today and it's rather > > gratuitously ugly, but probably not too hard to support. > > Apparently, it's also slower on some CPUs: > http://www.yellowdog-board.com/viewtopic.php?p=23037#p23037 I think that may not be true with newer silicon. IBM apparently no longer ship 32 bit userspace compatibility code. You can get access to new hardware at no cost through their partner program. Plus I still have one. MIPS64 would be nice as the Chinese are making them (Loongson) for general use. Justin > > Is hppa the same as pa-risc? If so, it's one I'd definitely like to > > omit. It's the only machine with a stack that grows upward, so it's a > Yes. > > > > m32r is live, but I'm not aware of much interest. > > > tilera and epiphany (the Parallela coprocessor) sound interesting, > > > but are likely to be limited in availability. > > > > Not familiar with them, but my guess would be they're interesting. In > > Tilera Tile: > http://www.tilera.com/products/processors > In brief, it's a 64-bit processor that comes with up to 100 cores per cpu > (last I checked), topping out around 1.6 GHz. Linux is the only OS. > > Epiphany: > http://www.adapteva.com/introduction/ > Used in this project: > http://www.kickstarter.com/projects/adapteva/parallella-a-supercomputer-for-everyone/ > In short, a multicore 32-bit risc cpu currently only used as a coprocessor. > Not really a candidate for a port, but if it ever does get a full Linux, > it might be interesting. > > > embedded, everything has niche uses. On the high-end server side, on > > the other hand, anything but x86_64 (for straight power) or ARM (for > > cutting the primary cost of a data center: electricity) is madness. In > > other words, I think there's a lot more value in supporting diversity > > on the embedded side than on the enterprise side. > > Power has a bit of the enterprise, too; it's got lower power usage (vs > x86, no comparisons with ARM I'm aware of), and currently holds the > highest clock speed of any stock cpu. > But I'd somehow expect embedded to be more open to a new libc than > enterprise; for the latter, we'd need either a big vendor using musl > or a distro (Debian/RHEL/SLES) using it...most likely in place of > klibc / newlib / dietlibc, for static rescue tools or for the initrd. > > Which reminds me... > I've been in contact with someone working on a musl package for Debian: > https://github.com/wermut/musl/ > He's currently looking for a sponsor/mentor, and has packaging that is > co-installable with libc6. > > Isaac Dunham > [-- Attachment #2: Type: text/html, Size: 6853 bytes --] ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 7:21 ` Justin Cormack @ 2013-06-30 12:02 ` Rich Felker 2013-07-04 18:13 ` Rob Landley 1 sibling, 0 replies; 30+ messages in thread From: Rich Felker @ 2013-06-30 12:02 UTC (permalink / raw) To: musl On Sun, Jun 30, 2013 at 08:21:25AM +0100, Justin Cormack wrote: > > > > Do we currently support 64-bit ppc? > > > > > > No, but 32-bit apps can run on 64-bit kernel as far as I know. I was > > > just looking at the 64-bit ABI earlier today and it's rather > > > gratuitously ugly, but probably not too hard to support. > > > > Apparently, it's also slower on some CPUs: > > http://www.yellowdog-board.com/viewtopic.php?p=23037#p23037 > > I think that may not be true with newer silicon. IBM apparently no longer > ship 32 bit userspace compatibility code. Are you talking about AIX or a build of Linux they ship? My guess would be they're just not shipping 32-bit libs, which wouldn't really make any difference if your intent is to use a musl ecosystem instead. If it's 32-bit support turned off in the kernel (is that even possible?), that could also easily be turned back on. > You can get access to new > hardware at no cost through their partner program. Plus I still have one. :) > MIPS64 would be nice as the Chinese are making them (Loongson) for general > use. You can also run 32-bit userspace on MIPS64. However if you want to do this, the n32 ABI (analogous to x32 on x86_64) is moderately more efficient and less ugly. So I agree MIPS n32 and maybe also n64 are interesting to support. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 7:21 ` Justin Cormack 2013-06-30 12:02 ` Rich Felker @ 2013-07-04 18:13 ` Rob Landley 2013-07-07 20:25 ` Isaac 1 sibling, 1 reply; 30+ messages in thread From: Rob Landley @ 2013-07-04 18:13 UTC (permalink / raw) To: musl; +Cc: musl On 06/30/2013 02:21:25 AM, Justin Cormack wrote: > MIPS64 would be nice as the Chinese are making them (Loongson) for > general > use. My mips64 target in aboriginal doesn't work, and hasn't worked for a longish time. Going back and running the old images doesn't work either, so I think qemu changed out from under me and I didn't notice. (Sparc did the same thing but I managed to -cpu "stop that" workaround for that.) Do you know where I can get a mips64 test environment? (Debian iso or something that boots under qemu?) Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-04 18:13 ` Rob Landley @ 2013-07-07 20:25 ` Isaac 0 siblings, 0 replies; 30+ messages in thread From: Isaac @ 2013-07-07 20:25 UTC (permalink / raw) To: musl On Thu, Jul 04, 2013 at 01:13:52PM -0500, Rob Landley wrote: > On 06/30/2013 02:21:25 AM, Justin Cormack wrote: > >MIPS64 would be nice as the Chinese are making them (Loongson) for > >general > >use. > > My mips64 target in aboriginal doesn't work, and hasn't worked for a > longish time. Going back and running the old images doesn't work > either, so I think qemu changed out from under me and I didn't > notice. (Sparc did the same thing but I managed to -cpu "stop that" > workaround for that.) > > Do you know where I can get a mips64 test environment? (Debian iso > or something that boots under qemu?) > Here's Debian's "netinst" CD for big-endian mips: http://cdimage.debian.org/debian-cd/current/mips/iso-cd/debian-7.1.0-mips-netinst.iso And here are Squeeze and Lenny mips images (use -M malta). http://people.debian.org/~aurel32/qemu/mips/ HTH, Isaac Dunham ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 6:42 ` Isaac 2013-06-30 7:21 ` Justin Cormack @ 2013-07-04 18:10 ` Rob Landley 2013-07-07 20:30 ` Isaac 1 sibling, 1 reply; 30+ messages in thread From: Rob Landley @ 2013-07-04 18:10 UTC (permalink / raw) To: musl; +Cc: musl On 06/30/2013 01:42:48 AM, Isaac wrote: > On Sun, Jun 30, 2013 at 01:34:09AM -0400, Rich Felker wrote: > > > m32r is live, but I'm not aware of much interest. > > > tilera and epiphany (the Parallela coprocessor) sound interesting, > > > but are likely to be limited in availability. > > > > Not familiar with them, but my guess would be they're interesting. > In > > Tilera Tile: > http://www.tilera.com/products/processors > In brief, it's a 64-bit processor that comes with up to 100 cores per > cpu > (last I checked), topping out around 1.6 GHz. Linux is the only OS. > > Epiphany: > http://www.adapteva.com/introduction/ > Used in this project: > http://www.kickstarter.com/projects/adapteva/parallella-a-supercomputer-for-everyone/ > In short, a multicore 32-bit risc cpu currently only used as a > coprocessor. > Not really a candidate for a port, but if it ever does get a full > Linux, > it might be interesting. I'm still trying to fluff out http://landley.net/aboriginal/architectures.html and even if I don't describe half the Linux architectures I interrogated git to get a list of when each one was added. Might be useful. > > embedded, everything has niche uses. On the high-end server side, on > > the other hand, anything but x86_64 (for straight power) or ARM (for > > cutting the primary cost of a data center: electricity) is madness. > In > > other words, I think there's a lot more value in supporting > diversity > > on the embedded side than on the enterprise side. > > Power has a bit of the enterprise, too; it's got lower power usage (vs > x86, no comparisons with ARM I'm aware of), and currently holds the > highest clock speed of any stock cpu. Half the game consoles out there are Power. (Dunno about the current generation.) Back when Praystation 3 supported Linux built-in, it was the cheapest way to get a 64 bit PPC system with the Cell processor stuff. (Which alas never took off because it was way too hard to program and they were just proprietary enough in the programming info to prevent open source people from ever quite managing to care. These days Hexagon has all the "3D rendering in software" buzz they used to brag about, but that's problematic to get a test environment working too...) Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-04 18:10 ` Rob Landley @ 2013-07-07 20:30 ` Isaac 0 siblings, 0 replies; 30+ messages in thread From: Isaac @ 2013-07-07 20:30 UTC (permalink / raw) To: musl On Thu, Jul 04, 2013 at 01:10:10PM -0500, Rob Landley wrote: > On 06/30/2013 01:42:48 AM, Isaac wrote: > >On Sun, Jun 30, 2013 at 01:34:09AM -0400, Rich Felker wrote: > >> > m32r is live, but I'm not aware of much interest. > >> > tilera and epiphany (the Parallela coprocessor) sound interesting, > >> > but are likely to be limited in availability. > >> > >> Not familiar with them, but my guess would be they're > >interesting. In > > > >Tilera Tile: > >http://www.tilera.com/products/processors > >In brief, it's a 64-bit processor that comes with up to 100 cores > >per cpu > >(last I checked), topping out around 1.6 GHz. Linux is the only OS. > > > >Epiphany: > >http://www.adapteva.com/introduction/ > >Used in this project: > >http://www.kickstarter.com/projects/adapteva/parallella-a-supercomputer-for-everyone/ > >In short, a multicore 32-bit risc cpu currently only used as a > >coprocessor. > >Not really a candidate for a port, but if it ever does get a full > >Linux, > >it might be interesting. > > I'm still trying to fluff out > http://landley.net/aboriginal/architectures.html and even if I don't > describe half the Linux architectures I interrogated git to get a > list of when each one was added. Might be useful. > > >> embedded, everything has niche uses. On the high-end server side, on > >> the other hand, anything but x86_64 (for straight power) or ARM (for > >> cutting the primary cost of a data center: electricity) is > >madness. In > >> other words, I think there's a lot more value in supporting > >diversity > >> on the embedded side than on the enterprise side. > > > >Power has a bit of the enterprise, too; it's got lower power usage (vs > >x86, no comparisons with ARM I'm aware of), and currently holds the > >highest clock speed of any stock cpu. > > Half the game consoles out there are Power. (Dunno about the current > generation.) Back when Praystation 3 supported Linux built-in, it > was the cheapest way to get a 64 bit PPC system with the Cell > processor stuff. (Which alas never took off because it was way too Last generation was entirely Power (XBox and PS3 were Cell, Nintendo was PPC, I forget who else was relevant), but next generation XBox and PS4 are AMD (64bit x86). Thanks, Isaac Dunham ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 5:34 ` Rich Felker 2013-06-30 6:42 ` Isaac @ 2013-06-30 9:24 ` Rob Landley 2013-06-30 10:45 ` Szabolcs Nagy 2 siblings, 0 replies; 30+ messages in thread From: Rob Landley @ 2013-06-30 9:24 UTC (permalink / raw) To: musl; +Cc: musl On 06/30/2013 12:34:09 AM, Rich Felker wrote: > On Sat, Jun 29, 2013 at 10:20:45PM -0700, Isaac wrote: > > > 1.0.0 > > > Projected release: Early fall > > > Key targets: > > > - Polished documentation. > > > - Organized and coordinated publicity plan. > > > - At least one new exciting addition to make the release > noteworthy, > > > but which has no chance of breaking things that work. Best > candidate > > > would be one or more new ports, labeled experimental. > > > > How about s390 and ia64? ;-) > > s390 looks like a maybe. I'm not sufficiently familiar with it to call > it a no, and Rob seemed interested in supporting it at one time. Yup, qemu has fairly good support for it, klibc supports it, and I got a debian iso to boot under it once (and then be unusable because curses was misconfigured for the console type, but it ran). Between the three I should be able to cobble something up. In general, if klibc, qemu, and my old toolchain support something, I'm interested in getting basic musl support for it. May take a while, though... > ia64 is nothing but gratuitous incompatibility and arch-specific code > where it doesn't belong, all for an arch that was dead before it was > launched. I think it's officially dead now even, or maybe that's just > wishful thinking. I think when SGI finally goes under, it'll be dead. > > All joking aside, I'd say +1. > > And for ports, arm64, mips64 or mips n32, x32, and/or sh seem like > > interesting targets. > > Agreed, all of those look interesting. Super H might be another > candidate; IIRC it was used on some game consoles and automotive > control computers. I just got my sh4 target working again. Had to patch the kernel to stop poking register 0x18 (which makes qemu abort if you read from it, which is just sad)... > > While sparc is not "dead", basically leon is the only sparc cpu > that is > > alive and likely to provide an interested audience. > > And that's sparc32. > > I don't really know much about sparc except that the register windows > system looks ugly. The only sparc target I ever actually got working was sparc32. And the recent openbios upgrade means it hangs as soon as interrupts are enabled under qemu 1.5. But it should be fixable... > > m68k/coldfire are 32-bit only, slow, and largely obsolete with > little > > prospect of new development (Freescale is working on ppc and arm > systems), > > but there is some use of them in the embedded market, so I could > imagine a > > port being useful to someone. > > m68k is in some ways an arch I'd like to avoid, but if it's > interesting to people we could do it. qemu doesn't have decent support for it yet. Upstream qemu does coldfire (a nommu m68k subset). There's a fork that tries to emulate the mmu, but the kernel panics as soon as it tries to enable page tables. I've been poking... Laurent Vivier is it? I'd have to look it up and it's 4am here... It's on the todo list, but that requires qemu upgrades. > > Do we currently support 64-bit ppc? > > No, but 32-bit apps can run on 64-bit kernel as far as I know. I was > just looking at the 64-bit ABI earlier today and it's rather > gratuitously ugly, but probably not too hard to support. All the ppc images I've seen do 32 bit userspace under 64 bit kernels. The only architecture that really seems to make a committment to 64 bit userspace is x86_64, and even they're doing x32... (Not saying don't do it, I just haven't found good existing examples to copy.) > > ia64 appears to be limited in use/dying, besides not being the > ideal target. > > (big iron, and you'd pretty much need to interest Oracle and > similar companies > > before you get much use). > > hppa and alpha are most interesting for a computer historian. > > Is hppa the same as pa-risc? If so, it's one I'd definitely like to > omit. It's the only machine with a stack that grows upward, so it's a > good deal of added generality required for just one arch. And of > course like you say it's of interest only to historians. Hewlett Packard PA Risc. The Puffin project, hp-ux... deadish now, I think. > Alpha was kind of interesting, but it's just as dead I think... If I > remember right, the kernel support was broken for years and nobody > realized... Alpha was left behind when Compaq bought the corpse of DEC. Since Compaq didn't make processors it let the Alpha design team go, and AMD snapped them up and had them design the Athlon, and later the Opteron. (That's why it plugs into the Alpha's ev6 bus.) I consider supporting it about on par with supporting m68k, it's a historically important chip and a reasonably nice design, now obsolete but still well supported by hobbyists who remember it fondly. Unfortunately, it has the same qemu problem that m68k does: application emulation is there but system emulation doesn't do the MMU right. (I'm told this has improved since last I looked...) > > m32r is live, but I'm not aware of much interest. > > tilera and epiphany (the Parallela coprocessor) sound interesting, > > but are likely to be limited in availability. > > Not familiar with them, but my guess would be they're interesting. In > embedded, everything has niche uses. On the high-end server side, on > the other hand, anything but x86_64 (for straight power) or ARM (for > cutting the primary cost of a data center: electricity) is madness. In > other words, I think there's a lot more value in supporting diversity > on the embedded side than on the enterprise side. I've been researching this to update http://landley.net/aboriginal/architectures.html but the research is ongoing. (Bits of that page trail off in midsentence...) My problem is when I get started on computer history I blather at huge length. Still, the "when support was added for each architecture" list might be useful as a frame of reference... > Rich Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 5:34 ` Rich Felker 2013-06-30 6:42 ` Isaac 2013-06-30 9:24 ` Rob Landley @ 2013-06-30 10:45 ` Szabolcs Nagy 2013-06-30 21:29 ` Luca Barbato 2 siblings, 1 reply; 30+ messages in thread From: Szabolcs Nagy @ 2013-06-30 10:45 UTC (permalink / raw) To: musl * Rich Felker <dalias@aerifal.cx> [2013-06-30 01:34:09 -0400]: > On Sat, Jun 29, 2013 at 10:20:45PM -0700, Isaac wrote: > > m32r is live, but I'm not aware of much interest. > > tilera and epiphany (the Parallela coprocessor) sound interesting, > > Not familiar with them, but my guess would be they're interesting. In > embedded, everything has niche uses. On the high-end server side, on > the other hand, anything but x86_64 (for straight power) or ARM (for > cutting the primary cost of a data center: electricity) is madness. In > other words, I think there's a lot more value in supporting diversity > on the embedded side than on the enterprise side. i assume for servers x32 (small address space, but many registers) and aarch64 (low power) can be still relevant ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 10:45 ` Szabolcs Nagy @ 2013-06-30 21:29 ` Luca Barbato 0 siblings, 0 replies; 30+ messages in thread From: Luca Barbato @ 2013-06-30 21:29 UTC (permalink / raw) To: musl On 06/30/2013 12:45 PM, Szabolcs Nagy wrote: > i assume for servers x32 (small address space, but many registers) x32 is sort of underperforming last time it was discussed. > and aarch64 (low power) can be still relevant we could throw in the pot bfin, but I'm not sure if it is worth the hassle. lu ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-30 5:20 ` Isaac 2013-06-30 5:34 ` Rich Felker @ 2013-07-05 15:12 ` Rob Landley 1 sibling, 0 replies; 30+ messages in thread From: Rob Landley @ 2013-07-05 15:12 UTC (permalink / raw) To: musl; +Cc: musl On 06/30/2013 12:20:45 AM, Isaac wrote: > On Sat, Jun 29, 2013 at 07:50:41PM -0400, Rich Felker wrote: > > Hi all, > > > > Here is a VERY tentative, proposed roadmap towards a 1.0 release of > > musl. Comments welcome! > > > > 0.9.11 > > Projected release: ASAP > > No further goals at the moment except fixing additional bugs found. > > +1 > > > 0.9.12 > > Projected release: Mid to late July > > Key targets: > > - Overhaul of time handling, including zoneinfo support. > > - Overhaul resolver to better provide legacy APIs without code dup. > > - Hybrid automatic/manual audit for cruft and code smells. > > - Resolve symlink direction issue for dynamic linker. > > - Affinity/cpuset interfaces. > > > 0.9.13 > > Projected release: Early August > > Key targets: > > - Full C++ ABI compatibility with glibc/LSB. > > - Support for all remaining iconv charsets of interest (KR/TW/HK). > > - Possible overhaul of iconv for performance and clarity/simplicity. > > - Possibly add stateful iconv support. > > - Establish formal procedure for regression testing. > > > 0.9.14 > > Projected release: End of summer > > Key targets: > > - Complete documentation draft. > > - Performance testing on under-tested archs, fixing bottlenecks hit. > > - Review for gratuitous application breakage (anything that could be > > fixed with trivial changes that don't hurt musl's quality). > > > 1.0.0 > > Projected release: Early fall > > Key targets: > > - Polished documentation. > > - Organized and coordinated publicity plan. > > - At least one new exciting addition to make the release noteworthy, > > but which has no chance of breaking things that work. Best > candidate > > would be one or more new ports, labeled experimental. > > How about s390 and ia64? ;-) > > All joking aside, I'd say +1. > And for ports, arm64, mips64 or mips n32, x32, and/or sh seem like > interesting targets. > While sparc is not "dead", basically leon is the only sparc cpu that > is > alive and likely to provide an interested audience. > And that's sparc32. I only ever did sparc32 in aboriginal. (And _just_ got it fixed to work with qemu 1.5.) There's no uClibc support for 64 bit userspace on most non-x86 targets. (There's Alpha, but qemu doesn't provide any board emulations for that, or the MMU-manipulation instructions. Just application emulation.) > m68k/coldfire are 32-bit only, slow, and largely obsolete with little > prospect of new development (Freescale is working on ppc and arm > systems), > but there is some use of them in the embedded market, so I could > imagine a > port being useful to someone. The m68k target in qemu still isn't quite complete. Laurent Vivier's been fluffing it out at git://gitorious.org/qemu-m68k/qemu-m68k -b q800 (the last m68k mac, had a maximum of 256 megs but that's enough to compile stuff natively). But I haven't poked him about it in a while... > Do we currently support 64-bit ppc? > > ia64 appears to be limited in use/dying, besides not being the ideal > target. > (big iron, and you'd pretty much need to interest Oracle and similar > companies > before you get much use). Let ia64 die. > hppa and alpha are most interesting for a computer historian. Which would be me. :) > m32r is live, but I'm not aware of much interest. > tilera and epiphany (the Parallela coprocessor) sound interesting, > but are likely to be limited in availability. Hexagon's in every snapdragon chipset which is basically any Android phone with a qualcomm SOC. It's billed as a multimedia coprocessor, but Linux got ported to it in 2010 and most of the bits are upstream now. There's a linux-hexagon list and everything. The downside is qemu doesn't have support for it, and test systems are hard to come by. (In theory a phone _could_ run vanilla linux. In practice you need an obscureish bootloader and a USB serial driver to talk to the outside world, I've never gotten it set up right. My contract working on it used Comet boards.) Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-29 23:50 Proposed roadmap to 1.0 Rich Felker 2013-06-30 2:53 ` Rob Landley 2013-06-30 5:20 ` Isaac @ 2013-06-30 5:49 ` Strake 2013-07-17 16:02 ` Rich Felker 2013-07-24 14:14 ` orc 4 siblings, 0 replies; 30+ messages in thread From: Strake @ 2013-06-30 5:49 UTC (permalink / raw) To: musl On 29/06/2013, Rich Felker <dalias@aerifal.cx> wrote: > 1.0.0 > Projected release: Early fall > Key targets: > - Polished documentation. > - Organized and coordinated publicity plan. > - At least one new exciting addition to make the release noteworthy, > but which has no chance of breaking things that work. Best candidate > would be one or more new ports, labeled experimental. MMIX! ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-29 23:50 Proposed roadmap to 1.0 Rich Felker ` (2 preceding siblings ...) 2013-06-30 5:49 ` Strake @ 2013-07-17 16:02 ` Rich Felker 2013-07-24 18:36 ` Rob Landley 2013-07-24 14:14 ` orc 4 siblings, 1 reply; 30+ messages in thread From: Rich Felker @ 2013-07-17 16:02 UTC (permalink / raw) To: musl On Sat, Jun 29, 2013 at 07:50:41PM -0400, Rich Felker wrote: > 0.9.11 > Projected release: ASAP > No further goals at the moment except fixing additional bugs found. Done. > 0.9.12 > Projected release: Mid to late July > Key targets: > - Overhaul of time handling, including zoneinfo support. Done, but may need further testing before thinking about release. > - Overhaul resolver to better provide legacy APIs without code dup. I haven't started looking at this again yet. If I don't get to it soon I would not mind pushing it back since I'd rather keep momentum of the release schedule than meet every single goal. But I don't think it's particularly hard either. > - Hybrid automatic/manual audit for cruft and code smells. Any suggestions on techniques to use here? > - Resolve symlink direction issue for dynamic linker. This issue has been around way too long. I really want it fixed for the next release. Any ideas on how to best allow "make install" without permissions or overriding $syslibdir? The idea is I want it to be possible to install a _development_ environment in a user-writable $prefix that generates binaries whose PT_INTERP is the standard /lib/ld-musl-$(ARCH).so.1. This would not be a working runtime environment, of course. If I recall, we had some decent ideas before; maybe I should look back at them. > - Affinity/cpuset interfaces. Last time I started working on this, I got sick of it before I got very far. There are just so many tedious macros/inline-functions to implement. I was also frustrated with having to put so much code in a public header. For this, I'd really like some help on both: - ideas for making it less hideous, and - actually writing it out. With the above in mind, I think we're mostly on schedule for the release roadmap. And we might be partly ahead of schedule on other items: > 0.9.13 > Projected release: Early August > Key targets: > [...] > - Establish formal procedure for regression testing. nsz is working on the test package which looks like it includes some regression testing already. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-17 16:02 ` Rich Felker @ 2013-07-24 18:36 ` Rob Landley 2013-07-24 19:47 ` Rich Felker 2013-07-25 10:29 ` Timo Teras 0 siblings, 2 replies; 30+ messages in thread From: Rob Landley @ 2013-07-24 18:36 UTC (permalink / raw) To: musl; +Cc: musl On 07/17/2013 11:02:05 AM, Rich Felker wrote: > > - Affinity/cpuset interfaces. > > Last time I started working on this, I got sick of it before I got > very far. There are just so many tedious macros/inline-functions to > implement. I was also frustrated with having to put so much code in a > public header. For this, I'd really like some help on both: > > - ideas for making it less hideous, and > - actually writing it out. For toybox I ignored the glibc interfaces and just used the raw syscall, manipulating the arguments myself with bit shifts. Let's see... Wow the man 3 CPU_SET macros are crazy. Very first one, CPU_ZERO() does not specify the size of the array. So how does it determine it? It's gotta be getting it from cpu_set_t which is defined in /usr/include/*/bits/sched.h: /* Data structure to describe CPU mask. */ typedef struct { __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS]; } cpu_set_t; So what's _NCPUBITS? /* Size definition for CPU sets. */ # define __CPU_SETSIZE 1024 # define __NCPUBITS (8 * sizeof (__cpu_mask)) /* Type for array elements in 'cpu_set_t'. */ typedef unsigned long int __cpu_mask; So... it's hardwired to 1024 cpus. I don't think there _is_ a way to make this non-ugly. What actually uses this? Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 18:36 ` Rob Landley @ 2013-07-24 19:47 ` Rich Felker 2013-07-25 7:25 ` Daniel Cegiełka 2013-07-27 5:30 ` Rob Landley 2013-07-25 10:29 ` Timo Teras 1 sibling, 2 replies; 30+ messages in thread From: Rich Felker @ 2013-07-24 19:47 UTC (permalink / raw) To: musl On Wed, Jul 24, 2013 at 01:36:53PM -0500, Rob Landley wrote: > On 07/17/2013 11:02:05 AM, Rich Felker wrote: > >> - Affinity/cpuset interfaces. > > > >Last time I started working on this, I got sick of it before I got > >very far. There are just so many tedious macros/inline-functions to > >implement. I was also frustrated with having to put so much code in a > >public header. For this, I'd really like some help on both: > > > >- ideas for making it less hideous, and > >- actually writing it out. > > For toybox I ignored the glibc interfaces and just used the raw > syscall, manipulating the arguments myself with bit shifts. Let's > see... > > Wow the man 3 CPU_SET macros are crazy. Very first one, CPU_ZERO() > does not specify the size of the array. So how does it determine it? > It's gotta be getting it from cpu_set_t which is defined in > /usr/include/*/bits/sched.h: There's a new set of macros, all ending in _S, which take an additional size argument. However the size is in bytes and it's unclear to me how the caller is supposed to know the required alignment (e.g. making a 3-byte set would not be valid). > /* Data structure to describe CPU mask. */ > typedef struct > { > __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS]; > } cpu_set_t; > > So what's _NCPUBITS? > > /* Size definition for CPU sets. */ > # define __CPU_SETSIZE 1024 > # define __NCPUBITS (8 * sizeof (__cpu_mask)) > > /* Type for array elements in 'cpu_set_t'. */ > typedef unsigned long int __cpu_mask; > > So... it's hardwired to 1024 cpus. > > I don't think there _is_ a way to make this non-ugly. What actually > uses this? That was my question about the whole affinity system in general. My view is that it's stupid micro-management of scheduling that should be done by the kernel, and that if the kernel's not doing a good enough job of managing which cpu a task runs on, the kernel scheduler should be fixed rather than adding hacks in apps. With that said, affinity is useful for debugging certain race conditions that are easier to hit on single-core. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 19:47 ` Rich Felker @ 2013-07-25 7:25 ` Daniel Cegiełka 2013-07-25 7:40 ` Rich Felker 2013-07-27 5:30 ` Rob Landley 1 sibling, 1 reply; 30+ messages in thread From: Daniel Cegiełka @ 2013-07-25 7:25 UTC (permalink / raw) To: musl 2013/7/24 Rich Felker <dalias@aerifal.cx>: > > That was my question about the whole affinity system in general. My > view is that it's stupid micro-management of scheduling that should be > done by the kernel, and that if the kernel's not doing a good enough > job of managing which cpu a task runs on, the kernel scheduler should > be fixed rather than adding hacks in apps. Not always. Sometimes the kernel scheduler isn't the solution and you care about isolating applications from the kernel/os. How do you do it without this feature? http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html musl is a very good candidate for use in HPC, so this functionality would be very valuable. Daniel ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-25 7:25 ` Daniel Cegiełka @ 2013-07-25 7:40 ` Rich Felker 0 siblings, 0 replies; 30+ messages in thread From: Rich Felker @ 2013-07-25 7:40 UTC (permalink / raw) To: musl On Thu, Jul 25, 2013 at 09:25:39AM +0200, Daniel Cegiełka wrote: > 2013/7/24 Rich Felker <dalias@aerifal.cx>: > > > > > That was my question about the whole affinity system in general. My > > view is that it's stupid micro-management of scheduling that should be > > done by the kernel, and that if the kernel's not doing a good enough > > job of managing which cpu a task runs on, the kernel scheduler should > > be fixed rather than adding hacks in apps. > > Not always. Sometimes the kernel scheduler isn't the solution and you > care about isolating applications from the kernel/os. How do you do it > without this feature? > > http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html It's a nice article and I don't have data to argue with its assessment of the current bottlenecks, but I disagree with the philosophy that the solution is to bypass the kernel, in much the same way that I disagree with the philosophy (of APR, glib, gnulib, etc.) that when the libc is broken you should bypass the libc. My philosophy is that if the libc is broken, you should fix the libc, and if the kernel is broken (which it is, in many ways, especially when it comes to having too much bloated abstraction in the wrong places, not enough in the right places, and way too much need for synchronization), then you should fix the kernel. Admittedly that's not an easy task, and I should probably shut up until/unless I'm willing to do it. > musl is a very good candidate for use in HPC, so this functionality > would be very valuable. I don't disagree with that. Despite my dislike for the whole affinity system, I've deemed it important enough to be a big 1.0 roadmap item. :-) Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 19:47 ` Rich Felker 2013-07-25 7:25 ` Daniel Cegiełka @ 2013-07-27 5:30 ` Rob Landley 1 sibling, 0 replies; 30+ messages in thread From: Rob Landley @ 2013-07-27 5:30 UTC (permalink / raw) To: musl; +Cc: musl On 07/24/2013 02:47:19 PM, Rich Felker wrote: > > So... it's hardwired to 1024 cpus. > > > > I don't think there _is_ a way to make this non-ugly. What actually > > uses this? > > That was my question about the whole affinity system in general. My > view is that it's stupid micro-management of scheduling that should be > done by the kernel, and that if the kernel's not doing a good enough > job of managing which cpu a task runs on, the kernel scheduler should > be fixed rather than adding hacks in apps. There are arguments in favor of it, mostly realtime and userspace power management. That said, the only thing that _needs_ to use this stuff is taskset, and everything else can get called _by_ taskset. Toybox implements taskset doesn't use anything out of libc to do so: syscall wrappers and managing its own data structures. Still, you presumably want to build an unmodified util-linux and busybox. The important thing isn't what glibc does, it's what the man pages say. I often start by implementing what the documentation says, building/running real packages, and then submitting bug reports against the docs. The sched_getaffinity() and sched_setaffinity() syscall wrappers seem obvious, and then the _S versions of the macros. Round each size up to the next sizeof(long) bytes. That's pretty much the "should implement this, push patches upstream to make things use 'em" level. man 3 CPU_SET says: CPU_ALLOC() CPU_FREE() CPU_ALLOC_SIZE() // trivial CPU_ZERO_S() CPU_SET_S() CPU_CLR_S() CPU_ISSET_S() CPU_COUNT_S() CPU_AND_S() CPU_OR_S() CPU_XOR_S() CPU_EQUAL_S() That set isn't _that_ disgusting. CPU_ALLOC_SIZE(size) is just (((((size+7)/8)+3)/4)*4 which makes CPU_ALLOC(size) just be malloc(CPU_ALLOC_SIZE(size)) and CPU_FREE(x) is just free(x), and then CPU_ZERO_S(size, set) becomes memset(set, 0, size)... I've posted the set/isset equations here before, and/or/xor are just a for loop where CPU_AND_S(size, dest, src1, src2) is just "do {unsigned __i = size/4; while (__i) {dest[__i] = src1[__i] && src2[__i]; __i--;} } while (0)" or similar, for CPU_COUNT_S you can just cheat and do a bitwise loop with isset (this ain't performance critical)... If you're bored, you could then make the non-S versions can be macros that call the _S versions with a hardwired CPU_SETSIZE argument, and obviously that's #defined to 1024. But "don't use deprecated interfaces" is also a reasonably response. (If ever there's a reason to NOT define stuff unless you #define GNU_DAMMIT it would be the non-S versions of these macros. #ifdef GNU_BRAIN_DAMAGE...) It's evil, but not _that_ evil... Rob ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 18:36 ` Rob Landley 2013-07-24 19:47 ` Rich Felker @ 2013-07-25 10:29 ` Timo Teras 1 sibling, 0 replies; 30+ messages in thread From: Timo Teras @ 2013-07-25 10:29 UTC (permalink / raw) To: musl; +Cc: rob On Wed, 24 Jul 2013 13:36:53 -0500 Rob Landley <rob@landley.net> wrote: > On 07/17/2013 11:02:05 AM, Rich Felker wrote: > > > - Affinity/cpuset interfaces. > > > > Last time I started working on this, I got sick of it before I got > > very far. There are just so many tedious macros/inline-functions to > > implement. I was also frustrated with having to put so much code in > > a public header. For this, I'd really like some help on both: > > > > - ideas for making it less hideous, and > > - actually writing it out. > > For toybox I ignored the glibc interfaces and just used the raw > syscall, manipulating the arguments myself with bit shifts. Let's > see... > > Wow the man 3 CPU_SET macros are crazy. Very first one, CPU_ZERO() > does not specify the size of the array. So how does it determine it? > It's gotta be getting it from cpu_set_t which is defined in > /usr/include/*/bits/sched.h: > > /* Data structure to describe CPU mask. */ > typedef struct > { > __cpu_mask __bits[__CPU_SETSIZE / __NCPUBITS]; > } cpu_set_t; > > So what's _NCPUBITS? > > /* Size definition for CPU sets. */ > # define __CPU_SETSIZE 1024 > # define __NCPUBITS (8 * sizeof (__cpu_mask)) > > /* Type for array elements in 'cpu_set_t'. */ > typedef unsigned long int __cpu_mask; > > So... it's hardwired to 1024 cpus. > > I don't think there _is_ a way to make this non-ugly. What actually > uses this? Seems also many SMP-aware programs use sched_getaffinity to figure out how many threads to launch. Including, but not limited to, util-linux (nproc) and gccgo. Most should autodetect this, but gccgo seems to just unconditionally use it. Speaking of gccgo, it seems that gcc 4.8.1 without any changes does not compile with musl due to: - libgo/mksysinfo.sh: conflicting kernel and libc #includes - libgo/runtime/getncpu-linux.c: uses sched_getaffinity + cpu_set_t - libgo/runtime/proc.c: uses make/swap/getcontext, and needs configure to detect if TLS segment register (fs/gs on x86) is part of the context or not The first two can be patched easily. But lack of *context is a stopper. I think I'll need the *context for few other projects as well... - Timo ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-06-29 23:50 Proposed roadmap to 1.0 Rich Felker ` (3 preceding siblings ...) 2013-07-17 16:02 ` Rich Felker @ 2013-07-24 14:14 ` orc 2013-07-24 14:42 ` Rich Felker 4 siblings, 1 reply; 30+ messages in thread From: orc @ 2013-07-24 14:14 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 2773 bytes --] On Sat, 29 Jun 2013 19:50:41 -0400 Rich Felker <dalias@aerifal.cx> wrote: > Hi all, > > Here is a VERY tentative, proposed roadmap towards a 1.0 release of > musl. Comments welcome! > > Rich > > > > 0.9.11 > Projected release: ASAP > No further goals at the moment except fixing additional bugs found. > > 0.9.12 > Projected release: Mid to late July > Key targets: > - Overhaul of time handling, including zoneinfo support. > - Overhaul resolver to better provide legacy APIs without code dup. > - Hybrid automatic/manual audit for cruft and code smells. > - Resolve symlink direction issue for dynamic linker. > - Affinity/cpuset interfaces. > > 0.9.13 > Projected release: Early August > Key targets: > - Full C++ ABI compatibility with glibc/LSB. > - Support for all remaining iconv charsets of interest (KR/TW/HK). > - Possible overhaul of iconv for performance and clarity/simplicity. > - Possibly add stateful iconv support. > - Establish formal procedure for regression testing. > > 0.9.14 > Projected release: End of summer > Key targets: > - Complete documentation draft. > - Performance testing on under-tested archs, fixing bottlenecks hit. > - Review for gratuitous application breakage (anything that could be > fixed with trivial changes that don't hurt musl's quality). > > 1.0.0 > Projected release: Early fall > Key targets: > - Polished documentation. > - Organized and coordinated publicity plan. > - At least one new exciting addition to make the release noteworthy, > but which has no chance of breaking things that work. Best candidate > would be one or more new ports, labeled experimental. Hi Rich, (This probably should go into 1.0 wishlist thread, but this one freshier) While building a file server/router I found some bugs/incompatibilities in getaddrinfo() and getifaddrs(). getaddrinfo() does not reports IPv6 available when asked with AF_UNSPEC. Thus, servers like openssh or tinc (vpn daemon) still bind only IPv4 socket when they configured to bind IPv4 and IPv6 sockets. getifaddrs() does not returns AF_PACKET like glibc does, so list of all system interfaces is incomplete (does not shows inactive interfaces). I attached patch for getaddrinfo() (adopt it if you need it) and test program. I still have no any clues about getifaddrs(), but that is not critical. It is usually implemented with help of netlink (and possibly there is no other way, maybe some /proc file will give a list) musl-git (from 0.9.10 release). Unfortunately your git interface and website were down at the time this email written and I cannot see any changes in getaddrinfo.c from that version now. I checked https://github.com/idunham/musl/blob/master/src/network/getaddrinfo.c, it seems to be same. If I am missed something, let me know. [-- Attachment #2: gai-test.c --] [-- Type: application/octet-stream, Size: 710 bytes --] #include <stdio.h> #include <sys/types.h> #include <sys/socket.h> #include <netdb.h> #include <netinet/in.h> int main(void) { struct addrinfo *ai, *aip, hint = {0}; int err; char hostn[256], portn[10]; hint.ai_family = AF_UNSPEC; hint.ai_socktype = SOCK_STREAM; hint.ai_protocol = IPPROTO_TCP; hint.ai_flags = AI_PASSIVE; err = getaddrinfo(NULL, "22", &hint, &ai); if (err || !ai) return 1; for (aip = ai; aip; aip = aip->ai_next) { err = getnameinfo(aip->ai_addr, aip->ai_addrlen, hostn, sizeof(hostn), portn, sizeof(portn), NI_NUMERICHOST|NI_NUMERICSERV); if (err) { printf("%p: invalid\n", aip); continue; } printf("%p: %s:%s\n", aip, hostn, portn); } freeaddrinfo(ai); return 0; } [-- Attachment #3: musl-0.9.10git-getaddrinfo-ipv4-ipv6.patch --] [-- Type: application/octet-stream, Size: 1561 bytes --] --- musl/src/network/getaddrinfo.c.orig +++ musl/src/network/getaddrinfo.c @@ -100,23 +100,31 @@ } if (!host) { - if (family == AF_UNSPEC) family = AF_INET; - buf = calloc(sizeof *buf, 1+EXTRA); + int unspec = 0, idx = 0; + if (family == AF_UNSPEC) { unspec = 1; family = AF_INET; } + buf = calloc(sizeof *buf, (unspec ? 2 : 1)+EXTRA); if (!buf) return EAI_MEMORY; - buf->ai.ai_protocol = proto; - buf->ai.ai_socktype = type; - buf->ai.ai_addr = (void *)&buf->sa; - buf->ai.ai_addrlen = family==AF_INET6 ? sizeof sa.sin6 : sizeof sa.sin; - buf->ai.ai_family = family; - buf->sa.sin.sin_family = family; - buf->sa.sin.sin_port = port; +_next6: + (buf+idx)->ai.ai_protocol = proto; + (buf+idx)->ai.ai_socktype = type; + (buf+idx)->ai.ai_addr = (void *)&(buf+idx)->sa; + (buf+idx)->ai.ai_addrlen = family==AF_INET6 ? sizeof sa.sin6 : sizeof sa.sin; + (buf+idx)->ai.ai_family = family; + (buf+idx)->sa.sin.sin_family = family; + (buf+idx)->sa.sin.sin_port = port; + if (unspec) (buf+idx)->ai.ai_next = &(buf+1)->ai; if (!(flags & AI_PASSIVE)) { if (family == AF_INET) { - 0[(uint8_t*)&buf->sa.sin.sin_addr.s_addr]=127; - 3[(uint8_t*)&buf->sa.sin.sin_addr.s_addr]=1; - } else buf[0].sa.sin6.sin6_addr.s6_addr[15] = 1; + 0[(uint8_t*)&(buf+idx)->sa.sin.sin_addr.s_addr]=127; + 3[(uint8_t*)&(buf+idx)->sa.sin.sin_addr.s_addr]=1; + } else (buf+idx)[0].sa.sin6.sin6_addr.s6_addr[15] = 1; } *res = &buf->ai; + if (unspec) { + idx = 1; unspec = 0; + family = AF_INET6; + goto _next6; + } return 0; } ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 14:14 ` orc @ 2013-07-24 14:42 ` Rich Felker 2013-07-24 15:29 ` orc 0 siblings, 1 reply; 30+ messages in thread From: Rich Felker @ 2013-07-24 14:42 UTC (permalink / raw) To: musl On Wed, Jul 24, 2013 at 10:14:10PM +0800, orc wrote: > While building a file server/router I found some bugs/incompatibilities > in getaddrinfo() and getifaddrs(). > > getaddrinfo() does not reports IPv6 available when asked with > AF_UNSPEC. Thus, servers like openssh or tinc (vpn daemon) still bind > only IPv4 socket when they configured to bind IPv4 and IPv6 sockets. Indeed, this is an oversight. > getifaddrs() does not returns AF_PACKET like glibc does, so list of > all system interfaces is incomplete (does not shows inactive > interfaces). Is there a use case you want this for? I remember when we added getifaddrs this was discussed, and I was hesitant to add AF_PACKET because it's using some deprecated version of some structure where the fields are too small to store the values they're supposed to represent. I'd have to look through the mailing list and/or IRC logs to recall the details, though. I'm not entirely opposed to it if there's a serious need, but at the time it seemed like a poorly designed interface. > I attached patch for getaddrinfo() (adopt it if you need it) > and test program. I think it could be cleaner/simpler but I might just commit it as-is for now and wait to clean it up until the getaddrinfo cleanup/overhaul which was scheduled for this release cycle but will get pushed back to the next. > I still have no any clues about getifaddrs(), but > that is not critical. It is usually implemented with help of netlink > (and possibly there is no other way, maybe some /proc file will give a > list) musl's is entirely based on /proc because the netlink stuff is undocumented, seemed really bloated, and I didn't want to risk pulling in code that was inadvertently derived from GNU code. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 14:42 ` Rich Felker @ 2013-07-24 15:29 ` orc 2013-07-24 16:04 ` Rich Felker 0 siblings, 1 reply; 30+ messages in thread From: orc @ 2013-07-24 15:29 UTC (permalink / raw) To: musl On Wed, 24 Jul 2013 10:42:45 -0400 Rich Felker <dalias@aerifal.cx> wrote: > On Wed, Jul 24, 2013 at 10:14:10PM +0800, orc wrote: > > While building a file server/router I found some > > bugs/incompatibilities in getaddrinfo() and getifaddrs(). > > > > getaddrinfo() does not reports IPv6 available when asked with > > AF_UNSPEC. Thus, servers like openssh or tinc (vpn daemon) still > > bind only IPv4 socket when they configured to bind IPv4 and IPv6 > > sockets. > > Indeed, this is an oversight. > > > getifaddrs() does not returns AF_PACKET like glibc does, so list of > > all system interfaces is incomplete (does not shows inactive > > interfaces). > > Is there a use case you want this for? I remember when we added > getifaddrs this was discussed, and I was hesitant to add AF_PACKET > because it's using some deprecated version of some structure where the > fields are too small to store the values they're supposed to > represent. I'd have to look through the mailing list and/or IRC logs > to recall the details, though. I'm not entirely opposed to it if > there's a serious need, but at the time it seemed like a poorly > designed interface. I faced problem when I tried to build dhcpcd with musl. I dropped dhcpcd then, found simple default.script for udhcpc and forgot about any problems. > > > I attached patch for getaddrinfo() (adopt it if you need it) > > and test program. > > I think it could be cleaner/simpler but I might just commit it as-is > for now and wait to clean it up until the getaddrinfo cleanup/overhaul > which was scheduled for this release cycle but will get pushed back to > the next. Thanks for review. I think I can apply it now on server :) > > > I still have no any clues about getifaddrs(), but > > that is not critical. It is usually implemented with help of netlink > > (and possibly there is no other way, maybe some /proc file will > > give a list) > > musl's is entirely based on /proc because the netlink stuff is > undocumented, seemed really bloated, and I didn't want to risk pulling > in code that was inadvertently derived from GNU code. I tried already netlink and dropped it since udhcpc works nicely. I really agree that it is too opaque. > > Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 15:29 ` orc @ 2013-07-24 16:04 ` Rich Felker 2013-07-24 16:25 ` orc 0 siblings, 1 reply; 30+ messages in thread From: Rich Felker @ 2013-07-24 16:04 UTC (permalink / raw) To: musl On Wed, Jul 24, 2013 at 11:29:42PM +0800, orc wrote: > > > getifaddrs() does not returns AF_PACKET like glibc does, so list of > > > all system interfaces is incomplete (does not shows inactive > > > interfaces). > > > > Is there a use case you want this for? I remember when we added > > getifaddrs this was discussed, and I was hesitant to add AF_PACKET > > because it's using some deprecated version of some structure where the > > fields are too small to store the values they're supposed to > > represent. I'd have to look through the mailing list and/or IRC logs > > to recall the details, though. I'm not entirely opposed to it if > > there's a serious need, but at the time it seemed like a poorly > > designed interface. > > I faced problem when I tried to build dhcpcd with musl. I > dropped dhcpcd then, found simple default.script for udhcpc and forgot > about any problems. Do you know what dhcpcd needs it for? If it's just automatic binding when you don't specify an interface, that's probably a bad idea anyway... But maybe we should support it. > > > I attached patch for getaddrinfo() (adopt it if you need it) > > > and test program. > > > > I think it could be cleaner/simpler but I might just commit it as-is > > for now and wait to clean it up until the getaddrinfo cleanup/overhaul > > which was scheduled for this release cycle but will get pushed back to > > the next. > > Thanks for review. I think I can apply it now on server :) I didn't really "review" it, but as long as you tested it, it's probably fine. There are only 3 possible code paths here, not an infinite family of them, so as long as each of the 3 works for you it should be fine. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 16:04 ` Rich Felker @ 2013-07-24 16:25 ` orc 2013-07-24 19:41 ` Rich Felker 0 siblings, 1 reply; 30+ messages in thread From: orc @ 2013-07-24 16:25 UTC (permalink / raw) To: musl On Wed, 24 Jul 2013 12:04:09 -0400 Rich Felker <dalias@aerifal.cx> wrote: > On Wed, Jul 24, 2013 at 11:29:42PM +0800, orc wrote: > > > > getifaddrs() does not returns AF_PACKET like glibc does, so > > > > list of all system interfaces is incomplete (does not shows > > > > inactive interfaces). > > > > > > Is there a use case you want this for? I remember when we added > > > getifaddrs this was discussed, and I was hesitant to add AF_PACKET > > > because it's using some deprecated version of some structure > > > where the fields are too small to store the values they're > > > supposed to represent. I'd have to look through the mailing list > > > and/or IRC logs to recall the details, though. I'm not entirely > > > opposed to it if there's a serious need, but at the time it > > > seemed like a poorly designed interface. > > > > I faced problem when I tried to build dhcpcd with musl. I > > dropped dhcpcd then, found simple default.script for udhcpc and > > forgot about any problems. > > Do you know what dhcpcd needs it for? If it's just automatic binding > when you don't specify an interface, that's probably a bad idea > anyway... But maybe we should support it. It listed interfaces internally, compared command line argument with each, and if not found, exited with error. An interface of question was down at the moment, so was not listed by getifaddrs(). > > > > > I attached patch for getaddrinfo() (adopt it if you need it) > > > > and test program. > > > > > > I think it could be cleaner/simpler but I might just commit it > > > as-is for now and wait to clean it up until the getaddrinfo > > > cleanup/overhaul which was scheduled for this release cycle but > > > will get pushed back to the next. > > > > Thanks for review. I think I can apply it now on server :) > > I didn't really "review" it, but as long as you tested it, it's > probably fine. There are only 3 possible code paths here, not an > infinite family of them, so as long as each of the 3 works for you > it should be fine. OK, just to be sure. I tested it in 3 possible ways, anything works nice. > > Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
* Re: Proposed roadmap to 1.0 2013-07-24 16:25 ` orc @ 2013-07-24 19:41 ` Rich Felker 0 siblings, 0 replies; 30+ messages in thread From: Rich Felker @ 2013-07-24 19:41 UTC (permalink / raw) To: musl On Thu, Jul 25, 2013 at 12:25:06AM +0800, orc wrote: > > > I faced problem when I tried to build dhcpcd with musl. I > > > dropped dhcpcd then, found simple default.script for udhcpc and > > > forgot about any problems. > > > > Do you know what dhcpcd needs it for? If it's just automatic binding > > when you don't specify an interface, that's probably a bad idea > > anyway... But maybe we should support it. > > It listed interfaces internally, compared command line argument with > each, and if not found, exited with error. An interface of question was > down at the moment, so was not listed by getifaddrs(). OK, this is just gratuitous brokenness/non-portability. if_nametoindex can be used to check whether an interface name is valid, and if_nameindex gives you a list of all interfaces. Rich ^ permalink raw reply [flat|nested] 30+ messages in thread
end of thread, other threads:[~2013-07-27 5:30 UTC | newest] Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-06-29 23:50 Proposed roadmap to 1.0 Rich Felker 2013-06-30 2:53 ` Rob Landley 2013-06-30 3:01 ` Rich Felker 2013-06-30 5:20 ` Isaac 2013-06-30 5:34 ` Rich Felker 2013-06-30 6:42 ` Isaac 2013-06-30 7:21 ` Justin Cormack 2013-06-30 12:02 ` Rich Felker 2013-07-04 18:13 ` Rob Landley 2013-07-07 20:25 ` Isaac 2013-07-04 18:10 ` Rob Landley 2013-07-07 20:30 ` Isaac 2013-06-30 9:24 ` Rob Landley 2013-06-30 10:45 ` Szabolcs Nagy 2013-06-30 21:29 ` Luca Barbato 2013-07-05 15:12 ` Rob Landley 2013-06-30 5:49 ` Strake 2013-07-17 16:02 ` Rich Felker 2013-07-24 18:36 ` Rob Landley 2013-07-24 19:47 ` Rich Felker 2013-07-25 7:25 ` Daniel Cegiełka 2013-07-25 7:40 ` Rich Felker 2013-07-27 5:30 ` Rob Landley 2013-07-25 10:29 ` Timo Teras 2013-07-24 14:14 ` orc 2013-07-24 14:42 ` Rich Felker 2013-07-24 15:29 ` orc 2013-07-24 16:04 ` Rich Felker 2013-07-24 16:25 ` orc 2013-07-24 19:41 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).