Revisiting 1.0 wishlist

Revisiting 1.0 wishlist

[Rich Felker]

About 18 months ago, shortly after the initial public release of musl,
I posted a wishlist of what I thought it would take to consider musl
to have reached "1.0" quality. The other day I was revisiting that,
and thought I'd post a summary of my thoughts to the list as a
starting point for further discussion.

Completed goals:
- coverage for all of C99 and POSIX base and supported option groups
- character class handling sync'd to current Unicode
- dynamic loading (except in static-linked apps)
- C++ support

In addition, significant progress has been made on the open-ended
goals of application compatibility and ability to load/run some
glibc-linked binaries (applications and libraries). Part of the goal
originally stated in the wishlist was to determine a collection of
"important" applications and ensure compatibility against them. I
think now would be a good time to start doing that. Perhaps LFS (Linux
From Scratch) might make a good base set to start with, especially
since lots of people building their own systems who might use musl
will be starting with LFS as a guide. We could add and remove some
packages from the list as desired.

Aside from yet-to-be-defined compatibility goals, the only thing
missing from musl that was in the original 1.0 wishlist is
documentation.

There is also one other goal I introduced later, on which I think 1.0
needs to depend: support for a to-be-determined set of additional
legacy character encodings in iconv. At the very least, the major
legacy encodings for Korean and Traditional Chinese should be
included, and it may also be desirable to add support for stateful
encodings (ISO-2022). Aside from these, I believe all encodings
important for supporting legacy data on the web, in email, etc. are
supported.

That's about it for now. As usual, comments are welcome.

Rich

Re: Revisiting 1.0 wishlist

[Isaac Dunham]

On Tue, 20 Nov 2012 00:09:40 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> In addition, significant progress has been made on the open-ended
> goals of application compatibility and ability to load/run some
> glibc-linked binaries (applications and libraries). Part of the goal
For what it's worth:
Although I can run ksh93 with a patched musl (see the g_hack branch* if you're curious), it crashes on executing any external command.
I have yet to track down whether this is an issue with the patches, UB, or ABI incompatability in musl.
> originally stated in the wishlist was to determine a collection of
> "important" applications and ensure compatibility against them. I
> think now would be a good time to start doing that. Perhaps LFS (Linux
> From Scratch) might make a good base set to start with, especially
> since lots of people building their own systems who might use musl
> will be starting with LFS as a guide. We could add and remove some
> packages from the list as desired.
From the LFS list, I'm not sure about DejaGnu, texinfo, tcl (I think some versions work, but don't know about all of them), sysklogd, expect, or systemd (I think LFS wants that for udev); sysvinit works as expected apart from who/related utils, plus previous runlevel changes aren't known (utmp).
Beyond LFS, I'd suggest:
-Xorg, with Intel/Radeon/Nouveau/modesetting/vesa/fbdev drivers.
These *should* work at present, as well as GTK2.
-Python is too widely used on Linux for incompatability to be easily overlooked.  I know sabotage builds it, are patches required?
-A full LAMP stack or similar (LAMP currently means upstreaming anything needed for Apache/PHP to build, and getting MySQL or MariaDB working...PostgreSQL is the DB I'd want, though ;))

Longer-term, making sure Wine, Java, Qt, GTK3, and Webkit work is probably going to be important for relevance. I don't really like most of them, but that's where the action is.
> Aside from yet-to-be-defined compatibility goals, the only thing
> missing from musl that was in the original 1.0 wishlist is
> documentation.
> 
> There is also one other goal I introduced later, on which I think 1.0
> needs to depend: support for a to-be-determined set of additional
> legacy character encodings in iconv. At the very least, the major
> legacy encodings for Korean and Traditional Chinese should be
> included, and it may also be desirable to add support for stateful
> encodings (ISO-2022). Aside from these, I believe all encodings
> important for supporting legacy data on the web, in email, etc. are
> supported.


-- 
Isaac Dunham <idunham@lavabit.com>

Re: Revisiting 1.0 wishlist

[Luca Barbato]

On 11/20/12 6:09 AM, Rich Felker wrote:
> About 18 months ago, shortly after the initial public release of musl,
> I posted a wishlist of what I thought it would take to consider musl
> to have reached "1.0" quality. The other day I was revisiting that,
> and thought I'd post a summary of my thoughts to the list as a
> starting point for further discussion.
>
> Completed goals:
> - coverage for all of C99 and POSIX base and supported option groups
> - character class handling sync'd to current Unicode
> - dynamic loading (except in static-linked apps)
> - C++ support

> In addition, significant progress has been made on the open-ended
> goals of application compatibility and ability to load/run some
> glibc-linked binaries (applications and libraries). Part of the goal
> originally stated in the wishlist was to determine a collection of
> "important" applications and ensure compatibility against them. I
> think now would be a good time to start doing that. Perhaps LFS (Linux
>  From Scratch) might make a good base set to start with, especially
> since lots of people building their own systems who might use musl
> will be starting with LFS as a guide. We could add and remove some
> packages from the list as desired.

Some people asked me recently about adding musl to Gentoo properly (as 
in `crossdev musl` and musl stage3) I'll review my old ebuild and look 
again at the few issues that prevented crossdev to work completely soon,
probably I'll have to ask you questions, surely would be nice if the 
result would let us have a Gentoo/musl 1.0 =)

lu

Re: Revisiting 1.0 wishlist

[Roy]

Rich Felker <dalias <at> aerifal.cx> writes:

> support for a to-be-determined set of additional
> legacy character encodings in iconv. At the very
> least, the major legacy encodings for Korean and
> Traditional Chinese should be included, and it 
> may also be desirable to add support for stateful
> encodings (ISO-2022). Aside from these, I believe
> all encodings important for supporting legacy data
> on the web, in email, etc. are supported.

Does it include EBCDIC, especially the stateful(SI-SO)
 Double-Byte Character Set(DBCS) EBCDICs?
The glibc gconv module support DBCS EBCDIC which GNU
 libiconv does not support.
(I asked libiconv devs and they told me they do 
not have intention to support them).

Re: Re: Revisiting 1.0 wishlist

[Rich Felker]

On Tue, Nov 20, 2012 at 01:40:01PM +0000, Roy wrote:
> Rich Felker <dalias <at> aerifal.cx> writes:
> 
> > support for a to-be-determined set of additional
> > legacy character encodings in iconv. At the very
> > least, the major legacy encodings for Korean and
> > Traditional Chinese should be included, and it 
> > may also be desirable to add support for stateful
> > encodings (ISO-2022). Aside from these, I believe
> > all encodings important for supporting legacy data
> > on the web, in email, etc. are supported.
> 
> Does it include EBCDIC, especially the stateful(SI-SO)
>  Double-Byte Character Set(DBCS) EBCDICs?
> The glibc gconv module support DBCS EBCDIC which GNU
>  libiconv does not support.
> (I asked libiconv devs and they told me they do 
> not have intention to support them).

At present, musl's iconv does not support any stateful encodings (the
iconv_t it not actually a pointer to any state; it's just a bitfield
storing the source and dest encodings directly), nor does it support
EBCDIC at all. In general I believe stateful encodings may be relevant
(ISO-2022-JP used to be the main Japanese encoding used on IRC; is
that still true?) but I'm unsure whether even EBCDIC itself (much less
derived DBCS's) has modern relevance with respect to iconv. I think
the important question to ask is: "Is there any chance of an
application which uses iconv to support receiving data in varied
character encodings receiving data as EBCDIC?" My guess is no, but if
you know usage cases I'll listen.

Rich

Re: Revisiting 1.0 wishlist

[Roy]

Rich Felker <dalias <at> aerifal.cx> writes:
> At present, musl's iconv does not support any stateful encodings (the
> iconv_t it not actually a pointer to any state; it's just a bitfield
> storing the source and dest encodings directly), nor does it support
> EBCDIC at all. In general I believe stateful encodings may be relevant
> (ISO-2022-JP used to be the main Japanese encoding used on IRC; is
> that still true?) but I'm unsure whether even EBCDIC itself (much less
> derived DBCS's) has modern relevance with respect to iconv. I think
> the important question to ask is: "Is there any chance of an
> application which uses iconv to support receiving data in varied
> character encodings receiving data as EBCDIC?" My guess is no, but if
> you know usage cases I'll listen.
From what I heard from users and coworkers, they backup data from
AIX mainframe without converting, and sometimes they need to view
backup data without connecting to mainframe, and they pipe iconv
output to pager for the purpose.
IIRC the Japanese use UTF-8 for newly created IRC networks, for
e-mail encoding, ISO-2022-JP, Shift-JIS, EUC-JP, and UTF-8 are
widely used.