documenting musl

documenting musl

[Rich Felker]

Hi all,

One item that's been on my todo list for a while, but didn't really
make it into the last email, is creating some kind of documentation
for musl. 

One document, which I would call simply the "manual", should cover
everything you need to know if you're writing applications against
musl or building existing applications against musl. It should contain
introductory materials on the standards and extensions musl aims to
support, feature test macros, etc. It should also document musl's
locale behavior, quality of implementation guarantees, and anything
ISO C or POSIX requires an implementation to document (i.e.
implementation-defined behavior). In some cases, the documentation may
defer to that for the compiler being used. The manual should further
document any additional behavior guarantees for functions beyond
what's required in the standard, as well as behavior for all supported
non-standard functions.

A second possible document would be oriented towards people wanting to
learn from the source, port musl to new platforms, add features or
customize it for unusual usage cases, reuse parts of musl in other
software, etc. This document would explain the source tree layout and
build system, use of weak symbols and object file dependency graph
issues, stdio and pthread internal interfaces including thread
cancellation architecture, porting requirements, and so on.

For getting the documentation done, and ideal situation would be for a
FOSS documentation guru with an interest in musl to show up and
volunteer to organize the documentation effort, write major portions,
and maintain it. In the absence of such a miracle, perhaps we could
turn the above ramblings into a sort of outline for the proposed
documentation, and use the wiki to draft unofficial documents that
cover some of the same topics. I think the wiki would be especially
appropriate for the developer/hacking documentation, since it's less
official in nature and more community-oriented.

Ideas? Volunteers?

Rich

Re: documenting musl

[Isaac Dunham]

[-- Attachment #1: Type: text/plain, Size: 2056 bytes --]

On Fri, 7 Sep 2012 22:40:06 -0400
Rich Felker <dalias@aerifal.cx> wrote:

> Hi all,
> 
> One item that's been on my todo list for a while, but didn't really
> make it into the last email, is creating some kind of documentation

> One document, which I would call simply the "manual", should cover
> everything you need to know if you're writing applications against
> musl or building existing applications against musl. It should contain
> introductory materials on the standards and extensions musl aims to
> support, feature test macros, etc. It should also document musl's
Just wrote up a preliminary draft for this part.
> locale behavior, quality of implementation guarantees, and anything

I'm not certain about these, myself.
> ISO C or POSIX requires an implementation to document (i.e.
> implementation-defined behavior). In some cases, the documentation may
> defer to that for the compiler being used. The manual should further
> document any additional behavior guarantees for functions beyond
> what's required in the standard, as well as behavior for all supported
> non-standard functions.
> 
> A second possible document would be oriented towards people wanting to
> learn from the source, port musl to new platforms, add features or

start with the "Porting" docs?

> customize it for unusual usage cases, reuse parts of musl in other

I presume this includes (for example) builds without src/complex/ and
stripping it down for embedded use? I've been doing a few experiments
in that way.

> software, etc. This document would explain the source tree layout and
> build system, use of weak symbols and object file dependency graph
> issues, stdio and pthread internal interfaces including thread
> cancellation architecture, porting requirements, and so on.

> Ideas? Volunteers?

I can do some of this.
I'm thinking about turning some of these into man pages.
eg, musl-standards, musl-features, musl-porting, musl-implementation...
Also, manpages for functions not covered by linux-manpages-dev (fgetln,
strl*, ...) _might_ be in order.

[-- Attachment #2: musl-man.txt --]
[-- Type: text/plain, Size: 2199 bytes --]

The unofficial musl manual

Feature test macros, standards:
Musl supports the following standards, with the given feature test macros or parameters:
X/Open 2008 (SUS4)	_XOPEN_SOURCE   >= 700
POSIX 2008		_POSIX_C_SOURCE >= 200809L, _POSIX_SOURCE
ISO C99			(requires -std=c99 or equivalent; detected using __STRICT_ANSI__, which gcc sets automatically)

The following standards are partly supported:
ISO C11			(WIP, threads and atomics aren't supported yet, and 
			Annex K is unlikely to be supported in the foreseeable 
			future; use -std=c11 or equivalent)
ANSI C89/ISO C90	A few details are incompatible with C99, and the 
			namespace respected is the C99 one.  However, there 
			shouldn't be breakage.
Older POSIX and SUS	_POSIX_C_SOURCE < 200809L, _XOPEN_SOURCE < 700
			Exposes legacy extensions as well as the modern 
			standard, rather than only the legacy standard.

The following feature test macros are also supported to the extent practical:
_GNU_SOURCE		Partial glibc compatability.
			Nonstandard functions with the same names as standard 
			ones (basename and several others) ARE NOT SUPPORTED:
			musl uses the ISO/POSIX/SUS definition unconditionally.
_LARGEFILE64_SOURCE	nop: musl only provides 64bit file io, so this just adds
			*64 aliases to the namespace
_BSD_SOURCE		While glibc is still stuck at BSD 4.4, musl provides
			much of the functionality available via _NETBSD_SOURCE
			on NetBSD. This includes strlcpy, strlcat, and fgetln.

By default (-std=gnu* or no arguments for GCC), musl currently sets 
_XOPEN_SOURCE to 700 and defines _BSD_SOURCE.
With musl 0.9.4 or older, musl defaulted to ANSI C99.

Detecting musl:
This is frequently asked about. It is not possible to detect musl for sure.
Key differences:
__linux and __ELF will both be defined, unlike with dietlibc.
unistd.h defines _XOPEN_VERSION to 700 and _POSIX_VERSION to 200809L. 
(This is what you should be checking for, unless you need nonstandard functions)
features.h defaults to defining _BSD_SOURCE and _XOPEN_SOURCE, rather than _BSD_SOURCE and _SVID_SOURCE; __USE_* macros are not present, nor are the __GLIBC* macros.
Using GNU basename syntax ( basename("/usr"); ) will make for segfaults.

Re: documenting musl

[Daniel Cegiełka]

btw. documentation - mandoc is a better option than (old/big/ugly)
groff for documentation.

http://www.undeadly.org/cgi?action=article&sid=20100604082319&mode=expanded

http://2009.asiabsdcon.org/papers/abc2009-P6B-paper.pdf

http://www.youtube.com/watch?v=z25FFo01Zpw

Daniel

Re: documenting musl

[Kurt H Maier]

On Sat, Sep 08, 2012 at 10:44:03AM +0200, Daniel Cegiełka wrote:
> btw. documentation - mandoc is a better option than (old/big/ugly)
> groff for documentation.

None of those links explain why they seem to think it's either groff or
roll-your-own, when there are plenty of lightweight roff
implementations.  Any hints?

Re: documenting musl

[Daniel Cegiełka]

2012/9/8 Kurt H Maier <khm-lists@intma.in>:
> On Sat, Sep 08, 2012 at 10:44:03AM +0200, Daniel Cegiełka wrote:
>> btw. documentation - mandoc is a better option than (old/big/ugly)
>> groff for documentation.
>
> None of those links explain why they seem to think it's either groff or
> roll-your-own, when there are plenty of lightweight roff
> implementations.  Any hints?

hmm... mandoc isn't lightweight? :)

# du -h | grep image/
20.0K	./image/usr/share/man/man1
8.0K	./image/usr/share/man/man8
32.0K	./image/usr/share/man
36.0K	./image/usr/share
248.0K	./image/usr/bin
288.0K	./image/usr
# ls -lh image/usr/bin/
total 244
lrwxrwxrwx    1 root     root           6 Sep  8 12:24 apropos -> mandoc
lrwxrwxrwx    1 root     root           6 Sep  8 12:24 makewhatis -> mandoc
-rwxr-xr-x    1 root     root      243.9K Sep  8 12:24 mandoc
lrwxrwxrwx    1 root     root           6 Sep  8 12:24 mandocdb -> mandoc
lrwxrwxrwx    1 root     root           6 Sep  8 12:24 whatis -> mandoc

compared to groff (gentoo):

(cut...)

7.1M	./image/usr/share
4.0K	./image/usr/lib64/groff/site-tmac
60K	./image/usr/lib64/groff/groffer
68K	./image/usr/lib64/groff
72K	./image/usr/lib64
3.4M	./image/usr/bin
11M	./image/usr
11M	./image

Daniel

Re: documenting musl

[Rich Felker]

On Sat, Sep 08, 2012 at 02:35:22PM +0200, Daniel Cegiełka wrote:
> 2012/9/8 Kurt H Maier <khm-lists@intma.in>:
> > On Sat, Sep 08, 2012 at 10:44:03AM +0200, Daniel Cegiełka wrote:
> >> btw. documentation - mandoc is a better option than (old/big/ugly)
> >> groff for documentation.
> >
> > None of those links explain why they seem to think it's either groff or
> > roll-your-own, when there are plenty of lightweight roff
> > implementations.  Any hints?
> 
> hmm... mandoc isn't lightweight? :)
> 
> # du -h | grep image/
> 20.0K	./image/usr/share/man/man1
> 8.0K	./image/usr/share/man/man8
> 32.0K	./image/usr/share/man
> 36.0K	./image/usr/share
> 248.0K	./image/usr/bin
> 288.0K	./image/usr
> # ls -lh image/usr/bin/
> total 244
> lrwxrwxrwx    1 root     root           6 Sep  8 12:24 apropos -> mandoc
> lrwxrwxrwx    1 root     root           6 Sep  8 12:24 makewhatis -> mandoc
> -rwxr-xr-x    1 root     root      243.9K Sep  8 12:24 mandoc
> lrwxrwxrwx    1 root     root           6 Sep  8 12:24 mandocdb -> mandoc
> lrwxrwxrwx    1 root     root           6 Sep  8 12:24 whatis -> mandoc
> 
> compared to groff (gentoo):
> 
> (cut...)
> 
> 7.1M	./image/usr/share
> 4.0K	./image/usr/lib64/groff/site-tmac
> 60K	./image/usr/lib64/groff/groffer
> 68K	./image/usr/lib64/groff
> 72K	./image/usr/lib64
> 3.4M	./image/usr/bin
> 11M	./image/usr
> 11M	./image

mandoc may be interesting, but I think the whole mandoc vs groff thing
is a distraction from the real topic. groff is a really ugly (GNU,
C++) version of the standard troff, of which there's a perfectly good
version based on the original troff somewhere, perhaps in heirloom.

At some point we may want man pages for various things, but the
documentation I started this thread to consider is something that
would probably be desirable to have in html and pdf output formats;
the input format is open not something I've thought too much about.

The content is the important part, and content may eventually be
derived/adapted into man pages too.. :)

Rich

Re: documenting musl

[Daniel Cegiełka]

2012/9/8 Rich Felker <dalias@aerifal.cx>:

> At some point we may want man pages for various things, but the
> documentation I started this thread to consider is something that
> would probably be desirable to have in html and pdf output formats;
> the input format is open not something I've thought too much about.

Exactly, and I recommend mandoc, because it's all supported.
'Output Formats' section:

http://mdocml.bsd.lv/mandoc.1.html

> The content is the important part, and content may eventually be
> derived/adapted into man pages too.. :)
>
> Rich

Re: documenting musl

[Kurt H Maier]

On Sat, Sep 08, 2012 at 02:35:22PM +0200, Daniel Cegiełka wrote:
> 2012/9/8 Kurt H Maier <khm-lists@intma.in>:
> > On Sat, Sep 08, 2012 at 10:44:03AM +0200, Daniel Cegiełka wrote:
> >> btw. documentation - mandoc is a better option than (old/big/ugly)
> >> groff for documentation.
> >
> > None of those links explain why they seem to think it's either groff or
> > roll-your-own, when there are plenty of lightweight roff
> > implementations.  Any hints?
> 
> hmm... mandoc isn't lightweight? :)
> 
<snip>
> 
> compared to groff (gentoo):
>
<snip>
> 
> Daniel

Now you're doing it. There are dozens of other roff implementations,
such as the one that comes with 9base, or the heirloom doctools, etc.
Why does everyone who uses mandoc ignore everything but groff?

Re: documenting musl

[Daniel Cegiełka]

2012/9/8 Kurt H Maier <khm-lists@intma.in>:
>> Daniel
>
> Now you're doing it. There are dozens of other roff implementations,
> such as the one that comes with 9base, or the heirloom doctools, etc.
> Why does everyone who uses mandoc ignore everything but groff?

# pwd
/tmp/9base-307800d4a01f
# head -n 3 troff/Makefile
# mk - mk unix port from plan9
# Depends on ../lib9

# make
9base build options:
CFLAGS   = -c -I. -DPLAN9PORT -DPREFIX="/usr/local/plan9" -D_GNU_SOURCE
LDFLAGS  = -static
CC       = cc
make[1]: Entering directory `/tmp/9base-307800d4a01f/lib9'
CC dirread.c
dirread.c: In function 'mygetdents':
dirread.c:33:2: error: implicit declaration of function 'getdirentries'
make[1]: *** [dirread.o] Error 1
make[1]: Leaving directory `/tmp/9base-307800d4a01f/lib9'
make: *** [all] Error 2

lib9? I don't know how lib9 is compatible with the musl...

regards,
Daniel

Re: documenting musl

[Kurt H Maier]

On Sat, Sep 08, 2012 at 03:12:04PM +0200, Daniel Cegiełka wrote:
> 
> lib9? I don't know how lib9 is compatible with the musl...
> 

Ok.  If you promise to stop pasting random terminal shit, I promise to
remember you don't care about anything at all except mandoc.

Re: documenting musl

[Daniel Cegiełka]

2012/9/8 Kurt H Maier <khm-lists@intma.in>:
>> lib9? I don't know how lib9 is compatible with the musl...
>
> Ok.  If you promise to stop pasting random terminal shit, I promise to
> remember you don't care about anything at all except mandoc.

ok.
correct me, but it lib9 is a libc implementation on Plan9 OS. I
understand that you (suckless.org) port it into Linux. Maybe it's
better to use musl instead lib9? I presented my opinion and I
understand that you have a different opinion, but please, why do you
think it would be a better solution than mandoc etc.? mandoc, like
musl, is a new and fresh solution with supports modern expectations
for the documentation system. Why old troff will be better?

best regards,
Daniel

Re: documenting musl

[Kurt H Maier]

On Sat, Sep 08, 2012 at 03:42:59PM +0200, Daniel Cegiełka wrote:
> think it would be a better solution than mandoc etc.? mandoc, like
> musl, is a new and fresh solution with supports modern expectations
> for the documentation system. Why old troff will be better?

mandoc is a program written with the sole aim of replacing groff.  groff
is overengineered and full of useless things and bad licensing.  I get
it.  However, mandoc is a one-trick pony, not suitable for general
typesetting.  why on earth would you have one command for typesetting,
and another command for typesetting in a subset of the first program's
input format?  it's silly.

this is a common paradigm in computing:  over time, a solution becomes
unpleasant.  A new tool is made to resolve the unpleasantness, but the
new tool takes for granted other unpleasant factors, which are then
standardized into the system with the acceptance of the new tool.

mandoc does nothing that roff wouldn't do before, and roff will do
anything mandoc will do.

but rich is right, this is all pointless.  we'll wind up using the doc
tools preferred by the person who actually writes the documentation, as
it should be.

Re: documenting musl

[John Spencer]

On 09/08/2012 03:12 PM, Daniel Cegiełka wrote:
> make[1]: Entering directory `/tmp/9base-307800d4a01f/lib9'
> CC dirread.c
> dirread.c: In function 'mygetdents':
> dirread.c:33:2: error: implicit declaration of function 'getdirentries'
> make[1]: *** [dirread.o] Error 1
> make[1]: Leaving directory `/tmp/9base-307800d4a01f/lib9'
> make: *** [all] Error 2
>
> lib9? I don't know how lib9 is compatible with the musl...
https://raw.github.com/rofl0r/sabotage/master/pkg/9base

Re: documenting musl

[Isaac Dunham]

On Sat, 8 Sep 2012 08:47:23 -0400
Kurt H Maier <khm-lists@intma.in> wrote:

> On Sat, Sep 08, 2012 at 02:35:22PM +0200, Daniel Cegiełka wrote:
> > 2012/9/8 Kurt H Maier <khm-lists@intma.in>:
> > > On Sat, Sep 08, 2012 at 10:44:03AM +0200, Daniel Cegiełka wrote:
> > >> btw. documentation - mandoc is a better option than
> > >> (old/big/ugly) groff for documentation.
> > >
> > > None of those links explain why they seem to think it's either
> > > groff or roll-your-own, when there are plenty of lightweight roff
> > > implementations.  Any hints?
> > 
> > hmm... mandoc isn't lightweight? :)
> > 

No, but it's not roff.
Yes, I use roff to write regular documents.
AFAICT if you want any old-format or portable manpages to work, you
may need groff anyhow (unsupported macros make it fallback to groff)


> > 
> > Daniel
> 
> Now you're doing it. There are dozens of other roff implementations,
> such as the one that comes with 9base, or the heirloom doctools, etc.
> Why does everyone who uses mandoc ignore everything but groff?

I'm using heirloom troff myself, and if I write the docs to use *roff
& kin, they will work with heirloom troff (though I _don't_ plan to
make it specific to heirloom-doctools).
Now that may require a little bit of hacking on their mandoc support
if you want mandoc format.  Several mandoc format pages, such as pcc.1,
don't work properly there.

Re: documenting musl

[Isaac Dunham]

Hello everyone,
I'm glad to see all the helpful discussion about the proper source
format for documentation. :P

Any comments on the _documentation_, as opposed to what format I'm
thinking about putting it into when I think it's ready?
(quoting it did mess up the text formatting, fyi: see the attachment to
my first post if you care.)
> The unofficial musl manual
> 
> Feature test macros, standards:
> Musl supports the following standards, with the given feature test
> macros or parameters: X/Open 2008 (SUS4)	_XOPEN_SOURCE   >= 700
> POSIX 2008		_POSIX_C_SOURCE >= 200809L, _POSIX_SOURCE
> ISO C99			(requires -std=c99 or equivalent;
> detected using __STRICT_ANSI__, which gcc sets automatically)
> 
> The following standards are partly supported:
> ISO C11			(WIP, threads and atomics aren't
> supported yet, and Annex K is unlikely to be supported in the
> foreseeable future; use -std=c11 or equivalent)
> ANSI C89/ISO C90	A few details are incompatible with C99, and
> the namespace respected is the C99 one.  However, there 
> 			shouldn't be breakage.
> Older POSIX and SUS	_POSIX_C_SOURCE < 200809L, _XOPEN_SOURCE <
> 700 Exposes legacy extensions as well as the modern 
> 			standard, rather than only the legacy
> standard.
> 
> The following feature test macros are also supported to the extent
> practical: _GNU_SOURCE		Partial glibc compatability.
> 			Nonstandard functions with the same names as
> standard ones (basename and several others) ARE NOT SUPPORTED:
> 			musl uses the ISO/POSIX/SUS definition
> unconditionally. _LARGEFILE64_SOURCE	nop: musl only provides
> 64bit file io, so this just adds *64 aliases to the namespace
> _BSD_SOURCE		While glibc is still stuck at BSD 4.4,
> musl provides much of the functionality available via _NETBSD_SOURCE
> 			on NetBSD. This includes strlcpy, strlcat,
> and fgetln.
> 
> By default (-std=gnu* or no arguments for GCC), musl currently sets 
> _XOPEN_SOURCE to 700 and defines _BSD_SOURCE.
> With musl 0.9.4 or older, musl defaulted to ANSI C99.
> 
> Detecting musl:
> This is frequently asked about. It is not possible to detect musl for
> sure. Key differences:
> __linux and __ELF will both be defined, unlike with dietlibc.
> unistd.h defines _XOPEN_VERSION to 700 and _POSIX_VERSION to 200809L. 
> (This is what you should be checking for, unless you need nonstandard
> functions) features.h defaults to defining _BSD_SOURCE and
> _XOPEN_SOURCE, rather than _BSD_SOURCE and _SVID_SOURCE; __USE_*
> macros are not present, nor are the __GLIBC* macros. Using GNU
> basename syntax ( basename("/usr"); ) will make for segfaults.
> 
> 

Re: documenting musl

[Szabolcs Nagy]

* Isaac Dunham <idunham@lavabit.com> [2012-09-07 23:36:58 -0700]:
> The unofficial musl manual
> 
> Feature test macros, standards:
> Musl supports the following standards, with the given feature test macros or parameters:
> X/Open 2008 (SUS4)	_XOPEN_SOURCE   >= 700
> POSIX 2008		_POSIX_C_SOURCE >= 200809L, _POSIX_SOURCE
> ISO C99			(requires -std=c99 or equivalent; detected using __STRICT_ANSI__, which gcc sets automatically)
> 

i might not use the proper terminology here
but the following needs to be made clear:

there is two kinds of "musl supports..":
- the environment in which musl can be built (see INSTALL document)
- the environment musl provides (this is what we document here)

the environment musl provides includes:
- run-time and abi support (eg. binaries built against glibc may work..)
- compile-time environment (with appropriate compiler flags)

> The following standards are partly supported:
> ISO C11			(WIP, threads and atomics aren't supported yet, and 
> 			Annex K is unlikely to be supported in the foreseeable 
> 			future; use -std=c11 or equivalent)
> ANSI C89/ISO C90	A few details are incompatible with C99, and the 
> 			namespace respected is the C99 one.  However, there 
> 			shouldn't be breakage.

(+ISO C94/AMD1)

c89 compatibility may worth some investigation

(i'd guess that only math.h breaks c89 compatibility seriously with
union compound literals and probably that could be worked around,

the c99 namespace and library changes and long long for int64
types are minor incompatibilities

for c89 code __STRICT_ANSI__ and __STDC_VERSION__<199901L is nedded
(same is true for c94 conformance which is c89 + the wchar_t mess)

actually even the c99 support is only true with appropriate
compiler extensions, eg. NAN cannot be properly defined using
c99 constructs, you need __builtin_nanf("") but in these
cases musl provides reasonable fallbacks)

> Older POSIX and SUS	_POSIX_C_SOURCE < 200809L, _XOPEN_SOURCE < 700
> 			Exposes legacy extensions as well as the modern 
> 			standard, rather than only the legacy standard.
> 
> The following feature test macros are also supported to the extent practical:
> _GNU_SOURCE		Partial glibc compatability.
> 			Nonstandard functions with the same names as standard 
> 			ones (basename and several others) ARE NOT SUPPORTED:
> 			musl uses the ISO/POSIX/SUS definition unconditionally.
> _LARGEFILE64_SOURCE	nop: musl only provides 64bit file io, so this just adds
> 			*64 aliases to the namespace

it's not a nop then

> _BSD_SOURCE		While glibc is still stuck at BSD 4.4, musl provides
> 			much of the functionality available via _NETBSD_SOURCE
> 			on NetBSD. This includes strlcpy, strlcat, and fgetln.
> 

> By default (-std=gnu* or no arguments for GCC), musl currently sets 
> _XOPEN_SOURCE to 700 and defines _BSD_SOURCE.
> With musl 0.9.4 or older, musl defaulted to ANSI C99.
> 

i wouldn't write this in gcc specific ways

"if the above feature test macros are not defined musl sets.."

> Detecting musl:
> This is frequently asked about. It is not possible to detect musl for sure.
> Key differences:
> __linux and __ELF will both be defined, unlike with dietlibc.
> unistd.h defines _XOPEN_VERSION to 700 and _POSIX_VERSION to 200809L. 
> (This is what you should be checking for, unless you need nonstandard functions)
> features.h defaults to defining _BSD_SOURCE and _XOPEN_SOURCE, rather than _BSD_SOURCE and _SVID_SOURCE; __USE_* macros are not present, nor are the __GLIBC* macros.
> Using GNU basename syntax ( basename("/usr"); ) will make for segfaults.
> 

i guess documenting all the implementation defined
behaviour, quality of implementation guarantees
and locale things will be the hard part

Re: documenting musl

[Justin Cormack]

On Sat, Sep 8, 2012 at 3:40 AM, Rich Felker <dalias@aerifal.cx> wrote:
> Hi all,
>
> One item that's been on my todo list for a while, but didn't really
> make it into the last email, is creating some kind of documentation
> for musl.
>
> One document, which I would call simply the "manual", should cover
> everything you need to know if you're writing applications against
> musl or building existing applications against musl. It should contain
> introductory materials on the standards and extensions musl aims to
> support, feature test macros, etc. It should also document musl's
> locale behavior, quality of implementation guarantees, and anything
> ISO C or POSIX requires an implementation to document (i.e.
> implementation-defined behavior). In some cases, the documentation may
> defer to that for the compiler being used. The manual should further
> document any additional behavior guarantees for functions beyond
> what's required in the standard, as well as behavior for all supported
> non-standard functions.
>
> A second possible document would be oriented towards people wanting to
> learn from the source, port musl to new platforms, add features or
> customize it for unusual usage cases, reuse parts of musl in other
> software, etc. This document would explain the source tree layout and
> build system, use of weak symbols and object file dependency graph
> issues, stdio and pthread internal interfaces including thread
> cancellation architecture, porting requirements, and so on.
>
> For getting the documentation done, and ideal situation would be for a
> FOSS documentation guru with an interest in musl to show up and
> volunteer to organize the documentation effort, write major portions,
> and maintain it. In the absence of such a miracle, perhaps we could
> turn the above ramblings into a sort of outline for the proposed
> documentation, and use the wiki to draft unofficial documents that
> cover some of the same topics. I think the wiki would be especially
> appropriate for the developer/hacking documentation, since it's less
> official in nature and more community-oriented.
>
> Ideas? Volunteers?

Sounds good. Happy to help. Have you got a model for what form it
might end up in? I find that it helps to know how it might end up
looking in order to get an idea of how much to write, and who the
audience is.

For example, do we want to produce a set of man pages for a
distribution that uses Musl at some point? Should we aim to produce
everything as an online reference first?

Maybe some examples will help to discuss what we want.

I think for the comprehensive documentation the wiki might be a pain.

Justin

Re: documenting musl

[Isaac Dunham]

On Sat, 8 Sep 2012 12:44:33 +0100
Justin Cormack <justin@specialbusservice.com> wrote:
> Sounds good. Happy to help. Have you got a model for what form it
> might end up in? I find that it helps to know how it might end up
> looking in order to get an idea of how much to write, and who the
> audience is.

Plan for mostly programmers/distro developers, with some regular
users who want to get something building.
At this point I'm thinking write plaintext, and we can add formatting
later.

> For example, do we want to produce a set of man pages for a
> distribution that uses Musl at some point? Should we aim to produce
> everything as an online reference first?

Most input formats can be converted to HTML. And that includes *roff.

linux-manpages-dev is documenting most linux libc versions as well as
portability issues, so it might make sense to just prepare patches that
will document where musl differs. But I'd suggest holding off until
0.9.5 is released to submit them, since there will be a few differences
in header behavior.
AFAICT, blowfish crypt, fgetln, strlcpy, and strlcat are the main
functions that stock glibc doesn't have. Manpages for those may be
helpful, since there aren't manpages for them.

> Maybe some examples will help to discuss what we want.

Perhaps look at the existing document on porting?
I think it's floating around on the list, and dates to ~0.9.3 (when
mips was merged).

> I think for the comprehensive documentation the wiki might be a pain.
> 
> Justin

Re: documenting musl

[nwmcsween]

IMO documentation should be inline with code, I've banged my head on the wall countless times reading musl source wrt linuxisms, etc. a good inline doc style is Rocco style literate inline documentation / Donald Knuth style literate documentation. I also doubt any normal inline documentation has any measurable compiler overhead.

On Sep 7, 2012, at 7:40 PM, Rich Felker <dalias@aerifal.cx> wrote:

> Hi all,
> 
> One item that's been on my todo list for a while, but didn't really
> make it into the last email, is creating some kind of documentation
> for musl. 
> 
> One document, which I would call simply the "manual", should cover
> everything you need to know if you're writing applications against
> musl or building existing applications against musl. It should contain
> introductory materials on the standards and extensions musl aims to
> support, feature test macros, etc. It should also document musl's
> locale behavior, quality of implementation guarantees, and anything
> ISO C or POSIX requires an implementation to document (i.e.
> implementation-defined behavior). In some cases, the documentation may
> defer to that for the compiler being used. The manual should further
> document any additional behavior guarantees for functions beyond
> what's required in the standard, as well as behavior for all supported
> non-standard functions.
> 
> A second possible document would be oriented towards people wanting to
> learn from the source, port musl to new platforms, add features or
> customize it for unusual usage cases, reuse parts of musl in other
> software, etc. This document would explain the source tree layout and
> build system, use of weak symbols and object file dependency graph
> issues, stdio and pthread internal interfaces including thread
> cancellation architecture, porting requirements, and so on.
> 
> For getting the documentation done, and ideal situation would be for a
> FOSS documentation guru with an interest in musl to show up and
> volunteer to organize the documentation effort, write major portions,
> and maintain it. In the absence of such a miracle, perhaps we could
> turn the above ramblings into a sort of outline for the proposed
> documentation, and use the wiki to draft unofficial documents that
> cover some of the same topics. I think the wiki would be especially
> appropriate for the developer/hacking documentation, since it's less
> official in nature and more community-oriented.
> 
> Ideas? Volunteers?
> 
> Rich

Re: documenting musl

[Rich Felker]

On Sat, Sep 08, 2012 at 08:09:51PM -0700, nwmcsween@gmail.com wrote:
> IMO documentation should be inline with code, I've banged my head on
> the wall countless times reading musl source wrt linuxisms, etc. a
> good inline doc style is Rocco style literate inline documentation /
> Donald Knuth style literate documentation. I also doubt any normal
> inline documentation has any measurable compiler overhead.

I acknowledge this would have benefits for understanding how the code
works (the hackers' manual), but it's not a substitute for a
standalone manual for somebody who's not trying to understand the code
but just wants to use it. If we were not implementing standards (real
and de facto ones) but instead newly designed interfaces, it might
make sense to have interface contract documentation for library users
inline with the code (or headers) and generate standalone docs from
there. But documenting all the outward behavior of every function in
libc is really orthogonal to musl, and POSIX has already done a fine
job of most of it...

Rich

Re: documenting musl

[nwmcsween]

Posix only describes what it must do not how it does it as musl vs glibc vs any other libc in my opinion anyways
On Sep 8, 2012, at 9:15 PM, Rich Felker <dalias@aerifal.cx> wrote:

> On Sat, Sep 08, 2012 at 08:09:51PM -0700, nwmcsween@gmail.com wrote:
>> IMO documentation should be inline with code, I've banged my head on
>> the wall countless times reading musl source wrt linuxisms, etc. a
>> good inline doc style is Rocco style literate inline documentation /
>> Donald Knuth style literate documentation. I also doubt any normal
>> inline documentation has any measurable compiler overhead.
> 
> I acknowledge this would have benefits for understanding how the code
> works (the hackers' manual), but it's not a substitute for a
> standalone manual for somebody who's not trying to understand the code
> but just wants to use it. If we were not implementing standards (real
> and de facto ones) but instead newly designed interfaces, it might
> make sense to have interface contract documentation for library users
> inline with the code (or headers) and generate standalone docs from
> there. But documenting all the outward behavior of every function in
> libc is really orthogonal to musl, and POSIX has already done a fine
> job of most of it...
> 
> Rich

Re: documenting musl

[Rich Felker]

On Sat, Sep 08, 2012 at 09:15:47PM -0700, nwmcsween@gmail.com wrote:
> Posix only describes what it must do not how it does it as musl vs
> glibc vs any other libc in my opinion anyways

My point was that the what vs how is an important distinction, and I
don't think it's useful for us to duplicate the "what", which POSIX
already did very well. The how is of course interesting to anybody
reading the code, and inline comments may be a good place to put that.

Rich

Re: documenting musl

[Justin Cormack]

On Sun, Sep 9, 2012 at 5:15 AM, Rich Felker <dalias@aerifal.cx> wrote:
> On Sat, Sep 08, 2012 at 08:09:51PM -0700, nwmcsween@gmail.com wrote:
>> IMO documentation should be inline with code, I've banged my head on
>> the wall countless times reading musl source wrt linuxisms, etc. a
>> good inline doc style is Rocco style literate inline documentation /
>> Donald Knuth style literate documentation. I also doubt any normal
>> inline documentation has any measurable compiler overhead.
>
> I acknowledge this would have benefits for understanding how the code
> works (the hackers' manual), but it's not a substitute for a
> standalone manual for somebody who's not trying to understand the code
> but just wants to use it. If we were not implementing standards (real
> and de facto ones) but instead newly designed interfaces, it might
> make sense to have interface contract documentation for library users
> inline with the code (or headers) and generate standalone docs from
> there. But documenting all the outward behavior of every function in
> libc is really orthogonal to musl, and POSIX has already done a fine
> job of most of it...

I assumed he meant writing the docs inline but then pulling them out
to generate the standalone docs.

eg the best place to write the note that dup2 will retry if it gets
EBUSY, which needs to go on the man page so the user knows that she
does not need to write the code to retry instead would be in dup2.c,
so it gets maintained if that race condition goes away one day (say,
just picking an imaginary example). A script can then go and pull this
into the man page generation. Its a device to keep the docs up to date
partly.

I think though that because of the relatively fixed scope of a libc
and the fact that the update process is disciplined it is less of an
issue, and anyway the main thing is to write the docs for now.

Justin