* Initial draft of musl documentation/manual @ 2013-09-01 4:41 Rich Felker 2013-09-01 9:43 ` Luca Barbato ` (4 more replies) 0 siblings, 5 replies; 19+ messages in thread From: Rich Felker @ 2013-09-01 4:41 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1011 bytes --] Hi all, The big agenda item for the next release is drafting the official documentation for musl. While this task still has a long way to go, and much of it is still a skeleton, there's also a decent amount of actual content, and it's helping me come up with ideas for other content that should be included. Comments on the draft are welcome, especially on the matter of additional topics that readers might expect to find which I have overlooked.. For reference, it's based on the outline in the "New docs outline" thread on the mailing list, from 2012-11-15. One more thing: the format of the document is presently Markdown (http://daringfireball.net/projects/markdown/). I may switch to something else in the future, but Markdown tends to be my favorite from a standpoint of being fully readable as plain text and also producing decent formatted output in HTML (and other formats, via conversion from HTML). Comments on this bikeshed topic are also welcome, I suppose, unless it gets out of hand... Rich [-- Attachment #2: manual4.txt --] [-- Type: text/plain, Size: 5576 bytes --] musl ==== Part I - About musl ------------------- ### Introduction and Scope musl is an implementation of the userspace portion of the standard library functionality described in the ISO C and POSIX standards, plus common extensions. It is both a component for use in Linux-based operating systems and a tool for building application binaries deployable on a wide range of Linux-based systems and non-Linux systems which can provide a compatible syscall API layer. ### Conformance statements The interfaces in musl are modeled upon and intended to conform to the requirements of the ISO C99 standard (ISO/IEC 9899-1999), including Annex F, and POSIX 2008 / Single Unix Standard Version 4, with all current technical corrigenda applied. However, musl has not been certified by any standards body, and no guarantee of conformance is made by the copyright holders or any other party with an interest in musl. Moreover, since musl provides only the userspace portion of the standard system interfaces, conformance to the requirements of POSIX depends in part on the behavior of the underlying kernel. Linux 2.6.39 or later is believed to be sufficient; earlier versions in the 2.6 series will work, but with varying degrees of non-conformance, particularly in the area of signal handling behavior and close-on-exec race conditions. Likewise, conformance to the requirements of ISO C, and especially Annex F (IEEE floating point semantics), depends in part on both the compiler used to build musl and the compiler used when building applications against musl. At this time there is no known fully conforming compiler. ### Supported Systems To be written. ### Build and Installation To be written. ### Installed Components #### Dynamic linking runtime `$(syslibdir)/ld-musl-$(ARCH).so.1` provides the dynamic linker, or "program interpreter", for dynamically linked ELF programs using musl. The absolute pathname to this file must be stored in all such programs. The build and installation system provided with musl sets it up as a symbolic link to `$(libdir)/libc.so`, but system integrators may choose to make it available in whichever ways they find suitable. #### Development environment Header files for use by the C compiler are installed in `$(includedir)`. The standard headers are fully self-contained, and do not make use of kernel-provided or compiler-provided headers or otherwise require such headers to be present. The file `libc.a` installed in `$(libdir)` provides the entire standard library implementation for static linking. The file `libc.so` provides the linker with access to the standard library's symbols for use at link-time in producing dynamic-linked binaries. It is not searched at runtime; the standard library is resolved as part of the program interpreter at `$(syslibdir)/ld-musl-$(ARCH).so.1`. Additional files `libm.a`, `librt.a`, `libpthread.a`, `libcrypt.a`, `libutil.a`, `libxnet.a`, `libresolv.a`, and `libdl.a` are provided in `$(libdir)` as empty library archives. They contain no code, but are present to satisfy the POSIX requirement that options of the form `-lm`, `-lpthread`, etc. be accepted by the `c99` compiler. Several bare object files are also included in `$(libdir)`: `crt1.o` and `Scrt1.o` are the normal and position-independent versions, respectively, of the entry point code linked into every program. `crti.o` and `crtn.o`, also linked into every program and into shared libraries, provide support for legacy means by which the compiler can arrange for global constructors and destructors to be executed. It is possible to setup a legacy-free compiler toolchain that does not need the `crti.o` and `crtn.o` files if desired. #### Compiler wrapper To be written. ### Filesystem Layout Dependencies musl aims to avoid imposing filesystem policy; however, the following minimal set of filesystems dependencies must be met in order for programs using musl to function correctly: * `/dev/null` - required by POSIX * `/tmp` - required by POSIX to exist as a directory, and used by various temporary file creation functions. * `/dev/shm` - must be a directory, and should have permissions 01777. If absent, POSIX shared memory and named semaphore interfaces will fail; programs not using these features will be unaffected. * `/proc` - must be a mount point for Linux procfs or a symlink to such. Several functions such as realpath, fexecve, and a number of the "at" functions added in POSIX 2008 need access to /proc to function correctly. * `/etc/resolv.conf` - needed to provide addresses of nameservers to be used for DNS lookups, unless a working nameserver is available on the loopback address. * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the "program interpreter" specified in the program's headers - if present, this will be processed as a text file containing the shared library search path, with components delimited by newlines or colons. If absent, a default path of `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by static-linked programs. Part II - Usage --------------- To be written. This part of the manual will deal with documenting implementation-defined behavior and further behaviors that are not required to be documented but for which musl makes additional guarantees. Part III - Implementation ------------------------- To be written. This part of the manual will document the implementation of musl, including matters such as source tree layout, built system, algorithms used, musl-internal APIs, coding style, and information on porting. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-01 4:41 Initial draft of musl documentation/manual Rich Felker @ 2013-09-01 9:43 ` Luca Barbato 2013-09-01 16:57 ` Rich Felker 2013-09-01 13:45 ` Vasily Kulikov ` (3 subsequent siblings) 4 siblings, 1 reply; 19+ messages in thread From: Luca Barbato @ 2013-09-01 9:43 UTC (permalink / raw) To: musl On 01/09/13 06:41, Rich Felker wrote: > One more thing: the format of the document is presently Markdown > (http://daringfireball.net/projects/markdown/). I may switch to > something else in the future, but Markdown tends to be my favorite > from a standpoint of being fully readable as plain text and also > producing decent formatted output in HTML (and other formats, via > conversion from HTML). Comments on this bikeshed topic are also > welcome, I suppose, unless it gets out of hand... Markdown to man converters are the best available (e.g. kramdown-man) and markdown is usually easy to grok. lu ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-01 9:43 ` Luca Barbato @ 2013-09-01 16:57 ` Rich Felker 2013-09-02 0:08 ` Luca Barbato 0 siblings, 1 reply; 19+ messages in thread From: Rich Felker @ 2013-09-01 16:57 UTC (permalink / raw) To: musl On Sun, Sep 01, 2013 at 11:43:36AM +0200, Luca Barbato wrote: > On 01/09/13 06:41, Rich Felker wrote: > > One more thing: the format of the document is presently Markdown > > (http://daringfireball.net/projects/markdown/). I may switch to > > something else in the future, but Markdown tends to be my favorite > > from a standpoint of being fully readable as plain text and also > > producing decent formatted output in HTML (and other formats, via > > conversion from HTML). Comments on this bikeshed topic are also > > welcome, I suppose, unless it gets out of hand... > > Markdown to man converters are the best available (e.g. kramdown-man) > and markdown is usually easy to grok. Excellent. I'm not sure man format would be terribly useful for the musl documentation I'm working on now, but it's good to know for future reference, especially if we do want to provide a few man pages that could be installed for musl. For the big doc, HTML and PDF are probably the more interesting output formats. The big thing I don't see how to do easily is produce a nice table of contents or index. Rich ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-01 16:57 ` Rich Felker @ 2013-09-02 0:08 ` Luca Barbato 2013-09-02 7:12 ` Ivan Kanakarakis 0 siblings, 1 reply; 19+ messages in thread From: Luca Barbato @ 2013-09-02 0:08 UTC (permalink / raw) To: musl On 01/09/13 18:57, Rich Felker wrote: > For the big doc, HTML and PDF are probably the more interesting output > formats. The big thing I don't see how to do easily is produce a nice > table of contents or index. http://kramdown.rubyforge.org/converter/html.html#toc Surprisingly easy =) lu ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-02 0:08 ` Luca Barbato @ 2013-09-02 7:12 ` Ivan Kanakarakis 0 siblings, 0 replies; 19+ messages in thread From: Ivan Kanakarakis @ 2013-09-02 7:12 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1157 bytes --] On 2 September 2013 03:08, Luca Barbato <lu_zero@gentoo.org> wrote: > On 01/09/13 18:57, Rich Felker wrote: > > For the big doc, HTML and PDF are probably the more interesting output > HTML is the usual output of markdown parsers. AFAIK, GitHub uses redcarpet(ruby)[0] on top of sundown(C)[1] > > formats. The big thing I don't see how to do easily is produce a nice > > table of contents or index. > > http://kramdown.rubyforge.org/converter/html.html#toc > > Surprisingly easy =) > > lu > Kramdown is a superset of markdown. You cannot have an automatically generated toc with the original/standard markdown. Asciidoc[2] has that by default (among lots of others things). MultiMarkdown(C)[3] also adds lots of features[4] to markdown, getting markdown a little closer to Asciidoc's featureset. [0]: https://github.com/vmg/redcarpet [1]: https://github.com/vmg/sundown [2]: http://asciidoc.org/ [3]: https://github.com/fletcher/MultiMarkdown-4 [4]: https://rawgithub.com/fletcher/human-markdown-reference/master/index.html -- *Ivan c00kiemon5ter Kanakarakis* >:3 [-- Attachment #2: Type: text/html, Size: 3810 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-01 4:41 Initial draft of musl documentation/manual Rich Felker 2013-09-01 9:43 ` Luca Barbato @ 2013-09-01 13:45 ` Vasily Kulikov 2013-09-01 16:54 ` Rich Felker 2013-09-02 7:11 ` Christian Wiese ` (2 subsequent siblings) 4 siblings, 1 reply; 19+ messages in thread From: Vasily Kulikov @ 2013-09-01 13:45 UTC (permalink / raw) To: musl On Sun, Sep 01, 2013 at 00:41 -0400, Rich Felker wrote: > ### Filesystem Layout Dependencies > > musl aims to avoid imposing filesystem policy; however, the following > minimal set of filesystems dependencies must be met in order for > programs using musl to function correctly: > > * `/dev/null` - required by POSIX > > * `/tmp` - required by POSIX to exist as a directory, and used by > various temporary file creation functions. > > * `/dev/shm` - must be a directory, and should have permissions 01777. > If absent, POSIX shared memory and named semaphore interfaces will > fail; programs not using these features will be unaffected. If you name such files which are important in case of specific API usage and are not needed if this API is not used, why not enumerate other files like: /dev/ptmx /dev/pts/* /dev/tty Or, even better, compile two lists. The first is a list of files which are needed for sure to be able just to start executable compiled against musl, and a list of should-have files which are needed for several non-critical APIs like POSIX shared memory stuff. Thanks, -- Vasily Kulikov http://www.openwall.com - bringing security into open computing environments ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-01 13:45 ` Vasily Kulikov @ 2013-09-01 16:54 ` Rich Felker 0 siblings, 0 replies; 19+ messages in thread From: Rich Felker @ 2013-09-01 16:54 UTC (permalink / raw) To: musl On Sun, Sep 01, 2013 at 05:45:48PM +0400, Vasily Kulikov wrote: > On Sun, Sep 01, 2013 at 00:41 -0400, Rich Felker wrote: > > ### Filesystem Layout Dependencies > > > > musl aims to avoid imposing filesystem policy; however, the following > > minimal set of filesystems dependencies must be met in order for > > programs using musl to function correctly: > > > > * `/dev/null` - required by POSIX > > > > * `/tmp` - required by POSIX to exist as a directory, and used by > > various temporary file creation functions. > > > > * `/dev/shm` - must be a directory, and should have permissions 01777. > > If absent, POSIX shared memory and named semaphore interfaces will > > fail; programs not using these features will be unaffected. > > If you name such files which are important in case of specific API usage > and are not needed if this API is not used, why not enumerate other > files like: > > /dev/ptmx > /dev/pts/* > /dev/tty Indeed, these are unintentional omissions. I will add them. > Or, even better, compile two lists. The first is a list of files which > are needed for sure to be able just to start executable compiled against > musl, That list would be empty except for the dynamic linker in the case of dynamic-linked programs. > and a list of should-have files which are needed for several > non-critical APIs like POSIX shared memory stuff. The problem is that basically all of the files/devices are only needed for certain particular features, and different people will have different ideas of what's important. For non-networked use, DNS/resolv.conf is unimportant. For many programs, /proc is not important, but I'd rather not document exactly what it's needed for, since it may be needed for more things in the future. Perhaps the best structure is: Group A: These are considered "required" in the sense that they're used internally and it's intentionally undocumented where they're used, as they might be used increasingly in the future. Group B: These are considered "optional" in the sense that they're used by specific interfaces and we document which interfaces need them, so you can omit them if you won't be using the associated features. Rich ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Initial draft of musl documentation/manual 2013-09-01 4:41 Initial draft of musl documentation/manual Rich Felker 2013-09-01 9:43 ` Luca Barbato 2013-09-01 13:45 ` Vasily Kulikov @ 2013-09-02 7:11 ` Christian Wiese 2013-09-06 1:12 ` Second " Rich Felker 2013-09-06 4:20 ` Third " Rich Felker 4 siblings, 0 replies; 19+ messages in thread From: Christian Wiese @ 2013-09-02 7:11 UTC (permalink / raw) To: musl Hi, On Sun, 1 Sep 2013 00:41:06 -0400, Rich Felker <dalias@aerifal.cx> wrote: > > One more thing: the format of the document is presently Markdown > (http://daringfireball.net/projects/markdown/). I may switch to > something else in the future, but Markdown tends to be my favorite > from a standpoint of being fully readable as plain text and also > producing decent formatted output in HTML (and other formats, via > conversion from HTML). Comments on this bikeshed topic are also > welcome, I suppose, unless it gets out of hand... > in my opinion the "being fully readable as plain text" is a good thing and should be preserved. I don't know much about markdown and its capabilities, and how easily or non-easily it is done to generate high-quality documents in different output formats from it. Asciidoc [1] for example fulfils this criteria, by being fully readable as plain text, and providing the capability to generate different kinds of output formats in a high quality. "The asciidoc(1) command translates AsciiDoc files to HTML, XHTML and DocBook markups. DocBook can be post-processed to presentation formats such as HTML, PDF, EPUB, DVI, LaTeX, roff, and Postscript using readily available Open Source tools." You can take a look at the asciidoc source file of the index page the asciddoc website is rendered from to get a slight overview how it looks like. Cheers, Chris [1] http://asciidoc.org/ [2] http://asciidoc.org/index.txt ^ permalink raw reply [flat|nested] 19+ messages in thread
* Second draft of musl documentation/manual 2013-09-01 4:41 Initial draft of musl documentation/manual Rich Felker ` (2 preceding siblings ...) 2013-09-02 7:11 ` Christian Wiese @ 2013-09-06 1:12 ` Rich Felker 2013-09-06 2:41 ` Szabolcs Nagy 2013-09-06 4:20 ` Third " Rich Felker 4 siblings, 1 reply; 19+ messages in thread From: Rich Felker @ 2013-09-06 1:12 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 239 bytes --] Here is the updated manual draft, with more (most) of Part I filled in. Parts II and III remain empty. I'm hoping to have some progress on the remaining parts before releasing 0.9.14, and to include the documentation in the release. Rich [-- Attachment #2: manual5.txt --] [-- Type: text/plain, Size: 11186 bytes --] musl ==== Part I - About musl ------------------- ### Introduction and Scope musl is an implementation of the userspace portion of the standard library functionality described in the ISO C and POSIX standards, plus common extensions. It is both a component for use in Linux-based operating systems and a tool for building application binaries deployable on a wide range of Linux-based systems and non-Linux systems which can provide a compatible syscall API layer. ### Conformance The interfaces in musl are modeled upon and intended to conform to the requirements of the ISO C99 standard (ISO/IEC 9899-1999), including Annex F, and POSIX 2008 / Single Unix Standard Version 4, with all current technical corrigenda applied. However, musl has not been certified by any standards body, and no guarantee of conformance is made by the copyright holders or any other party with an interest in musl. Moreover, since musl provides only the userspace portion of the standard system interfaces, conformance to the requirements of POSIX depends in part on the behavior of the underlying kernel. Linux 2.6.39 or later is believed to be sufficient; earlier versions in the 2.6 series will work, but with varying degrees of non-conformance, particularly in the area of signal handling behavior and close-on-exec race conditions. Likewise, conformance to the requirements of ISO C, and especially Annex F (IEEE floating point semantics), depends in part on both the compiler used to build musl and the compiler used when building applications against musl. At this time there is no known fully conforming compiler. ### Supported Targets * i386 * Requires support or kernel emulation of `cmpxchg` instruction, introduced on the 80486 * x86_64 * ARM * EABI, standard or hard-float VFP variant * Little-endian default; big-endian variants also supported * Compiler toolchains only support armv4t and later * MIPS * ABI is o32 * Big-endian default; little-endian variants also supported * MIPS2 or later, or kernel emulation of ll/sc (standard in Linux) is required * FPU or kernel float emulation (standard in Linux but disabled on some OpenWRT builds) is required * PowerPC * Only 32-bit is supported * Compiler toolchain must provide 64-bit long double, not IBM double-double or IEEE quad * For dynamic linking, compiler toolchain must be configured for "secure PLT" variant * Microblaze * Big-endian default; little-endian variants also supported * Soft-float ### Build and Installation The build system for musl uses the well-known `./configure` idiom. musl's configure script is not based on GNU autoconf, but is intended to closely match the configure API documented in the GNU Coding Standards. Alternatively, the provided template for `config.mak` may be edited by hand in place of running `./configure`. #### Prerequisites The only build-time prerequisites for musl are the standard POSIX shell and utilities, GNU Make (version 3.81 or later) and a freestanding C99 compiler toolchain targeting the desired instruction set architecture and ABI, with support for gcc-style inline assembly, weak aliases, and stand-alone assembly source files. The system used to build musl does not need to be Linux-based, nor do the Linux kernel headers need to be available. If support for dynamic linking is desired, some further requriements are placed on the compiler and linker. In particular, the linker must support the `-Bsymbolic-functions` option, and the compiler must not generate gratuitous GOT relocations where GOT-relative or PC-relative addressing could be used instead. #### Build options Running `./configure --help` from the top-level source directory will print usage information for configure. In most cases, the only options which should be needed are: * `--prefix`, used to control where musl will be installed. The prefix for musl defaults to `/usr/local/musl` rather than `/usr/local` to avoid breaking an existing non-musl environment on the host. If musl will be used as the primary system libc, prefix should usually be set to `/usr` or `/`. * `--syslibdir`, used to specify the location at which the dynamic linker should be installed and found at runtime. The default of `/lib` should only be overridden when installing in `/lib` is impossible, since the pathname of the dynamic linker is stored in all dynamic-linked executables, and executables using non-standard paths for the dynamic linker may be difficult to deploy on other systems. Both `--prefix` and `--syslibdir` should reflect the final runtime location where musl will be installed. If musl should be installed to a different location to prepare a package file or new target system image, the `DESTDIR` variable can be set when running `make install`. In this case, `DESTDIR` will be prepended to all installation paths, but will not be saved anywhere in the files installed. Other build options of interest are: * `CC=...`, to choose a non-default compiler. * `CFLAGS=...`, to pass custom options to the compiler. * `--disable-shared`, to disable building shared `libc.so` if it will not be needed. This cuts the build time in half. * `--disable-static`, to disable building `libc.a`. Other (empty) `.a` files are still built. This also cuts the build time in half. * `--enable-optimize=`*list*, where *list* is a comma-separated list of components (subdirectories of `src`, or glob patterns) which will be optimized at `-O3` rather than the default optimization level `-Os`. Manually specifying an optimization level in the provided `CFLAGS`, or using `--enable-debug` or `--disable-optimize`, will turn off default optimizations. * `--enable-warnings`, to turn on the recommended set of GCC warning options with which musl is intended to compile warning-free. * `--enable-debug`, to turn on debugging. Adding `-g` to `CFLAGS` manually also works. In the future, `--enable-debug` may also enable additional debugging features at the source level. See `./configure --help` for additional options. #### Compiling and Installing After running configure, run `make` to compile and `make install` to install. If desired, `make install` can be invoked directly without first running `make`, but it may be desirable to do these as separate steps if eleveated privileges are needed to install to the final destination. musl's makefile is fully declarative and non-recursive, and may be arbitrarily parallelized with the `-j` option. #### After Installation If installing for the first time and using dynamic linking, it may be necessary to create a path file for the dynamic linker. See `../etc/ld-musl-$(ARCH).path` in the next section of the manual. ### Installed Components In the following, `$(syslibdir)`, `$(includedir)`, and `$(libdir)` refer to the paths chosen at build time (by default, `/lib`, `$(prefix)/include`, and `$(prefix)/lib`, respectively) and `$(ARCH)` refers to the *full* name for the target CPU architecture/ABI, including the "subarch" component. #### Dynamic linking runtime `$(syslibdir)/ld-musl-$(ARCH).so.1` provides the dynamic linker, or "program interpreter", for dynamically linked ELF programs using musl. The absolute pathname to this file must be stored in all such programs. The build and installation system provided with musl sets it up as a symbolic link to `$(libdir)/libc.so`, but system integrators may choose to make it available in whichever ways they find suitable. #### Development environment Header files for use by the C compiler are installed in `$(includedir)`. The standard headers are fully self-contained, and do not make use of kernel-provided or compiler-provided headers or otherwise require such headers to be present. The file `libc.a` installed in `$(libdir)` provides the entire standard library implementation for static linking. The file `libc.so` provides the linker with access to the standard library's symbols for use at link-time in producing dynamic-linked binaries. It is not searched at runtime; the standard library is resolved as part of the program interpreter at `$(syslibdir)/ld-musl-$(ARCH).so.1`. Additional files `libm.a`, `librt.a`, `libpthread.a`, `libcrypt.a`, `libutil.a`, `libxnet.a`, `libresolv.a`, and `libdl.a` are provided in `$(libdir)` as empty library archives. They contain no code, but are present to satisfy the POSIX requirement that options of the form `-lm`, `-lpthread`, etc. be accepted by the `c99` compiler. Several bare object files are also included in `$(libdir)`: `crt1.o` and `Scrt1.o` are the normal and position-independent versions, respectively, of the entry point code linked into every program. `crti.o` and `crtn.o`, also linked into every program and into shared libraries, provide support for legacy means by which the compiler can arrange for global constructors and destructors to be executed. It is possible to setup a legacy-free compiler toolchain that does not need the `crti.o` and `crtn.o` files if desired. #### Compiler wrapper To be written. ### Filesystem Layout Dependencies musl aims to avoid imposing filesystem policy; however, the following minimal set of filesystems dependencies must be met in order for programs using musl to function correctly: * `/dev/null` - required by POSIX * `/dev/tty` - required by POSIX * `/tmp` - required by POSIX to exist as a directory, and used by various temporary file creation functions. * `/dev/shm` - must be a directory, and should have permissions 01777. If absent, POSIX shared memory and named semaphore interfaces will fail; programs not using these features will be unaffected. * `/dev/ptmx` - must exist and be accessible for read/write in order for pseudo-terminal opening to work. * `/dev/pts` - must be a mounted devpts filesystem in order for pseudo-terminal opening to work. * `/proc` - must be a mount point for Linux procfs or a symlink to such. Several functions such as realpath, fexecve, and a number of the "at" functions added in POSIX 2008 need access to /proc to function correctly. * `/etc/resolv.conf` - needed to provide addresses of nameservers to be used for DNS lookups, unless a working nameserver is available on the loopback address. * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the "program interpreter" specified in the program's headers - if present, this will be processed as a text file containing the shared library search path, with components delimited by newlines or colons. If absent, a default path of `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by static-linked programs. Part II - Usage --------------- To be written. This part of the manual will deal with documenting implementation-defined behavior and further behaviors that are not required to be documented but for which musl makes additional guarantees. Part III - Implementation ------------------------- To be written. This part of the manual will document the implementation of musl, including matters such as source tree layout, built system, algorithms used, musl-internal APIs, coding style, and information on porting. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Second draft of musl documentation/manual 2013-09-06 1:12 ` Second " Rich Felker @ 2013-09-06 2:41 ` Szabolcs Nagy 2013-09-06 2:57 ` Rich Felker 0 siblings, 1 reply; 19+ messages in thread From: Szabolcs Nagy @ 2013-09-06 2:41 UTC (permalink / raw) To: musl * Rich Felker <dalias@aerifal.cx> [2013-09-05 21:12:52 -0400]: > #### Prerequisites > > The only build-time prerequisites for musl are the standard POSIX > shell and utilities, GNU Make (version 3.81 or later) and a > freestanding C99 compiler toolchain targeting the desired instruction > set architecture and ABI, with support for gcc-style inline assembly, > weak aliases, and stand-alone assembly source files. > there are some optional extensions and compiler defined macros or builtins which may worth listing somewhere (noreturn, noalias, typeof, 1.0i,...) > #### Dynamic linking runtime > > `$(syslibdir)/ld-musl-$(ARCH).so.1` provides the dynamic linker, or > "program interpreter", for dynamically linked ELF programs using musl. > The absolute pathname to this file must be stored in all such > programs. The build and installation system provided with musl sets it > up as a symbolic link to `$(libdir)/libc.so`, but system integrators > may choose to make it available in whichever ways they find suitable. > in the usage section: ./libc.so pathname args ./ldd pathname > #### Development environment > > Header files for use by the C compiler are installed in > `$(includedir)`. The standard headers are fully self-contained, and do > not make use of kernel-provided or compiler-provided headers or > otherwise require such headers to be present. it is also worth noting that header files are not c99 (they are intended to be c89 except for long long and possibly wchar_t literals and should be usable with c++ as well) > #### Compiler wrapper > > To be written. > > > ### Filesystem Layout Dependencies > > musl aims to avoid imposing filesystem policy; however, the following > minimal set of filesystems dependencies must be met in order for > programs using musl to function correctly: > > * `/dev/null` - required by POSIX > > * `/dev/tty` - required by POSIX > > * `/tmp` - required by POSIX to exist as a directory, and used by > various temporary file creation functions. > > * `/dev/shm` - must be a directory, and should have permissions 01777. > If absent, POSIX shared memory and named semaphore interfaces will > fail; programs not using these features will be unaffected. > > * `/dev/ptmx` - must exist and be accessible for read/write in order > for pseudo-terminal opening to work. > > * `/dev/pts` - must be a mounted devpts filesystem in order for > pseudo-terminal opening to work. > > * `/proc` - must be a mount point for Linux procfs or a symlink to > such. Several functions such as realpath, fexecve, and a number of the > "at" functions added in POSIX 2008 need access to /proc to function > correctly. > > * `/etc/resolv.conf` - needed to provide addresses of nameservers to > be used for DNS lookups, unless a working nameserver is available on > the loopback address. > > * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the > "program interpreter" specified in the program's headers - if present, > this will be processed as a text file containing the shared library > search path, with components delimited by newlines or colons. If > absent, a default path of `"/lib:/usr/local/lib:/usr/lib"` will be > used. Not used by static-linked programs. > i'd probably repeat the program interpreter here again and i'd list all paths that may be used /bin/sh /etc/shadow or /etc/tcb/... /etc/passwd /etc/group default zoneinfo search paths /dev/log /etc/services /etc/hosts LD_PRELOAD and environment variable dependencies should be documented around here as well and the security policy for setuid binaries > Part II - Usage > --------------- > > To be written. This part of the manual will deal with documenting > implementation-defined behavior and further behaviors that are not > required to be documented but for which musl makes additional > guarantees. > > > > Part III - Implementation > ------------------------- > > To be written. This part of the manual will document the > implementation of musl, including matters such as source tree layout, > built system, algorithms used, musl-internal APIs, coding style, and > information on porting. the documentation looks good so far ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Second draft of musl documentation/manual 2013-09-06 2:41 ` Szabolcs Nagy @ 2013-09-06 2:57 ` Rich Felker 0 siblings, 0 replies; 19+ messages in thread From: Rich Felker @ 2013-09-06 2:57 UTC (permalink / raw) To: musl On Fri, Sep 06, 2013 at 04:41:21AM +0200, Szabolcs Nagy wrote: > * Rich Felker <dalias@aerifal.cx> [2013-09-05 21:12:52 -0400]: > > #### Prerequisites > > > > The only build-time prerequisites for musl are the standard POSIX > > shell and utilities, GNU Make (version 3.81 or later) and a > > freestanding C99 compiler toolchain targeting the desired instruction > > set architecture and ABI, with support for gcc-style inline assembly, > > weak aliases, and stand-alone assembly source files. > > > > there are some optional extensions and compiler defined macros > or builtins which may worth listing somewhere > (noreturn, noalias, typeof, 1.0i,...) Is 1.0i needed for compiling musl? I thought CMPLX() would be used, but I haven't checked. The others are not needed for compiling musl (used optionally, dependent on __GNUC__), but may be needed for compiling applications using certain headers. I'm documenting this in Part II. So far tgmath.h and complex.h are the only examples I have where a header _needs_ an extension to work. Do you know others I should document? > > #### Dynamic linking runtime > > > > `$(syslibdir)/ld-musl-$(ARCH).so.1` provides the dynamic linker, or > > "program interpreter", for dynamically linked ELF programs using musl. > > The absolute pathname to this file must be stored in all such > > programs. The build and installation system provided with musl sets it > > up as a symbolic link to `$(libdir)/libc.so`, but system integrators > > may choose to make it available in whichever ways they find suitable. > > > > in the usage section: > ../libc.so pathname args > ../ldd pathname Good idea. > > #### Development environment > > > > Header files for use by the C compiler are installed in > > `$(includedir)`. The standard headers are fully self-contained, and do > > not make use of kernel-provided or compiler-provided headers or > > otherwise require such headers to be present. > > it is also worth noting that header files are not c99 (they are > intended to be c89 except for long long and possibly wchar_t literals > and should be usable with c++ as well) Yes, I've already written that text in Part II (in progress right now). > > #### Compiler wrapper > > > > To be written. > > > > > > ### Filesystem Layout Dependencies > > > > musl aims to avoid imposing filesystem policy; however, the following > > minimal set of filesystems dependencies must be met in order for > > programs using musl to function correctly: > > > > * `/dev/null` - required by POSIX > > > > * `/dev/tty` - required by POSIX > > > > * `/tmp` - required by POSIX to exist as a directory, and used by > > various temporary file creation functions. > > > > * `/dev/shm` - must be a directory, and should have permissions 01777. > > If absent, POSIX shared memory and named semaphore interfaces will > > fail; programs not using these features will be unaffected. > > > > * `/dev/ptmx` - must exist and be accessible for read/write in order > > for pseudo-terminal opening to work. > > > > * `/dev/pts` - must be a mounted devpts filesystem in order for > > pseudo-terminal opening to work. > > > > * `/proc` - must be a mount point for Linux procfs or a symlink to > > such. Several functions such as realpath, fexecve, and a number of the > > "at" functions added in POSIX 2008 need access to /proc to function > > correctly. > > > > * `/etc/resolv.conf` - needed to provide addresses of nameservers to > > be used for DNS lookups, unless a working nameserver is available on > > the loopback address. > > > > * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the > > "program interpreter" specified in the program's headers - if present, > > this will be processed as a text file containing the shared library > > search path, with components delimited by newlines or colons. If > > absent, a default path of `"/lib:/usr/local/lib:/usr/lib"` will be > > used. Not used by static-linked programs. > > > > i'd probably repeat the program interpreter here again OK. > and i'd list all paths that may be used > > /bin/sh > > /etc/shadow or /etc/tcb/... > /etc/passwd > /etc/group > > default zoneinfo search paths > > /dev/log > > /etc/services > /etc/hosts Indeed. Thanks for finding these. > LD_PRELOAD and environment variable dependencies should be documented > around here as well Perhaps these (and the above filesystem dependencies) should be in Part II. Also, we might want to add separate parts for run-time and development usage. > and the security policy for setuid binaries Good idea. > > Part II - Usage > > --------------- > > > > To be written. This part of the manual will deal with documenting > > implementation-defined behavior and further behaviors that are not > > required to be documented but for which musl makes additional > > guarantees. > > > > > > > > Part III - Implementation > > ------------------------- > > > > To be written. This part of the manual will document the > > implementation of musl, including matters such as source tree layout, > > built system, algorithms used, musl-internal APIs, coding style, and > > information on porting. > > the documentation looks good so far Thanks! Rich ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Third draft of musl documentation/manual 2013-09-01 4:41 Initial draft of musl documentation/manual Rich Felker ` (3 preceding siblings ...) 2013-09-06 1:12 ` Second " Rich Felker @ 2013-09-06 4:20 ` Rich Felker 2013-09-06 15:14 ` Szabolcs Nagy ` (3 more replies) 4 siblings, 4 replies; 19+ messages in thread From: Rich Felker @ 2013-09-06 4:20 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 451 bytes --] Here is the third version of the docs/manual draft, with a lot more content filled in, some of it based on previous comments. The biggest task left is filling in all the implementation-defined behavior. For plain C functions, J.3.12 gives a nice checklist of things to document, but POSIX has a lot more things which are specified as implementation-defined for which I don't have such a nice checklist. Ideas on how to build one would be great. Rich [-- Attachment #2: manual6.txt --] [-- Type: text/plain, Size: 17927 bytes --] musl ==== Part I - About musl ------------------- ### Introduction and Scope musl is an implementation of the userspace portion of the standard library functionality described in the ISO C and POSIX standards, plus common extensions. It is both a component for use in Linux-based operating systems and a tool for building application binaries deployable on a wide range of Linux-based systems and non-Linux systems which can provide a compatible syscall API layer. ### Conformance The interfaces in musl are modeled upon and intended to conform to the requirements of the ISO C99 standard (ISO/IEC 9899-1999), including Annex F, and POSIX 2008 / Single Unix Standard Version 4, with all current technical corrigenda applied. However, musl has not been certified by any standards body, and no guarantee of conformance is made by the copyright holders or any other party with an interest in musl. Moreover, since musl provides only the userspace portion of the standard system interfaces, conformance to the requirements of POSIX depends in part on the behavior of the underlying kernel. Linux 2.6.39 or later is believed to be sufficient; earlier versions in the 2.6 series will work, but with varying degrees of non-conformance, particularly in the area of signal handling behavior and close-on-exec race conditions. Likewise, conformance to the requirements of ISO C, and especially Annex F (IEEE floating point semantics), depends in part on both the compiler used to build musl and the compiler used when building applications against musl. At this time there is no known fully conforming compiler. ### Supported Targets * i386 * Requires support or kernel emulation of `cmpxchg` instruction, introduced on the 80486 * x86_64 * ARM * EABI, standard or hard-float VFP variant * Little-endian default; big-endian variants also supported * Compiler toolchains only support armv4t and later * MIPS * ABI is o32 * Big-endian default; little-endian variants also supported * MIPS2 or later, or kernel emulation of ll/sc (standard in Linux) is required * FPU or kernel float emulation (standard in Linux but disabled on some OpenWRT builds) is required * PowerPC * Only 32-bit is supported * Compiler toolchain must provide 64-bit long double, not IBM double-double or IEEE quad * For dynamic linking, compiler toolchain must be configured for "secure PLT" variant * Microblaze * Big-endian default; little-endian variants also supported * Soft-float ### Build and Installation The build system for musl uses the well-known `./configure` idiom. musl's configure script is not based on GNU autoconf, but is intended to closely match the configure API documented in the GNU Coding Standards. Alternatively, the provided template for `config.mak` may be edited by hand in place of running `./configure`. #### Prerequisites The only build-time prerequisites for musl are the standard POSIX shell and utilities, GNU Make (version 3.81 or later) and a freestanding C99 compiler toolchain targeting the desired instruction set architecture and ABI, with support for gcc-style inline assembly, weak aliases, and stand-alone assembly source files. The system used to build musl does not need to be Linux-based, nor do the Linux kernel headers need to be available. If support for dynamic linking is desired, some further requriements are placed on the compiler and linker. In particular, the linker must support the `-Bsymbolic-functions` option, and the compiler must not generate gratuitous GOT relocations where GOT-relative or PC-relative addressing could be used instead. #### Build options Running `./configure --help` from the top-level source directory will print usage information for configure. In most cases, the only options which should be needed are: * `--prefix`, used to control where musl will be installed. The prefix for musl defaults to `/usr/local/musl` rather than `/usr/local` to avoid breaking an existing non-musl environment on the host. If musl will be used as the primary system libc, prefix should usually be set to `/usr` or `/`. * `--syslibdir`, used to specify the location at which the dynamic linker should be installed and found at runtime. The default of `/lib` should only be overridden when installing in `/lib` is impossible, since the pathname of the dynamic linker is stored in all dynamic-linked executables, and executables using non-standard paths for the dynamic linker may be difficult to deploy on other systems. Both `--prefix` and `--syslibdir` should reflect the final runtime location where musl will be installed. If musl should be installed to a different location to prepare a package file or new target system image, the `DESTDIR` variable can be set when running `make install`. In this case, `DESTDIR` will be prepended to all installation paths, but will not be saved anywhere in the files installed. Other build options of interest are: * `CC=...`, to choose a non-default compiler. * `CFLAGS=...`, to pass custom options to the compiler. * `--disable-shared`, to disable building shared `libc.so` if it will not be needed. This cuts the build time in half. * `--disable-static`, to disable building `libc.a`. Other (empty) `.a` files are still built. This also cuts the build time in half. * `--enable-optimize=`*list*, where *list* is a comma-separated list of components (subdirectories of `src`, or glob patterns) which will be optimized at `-O3` rather than the default optimization level `-Os`. Manually specifying an optimization level in the provided `CFLAGS`, or using `--enable-debug` or `--disable-optimize`, will turn off default optimizations. * `--enable-warnings`, to turn on the recommended set of GCC warning options with which musl is intended to compile warning-free. * `--enable-debug`, to turn on debugging. Adding `-g` to `CFLAGS` manually also works. In the future, `--enable-debug` may also enable additional debugging features at the source level. See `./configure --help` for additional options. #### Compiling and Installing After running configure, run `make` to compile and `make install` to install. If desired, `make install` can be invoked directly without first running `make`, but it may be desirable to do these as separate steps if eleveated privileges are needed to install to the final destination. musl's makefile is fully declarative and non-recursive, and may be arbitrarily parallelized with the `-j` option. #### After Installation If installing for the first time and using dynamic linking, it may be necessary to create a path file for the dynamic linker. See `../etc/ld-musl-$(ARCH).path` in the next section of the manual. ### Installed Components In the following, `$(syslibdir)`, `$(includedir)`, and `$(libdir)` refer to the paths chosen at build time (by default, `/lib`, `$(prefix)/include`, and `$(prefix)/lib`, respectively) and `$(ARCH)` refers to the *full* name for the target CPU architecture/ABI, including the "subarch" component. #### Dynamic linking runtime `$(syslibdir)/ld-musl-$(ARCH).so.1` provides the dynamic linker, or "program interpreter", for dynamically linked ELF programs using musl. The absolute pathname to this file must be stored in all such programs. The build and installation system provided with musl sets it up as a symbolic link to `$(libdir)/libc.so`, but system integrators may choose to make it available in whichever ways they find suitable. #### Development environment Header files for use by the C compiler are installed in `$(includedir)`. The standard headers are fully self-contained, and do not make use of kernel-provided or compiler-provided headers or otherwise require such headers to be present. The file `libc.a` installed in `$(libdir)` provides the entire standard library implementation for static linking. The file `libc.so` provides the linker with access to the standard library's symbols for use at link-time in producing dynamic-linked binaries. It is not searched at runtime; the standard library is resolved as part of the program interpreter at `$(syslibdir)/ld-musl-$(ARCH).so.1`. Additional files `libm.a`, `librt.a`, `libpthread.a`, `libcrypt.a`, `libutil.a`, `libxnet.a`, `libresolv.a`, and `libdl.a` are provided in `$(libdir)` as empty library archives. They contain no code, but are present to satisfy the POSIX requirement that options of the form `-lm`, `-lpthread`, etc. be accepted by the `c99` compiler. Several bare object files are also included in `$(libdir)`: `crt1.o` and `Scrt1.o` are the normal and position-independent versions, respectively, of the entry point code linked into every program. `crti.o` and `crtn.o`, also linked into every program and into shared libraries, provide support for legacy means by which the compiler can arrange for global constructors and destructors to be executed. It is possible to setup a legacy-free compiler toolchain that does not need the `crti.o` and `crtn.o` files if desired. #### Compiler wrapper Included with musl is a wrapper script `musl-gcc` which can be used with an existing GCC compiler toolchain to build programs using musl. If installed, the script itself is located at `$(bindir)/musl-gcc`, and a supporting GCC specs file it uses is located at `$(libdir)/musl-gcc.specs`. Part II - Runtime Usage ----------------------- ### Filesystem Layout Dependencies musl aims to avoid imposing filesystem policy; however, the following minimal set of filesystems dependencies must be met in order for programs using musl to function correctly: * `/dev/null` - device node, required by POSIX * `/dev/tty` - device node, required by POSIX * `/tmp` - required by POSIX to exist as a directory, and used by various temporary file creation functions. * `/bin/sh` - an executable file providing as POSIX-conforming shell * `/proc` - must be a mount point for Linux procfs or a symlink to such. Several functions such as realpath, fexecve, and a number of the "at" functions added in POSIX 2008 need access to /proc to function correctly. While some programs may operate correctly even without some or all of the above, musl's behavior in their absence is unspecified. ### Additional Pathnames Used * `/dev/log` - a UNIX domain socket to which the `syslog()` interface sends log messages. If absent or inaccessible, log messages will be discarded. * `/dev/shm` - a directory; should have permissions 01777. If absent, POSIX shared memory and named semaphore interfaces will fail; programs not using these features will be unaffected. * `/dev/ptmx` and `/dev/pts` - device node and devpts filesystem mount point, respectively. If absent or inaccessible, `posix_openpt()` and `openpty()` will fail. * `/etc/passwd` and `/etc/group` - text files containing the user and group databases, mappings between names and numeric ids, and group membership lists, in the standard traditional format. If absent, user and/or group lookups will fail. * `/etc/shadow` - text file containing shadow password hashes for some or all users. * `/etc/resolv.conf` - text file providing addresses of nameservers to be used for DNS lookups. If absent, DNS requests will be sent to the loopback address and will fail unless the host has its own nameserver. * `/etc/hosts` - text file mapping hostnames to IP addresses. * `/etc/services` - text file mapping network service names to port numbers. * `/usr/share/zoneinfo`, `/share/zoneinfo`, and `/etc/zoneinfo` - directories searched for time zone files when the `TZ` environment variable is set to a relative pathname. * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the "program interpreter" specified in the program's headers - if present, this will be processed as a text file containing the shared library search path, with components delimited by newlines or colons. If absent, a default path of `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by static-linked programs. Part III - Library Usage for Development ---------------------------------------- ### Compiler Support All public interfaces in musl, at both the header file and library level, are intended to be mostly compatible with any C99, C11, or C++ compiler targetting the same CPU architecture and ABI musl was built for. C89 compilers are also supported provided that they accept the `long long` type as an extension. A few public header files do, however, require compiler-specific extensions in order to provide the mandated standard features: * `complex.h` requires `1.0fi` to be accepted as a constant expression suitable for defining `_Complex_I`. * `tgmath.h` requires the `__typeof__` extension. * FIXME: is this list complete? ### System Header Files (Add some intro text on musl's headers.) #### Introduction to Namespace Issues Any C program using a library, whether the standard library or a third-party library, needs to observe a contract with the library regarding usage of identifiers - in particular, which identifiers are used as part of the library's public interface or header file implementation, and which identifiers are used by the application. Having a clear contract is especially important when the library being used is not a single fixed implementation, but may have multiple versions or multiple independent implementations. The canonical example of such a library is the standard library. ISO C reserves all identifiers which are not explicitly defined or reserved by the standard for use by the appliction. POSIX, however, exposes a number of additional identifiers, and popular extensions outside of the standards define even more. In order to support applications which are written with different expectations on which identifiers may be used for the application's purposes, and which ones are defined by the system, a mechanism must be provided for choosing *which contract* will be used. #### Introduction to Feature Test Macros To solve this problem, POSIX introduced the concept of *feature test macros*. These are macros which an application may define *prior to the inclusion of any system header* (either at the source level, or via `-D` options passed as arguments to the compiler) in order to request a particular namespace contract. POSIX 2008 specifies two such feature test macros: * `_POSIX_C_SOURCE`, defined to `200809L` to request all interfaces defined in the POSIX base standard. * `_XOPEN_SOURCE`, defined to `700` to request all interfaces defined under the XSI option in addition to POSIX base. No requirements are placed on the namespace when neither of these macros is defined by the application. If one or both of these macros is defined by the application, two constraints are placed on the system headers: * They must define all macros and declare all functions and objects which the standard specifies for that header to provide. * They must not make use of any identifier not specified or reserved for that header. There is, however, an exception to the second rule: since the standard does not define behavior when the application has defined macros whose names are reserved for system use, implementations may specify their own feature test macros to expose additional identifiers alongside the standard ones. This is what musl, and most other implementations of the standard library, do. #### Feature Test Macros Supported by musl If no feature test macros are defined, musl's headers operate in "default features" mode, exposing the equivalent of the `_BSD_SOURCE` option below. This corresponds fairly well to what most applications unaware of feature test macros expect, and also provides a number of more modern features. Otherwise, if at least one of the below-listed feature test macros is defined, they are treated additively, starting from pure ISO C as a base. Unless otherwise specified, musl ignores the value of the macro and only checks whether it is defined. * `__STRICT_ANSI__` Adds nothing; only suppresses the default features. This macro is defined automatically by GCC and other major compilers in strict standards-conformance modes. * `_POSIX_C_SOURCE` (or `_POSIX_SOURCE`) As specified by POSIX 2008; adds POSIX base. If defined to a value less than `200809L`, or if the deprecated version `_POSIX_SOURCE` is defined at all, interfaces which were removed from the standard but which are still in widespread use are also exposed. * `_XOPEN_SOURCE` As specified by POSIX 2008; adds all interfaces in POSIX including the XSI option. If defined to a value less than `700`, interfaces which were removed from the standard but which are still in widespread use are also exposed. * `_BSD_SOURCE` Adds everything above, plus a number of traditional and modern interfaces modeled after BSD systems, or supported on current BSD systems based on older standards such as SVID. * `_GNU_SOURCE` (or `_ALL_SOURCE`) Adds everything above, plus interfaces modelef after GNU libc extensions and interfaces for making use of Linux-specific features. The specific extensions provided by the nonstandard feature test macros are documented in subsequent sections of this manual. ### Implementation-Defined Behavior To be written. ### BSD-Compatible Extensions To be written. ### GNU-Compatible Extensions To be written. ### Linux Extensions To be written. ### Other Nonstandard Behaviors and Extensions To be written. ### Quality of Implementation Guarantees To be written. Part IV - Implementation ------------------------ To be written. This part of the manual will document the implementation of musl, including matters such as source tree layout, built system, algorithms used, musl-internal APIs, coding style, and information on porting. ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Third draft of musl documentation/manual 2013-09-06 4:20 ` Third " Rich Felker @ 2013-09-06 15:14 ` Szabolcs Nagy 2013-09-06 15:47 ` Rich Felker 2013-09-08 19:22 ` Ivan Kanakarakis ` (2 subsequent siblings) 3 siblings, 1 reply; 19+ messages in thread From: Szabolcs Nagy @ 2013-09-06 15:14 UTC (permalink / raw) To: musl * Rich Felker <dalias@aerifal.cx> [2013-09-06 00:20:43 -0400]: > document, but POSIX has a lot more things which are specified as > implementation-defined for which I don't have such a nice checklist. > Ideas on how to build one would be great. i would download the htmls from http://pubs.opengroup.org/onlinepubs/9699919799/download/index.html and then grep -r --color=always -C2 -i implementation-defined basedefs/ functions/ |less -R > ### Supported Targets > > * i386 > * Requires support or kernel emulation of `cmpxchg` instruction, > introduced on the 80486 maybe note that both i387 and sse fpus are supported > #### Compiler wrapper > > Included with musl is a wrapper script `musl-gcc` which can be used > with an existing GCC compiler toolchain to build programs using musl. > If installed, the script itself is located at `$(bindir)/musl-gcc`, > and a supporting GCC specs file it uses is located at > `$(libdir)/musl-gcc.specs`. > note that it uses ${REALGCC:-gcc}, not the compiler configured to build musl > ### Filesystem Layout Dependencies > > musl aims to avoid imposing filesystem policy; however, the following > minimal set of filesystems dependencies must be met in order for > programs using musl to function correctly: > > * `/dev/null` - device node, required by POSIX > > * `/dev/tty` - device node, required by POSIX > > * `/tmp` - required by POSIX to exist as a directory, and used by > various temporary file creation functions. > > * `/bin/sh` - an executable file providing as POSIX-conforming shell > > * `/proc` - must be a mount point for Linux procfs or a symlink to > such. Several functions such as realpath, fexecve, and a number of > the "at" functions added in POSIX 2008 need access to /proc to > function correctly. > > While some programs may operate correctly even without some or all of > the above, musl's behavior in their absence is unspecified. > i would still add a note about the hardcoded program interpreter path in dynamically linked elf binaries that is required so the kernel can execute those binaries (referring back to the dynamic linking section about ld-musl-* and libc.so) (arguably this is not a musl runtime fs layout dependency, you can interpret the binary as './libc.so name', but is worth mentioning here) > ### Additional Pathnames Used > > * `/dev/log` - a UNIX domain socket to which the `syslog()` interface > sends log messages. If absent or inaccessible, log messages will be > discarded. > > * `/dev/shm` - a directory; should have permissions 01777. If absent, > POSIX shared memory and named semaphore interfaces will fail; > programs not using these features will be unaffected. > > * `/dev/ptmx` and `/dev/pts` - device node and devpts filesystem mount > point, respectively. If absent or inaccessible, `posix_openpt()` and > `openpty()` will fail. > > * `/etc/passwd` and `/etc/group` - text files containing the user and > group databases, mappings between names and numeric ids, and group > membership lists, in the standard traditional format. If absent, > user and/or group lookups will fail. > > * `/etc/shadow` - text file containing shadow password hashes for some > or all users. > what about /etc/tcb/.. that can be used optionally? > * `/etc/resolv.conf` - text file providing addresses of nameservers to > be used for DNS lookups. If absent, DNS requests will be sent to the > loopback address and will fail unless the host has its own > nameserver. > > * `/etc/hosts` - text file mapping hostnames to IP addresses. > > * `/etc/services` - text file mapping network service names to port > numbers. > > * `/usr/share/zoneinfo`, `/share/zoneinfo`, and `/etc/zoneinfo` - > directories searched for time zone files when the `TZ` environment > variable is set to a relative pathname. > > * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the > "program interpreter" specified in the program's headers - if > present, this will be processed as a text file containing the shared > library search path, with components delimited by newlines or > colons. If absent, a default path of > `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by > static-linked programs. there is the legacy getusershell api which uses /etc/shells mntent.h defines MOUNTED as /etc/mtab but that's not hardcoded into the libc > Part III - Library Usage for Development > ---------------------------------------- > > ### Compiler Support > > All public interfaces in musl, at both the header file and library > level, are intended to be mostly compatible with any C99, C11, or C++ > compiler targetting the same CPU architecture and ABI musl was built > for. C89 compilers are also supported provided that they accept the > `long long` type as an extension. A few public header files do, stdint.h uses L' ' wide char literal which is technically not c89 but c94/amd1 there is another minor c89 issue (at least in math.h): in c89 struct/union/array members cannot be initialized from a function argument directly and gcc warns about them with -ansi -pedantic (it also warns about long long so it's not a big deal, but i can fix this) > however, require compiler-specific extensions in order to provide the > mandated standard features: > > * `complex.h` requires `1.0fi` to be accepted as a constant expression > suitable for defining `_Complex_I`. > > * `tgmath.h` requires the `__typeof__` extension. > > * FIXME: is this list complete? stdarg.h requires __builtin_va_* (+headers that use va_list) NAN in math.h with fenv access support requires __builtin_nanf("") (in theory if fenv access is on then 0/0.0f raises the invalid flag at runtime except when used in static initializers) for pedants stddef requires __builtin_offsetof > * `_GNU_SOURCE` (or `_ALL_SOURCE`) > > Adds everything above, plus interfaces modelef after GNU libc typo > extensions and interfaces for making use of Linux-specific > features. > > The specific extensions provided by the nonstandard feature test > macros are documented in subsequent sections of this manual. > ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Third draft of musl documentation/manual 2013-09-06 15:14 ` Szabolcs Nagy @ 2013-09-06 15:47 ` Rich Felker 2013-09-07 14:50 ` Szabolcs Nagy 0 siblings, 1 reply; 19+ messages in thread From: Rich Felker @ 2013-09-06 15:47 UTC (permalink / raw) To: musl On Fri, Sep 06, 2013 at 05:14:56PM +0200, Szabolcs Nagy wrote: > * Rich Felker <dalias@aerifal.cx> [2013-09-06 00:20:43 -0400]: > > document, but POSIX has a lot more things which are specified as > > implementation-defined for which I don't have such a nice checklist. > > Ideas on how to build one would be great. > > i would download the htmls from > > http://pubs.opengroup.org/onlinepubs/9699919799/download/index.html > > and then > > grep -r --color=always -C2 -i implementation-defined basedefs/ functions/ |less -R Probably need to make that merely "implementation-" to catch cases broken by hyphenation, unless the html avoids using such line breaks. > > ### Supported Targets > > > > * i386 > > * Requires support or kernel emulation of `cmpxchg` instruction, > > introduced on the 80486 > > maybe note that both i387 and sse fpus are supported Yes but it's not part of the ABI. > > #### Compiler wrapper > > > > Included with musl is a wrapper script `musl-gcc` which can be used > > with an existing GCC compiler toolchain to build programs using musl. > > If installed, the script itself is located at `$(bindir)/musl-gcc`, > > and a supporting GCC specs file it uses is located at > > `$(libdir)/musl-gcc.specs`. > > note that it uses ${REALGCC:-gcc}, not the compiler configured to > build musl I'll probably add an additional usage section for the wrapper. > > ### Filesystem Layout Dependencies > > > > musl aims to avoid imposing filesystem policy; however, the following > > minimal set of filesystems dependencies must be met in order for > > programs using musl to function correctly: > > > > * `/dev/null` - device node, required by POSIX > > > > * `/dev/tty` - device node, required by POSIX > > > > * `/tmp` - required by POSIX to exist as a directory, and used by > > various temporary file creation functions. > > > > * `/bin/sh` - an executable file providing as POSIX-conforming shell > > > > * `/proc` - must be a mount point for Linux procfs or a symlink to > > such. Several functions such as realpath, fexecve, and a number of > > the "at" functions added in POSIX 2008 need access to /proc to > > function correctly. > > > > While some programs may operate correctly even without some or all of > > the above, musl's behavior in their absence is unspecified. > > > > i would still add a note about the hardcoded program interpreter path > in dynamically linked elf binaries that is required so the kernel can > execute those binaries > (referring back to the dynamic linking section about ld-musl-* and libc.so) > (arguably this is not a musl runtime fs layout dependency, you can > interpret the binary as './libc.so name', but is worth mentioning here) Yes, I just forgot to add it. > > ### Additional Pathnames Used > > > > * `/dev/log` - a UNIX domain socket to which the `syslog()` interface > > sends log messages. If absent or inaccessible, log messages will be > > discarded. > > > > * `/dev/shm` - a directory; should have permissions 01777. If absent, > > POSIX shared memory and named semaphore interfaces will fail; > > programs not using these features will be unaffected. > > > > * `/dev/ptmx` and `/dev/pts` - device node and devpts filesystem mount > > point, respectively. If absent or inaccessible, `posix_openpt()` and > > `openpty()` will fail. > > > > * `/etc/passwd` and `/etc/group` - text files containing the user and > > group databases, mappings between names and numeric ids, and group > > membership lists, in the standard traditional format. If absent, > > user and/or group lookups will fail. > > > > * `/etc/shadow` - text file containing shadow password hashes for some > > or all users. > > what about /etc/tcb/.. that can be used optionally? Yes, this should still be added too. > > * `/etc/resolv.conf` - text file providing addresses of nameservers to > > be used for DNS lookups. If absent, DNS requests will be sent to the > > loopback address and will fail unless the host has its own > > nameserver. > > > > * `/etc/hosts` - text file mapping hostnames to IP addresses. > > > > * `/etc/services` - text file mapping network service names to port > > numbers. > > > > * `/usr/share/zoneinfo`, `/share/zoneinfo`, and `/etc/zoneinfo` - > > directories searched for time zone files when the `TZ` environment > > variable is set to a relative pathname. > > > > * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the > > "program interpreter" specified in the program's headers - if > > present, this will be processed as a text file containing the shared > > library search path, with components delimited by newlines or > > colons. If absent, a default path of > > `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by > > static-linked programs. > > there is the legacy getusershell api which uses /etc/shells This could be added as well. > mntent.h defines MOUNTED as /etc/mtab but that's not hardcoded > into the libc Probably doesn't need to be included. This is an app issue. > > Part III - Library Usage for Development > > ---------------------------------------- > > > > ### Compiler Support > > > > All public interfaces in musl, at both the header file and library > > level, are intended to be mostly compatible with any C99, C11, or C++ > > compiler targetting the same CPU architecture and ABI musl was built > > for. C89 compilers are also supported provided that they accept the > > `long long` type as an extension. A few public header files do, > > stdint.h uses L' ' wide char literal which is technically > not c89 but c94/amd1 OK, noted. > there is another minor c89 issue (at least in math.h): > in c89 struct/union/array members cannot be initialized from a function > argument directly and gcc warns about them with -ansi -pedantic > (it also warns about long long so it's not a big deal, but i > can fix this) Do you mean the initializers can't be non-constant-expressions? Or is the issue more subtle? > > however, require compiler-specific extensions in order to provide the > > mandated standard features: > > > > * `complex.h` requires `1.0fi` to be accepted as a constant expression > > suitable for defining `_Complex_I`. > > > > * `tgmath.h` requires the `__typeof__` extension. > > > > * FIXME: is this list complete? > > stdarg.h requires __builtin_va_* (+headers that use va_list) Yes. Thanks. > NAN in math.h with fenv access support requires __builtin_nanf("") > (in theory if fenv access is on then 0/0.0f raises the invalid > flag at runtime except when used in static initializers) Use of __builtin_nanf("") is conditional on __GNUC__. By default, 0.0f/0.0f is used. > for pedants stddef requires __builtin_offsetof This also has a non-__GNUC__ fallback, but it may fail to be a constant expression, yielding slightly incorrect semantics. > > * `_GNU_SOURCE` (or `_ALL_SOURCE`) > > > > Adds everything above, plus interfaces modelef after GNU libc > > typo You're the third one to report it. :-) Rich ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Third draft of musl documentation/manual 2013-09-06 15:47 ` Rich Felker @ 2013-09-07 14:50 ` Szabolcs Nagy 0 siblings, 0 replies; 19+ messages in thread From: Szabolcs Nagy @ 2013-09-07 14:50 UTC (permalink / raw) To: musl * Rich Felker <dalias@aerifal.cx> [2013-09-06 11:47:30 -0400]: > > there is another minor c89 issue (at least in math.h): > > in c89 struct/union/array members cannot be initialized from a function > > argument directly and gcc warns about them with -ansi -pedantic > > (it also warns about long long so it's not a big deal, but i > > can fix this) > > Do you mean the initializers can't be non-constant-expressions? Or is > the issue more subtle? c89 3.5.7: "All the expressions in an initializer for an object that has static storage duration or in an initializer list for an object that has aggregate or union type shall be constant expressions." only -pedantic warns for this though (which also warns for long long unless __extension__ is used like in the glibc headers) > > NAN in math.h with fenv access support requires __builtin_nanf("") > > (in theory if fenv access is on then 0/0.0f raises the invalid > > flag at runtime except when used in static initializers) > > Use of __builtin_nanf("") is conditional on __GNUC__. By default, > 0.0f/0.0f is used. > > > for pedants stddef requires __builtin_offsetof > > This also has a non-__GNUC__ fallback, but it may fail to be a > constant expression, yielding slightly incorrect semantics. > my point was that the fallbacks are not strictly correct (but yes they work well enough in practice) > > > Adds everything above, plus interfaces modelef after GNU libc > > > > typo > > You're the third one to report it. :-) good, that means at least 3ppl read it thus far :) ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Third draft of musl documentation/manual 2013-09-06 4:20 ` Third " Rich Felker 2013-09-06 15:14 ` Szabolcs Nagy @ 2013-09-08 19:22 ` Ivan Kanakarakis 2013-09-09 11:42 ` AW: " Bortis Kevin 2013-09-09 12:09 ` Bortis Kevin 3 siblings, 0 replies; 19+ messages in thread From: Ivan Kanakarakis @ 2013-09-08 19:22 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 666 bytes --] Maybe there should be a Legal/License section mentioning the release license etc On 6 September 2013 07:20, Rich Felker <dalias@aerifal.cx> wrote: > Here is the third version of the docs/manual draft, with a lot more > content filled in, some of it based on previous comments. The biggest > task left is filling in all the implementation-defined behavior. For > plain C functions, J.3.12 gives a nice checklist of things to > document, but POSIX has a lot more things which are specified as > implementation-defined for which I don't have such a nice checklist. > Ideas on how to build one would be great. > > Rich > -- *Ivan c00kiemon5ter Kanakarakis* >:3 [-- Attachment #2: Type: text/html, Size: 1305 bytes --] ^ permalink raw reply [flat|nested] 19+ messages in thread
* AW: Third draft of musl documentation/manual 2013-09-06 4:20 ` Third " Rich Felker 2013-09-06 15:14 ` Szabolcs Nagy 2013-09-08 19:22 ` Ivan Kanakarakis @ 2013-09-09 11:42 ` Bortis Kevin 2013-09-09 12:09 ` Bortis Kevin 3 siblings, 0 replies; 19+ messages in thread From: Bortis Kevin @ 2013-09-09 11:42 UTC (permalink / raw) To: musl [-- Attachment #1: Type: text/plain, Size: 1022 bytes --] Added a triple, cpu and ARCH table to the manual. This should help the users to guess the sugested triple and most importantly the config name under /etc/ld-musl-$(ARCH).path and of course the linker name. For architectures other than i386, x86_64 and arm, someone should look over the values. Regards Kevin > -----Ursprüngliche Nachricht----- > Von: Rich Felker [mailto:dalias@aerifal.cx] > Gesendet: Freitag, 6. September 2013 06:21 > An: musl@lists.openwall.com > Betreff: Re: [musl] Third draft of musl documentation/manual > > Here is the third version of the docs/manual draft, with a lot more content > filled in, some of it based on previous comments. The biggest task left is > filling in all the implementation-defined behavior. For plain C functions, > J.3.12 gives a nice checklist of things to document, but POSIX has a lot more > things which are specified as implementation-defined for which I don't have > such a nice checklist. > Ideas on how to build one would be great. > > Rich [-- Attachment #2: manual6-arch-tables.diff --] [-- Type: application/octet-stream, Size: 1134 bytes --] --- manual6.txt 2013-09-09 13:36:09.695361698 +0200 +++ manual6-mod.txt 2013-09-09 13:36:32.186602487 +0200 @@ -65,6 +65,25 @@ * Big-endian default; little-endian variants also supported * Soft-float +#### Sugested Triple Table + +<Triple> musl <inker name> + +<CPU>-linux-musl ld-musl-<arch>.so.1 +arm-linux-musleabi ld-musl-arm.so.1 +arm-linux-musleabihf ld-musl-armhf.so.1 + +#### CPU Table + +<CPU> musl <ARCH> + +i386 i386 +x86_64 x86_64 +arm arm, armhf +mips mips +mipsel mipsel +powerpc power +microblaze microblaze,microblazeel ### Build and Installation @@ -161,9 +180,6 @@ necessary to create a path file for the dynamic linker. See `../etc/ld-musl-$(ARCH).path` in the next section of the manual. - - - ### Installed Components In the following, `$(syslibdir)`, `$(includedir)`, and `$(libdir)` @@ -414,7 +430,7 @@ * `_GNU_SOURCE` (or `_ALL_SOURCE`) - Adds everything above, plus interfaces modelef after GNU libc + Adds everything above, plus interfaces modeled after GNU libc extensions and interfaces for making use of Linux-specific features. ^ permalink raw reply [flat|nested] 19+ messages in thread
* AW: Third draft of musl documentation/manual 2013-09-06 4:20 ` Third " Rich Felker ` (2 preceding siblings ...) 2013-09-09 11:42 ` AW: " Bortis Kevin @ 2013-09-09 12:09 ` Bortis Kevin 2013-09-09 15:49 ` Rich Felker 3 siblings, 1 reply; 19+ messages in thread From: Bortis Kevin @ 2013-09-09 12:09 UTC (permalink / raw) To: musl What happens if /etc/ld-musl-$(ARCH).path is: * present, but not readable? * present, but corrupt? Does it always fallback to the default "/lib:/usr/local/lib:/usr/lib"? The manual text does not describe these cases. ==================================== * `../etc/ld-musl-$(ARCH).path`, taken relative to the location of the "program interpreter" specified in the program's headers - if present, this will be processed as a text file containing the shared library search path, with components delimited by newlines or colons. If absent, a default path of `"/lib:/usr/local/lib:/usr/lib"` will be used. Not used by static-linked programs. ==================================== Kevin > -----Ursprüngliche Nachricht----- > Von: Rich Felker [mailto:dalias@aerifal.cx] > Gesendet: Freitag, 6. September 2013 06:21 > An: musl@lists.openwall.com > Betreff: Re: [musl] Third draft of musl documentation/manual > > Here is the third version of the docs/manual draft, with a lot more content > filled in, some of it based on previous comments. The biggest task left is > filling in all the implementation-defined behavior. For plain C functions, > J.3.12 gives a nice checklist of things to document, but POSIX has a lot more > things which are specified as implementation-defined for which I don't have > such a nice checklist. > Ideas on how to build one would be great. > > Rich ^ permalink raw reply [flat|nested] 19+ messages in thread
* Re: Third draft of musl documentation/manual 2013-09-09 12:09 ` Bortis Kevin @ 2013-09-09 15:49 ` Rich Felker 0 siblings, 0 replies; 19+ messages in thread From: Rich Felker @ 2013-09-09 15:49 UTC (permalink / raw) To: musl On Mon, Sep 09, 2013 at 02:09:46PM +0200, Bortis Kevin wrote: > What happens if /etc/ld-musl-$(ARCH).path is: > * present, but not readable? > * present, but corrupt? > > Does it always fallback to the default "/lib:/usr/local/lib:/usr/lib"? The present behavior is that it falls back on any failure to open the file, but if the file is opened and reading subsequently fails, no fallback is used. The intent was to avoid situations where resource-exhaustion attacks could cause the wrong path to be used. I suspect all fopen failures except ENOENT (e.g. ENFILE, EMFILE, ...) should also suppress the fallback. > The manual text does not describe these cases. "Present but not readable" should probably be documented and have a defined behavior. "Present but corrupt" is hard to define. In a sense, any file content is "valid" except possibly embedded null bytes, but most will not result in meaningful path searches. I'm doubtful that it's useful to document or promise any reasonable behavior when the file contains junk. Thanks for the comments. Rich x ^ permalink raw reply [flat|nested] 19+ messages in thread
end of thread, other threads:[~2013-09-09 15:49 UTC | newest] Thread overview: 19+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2013-09-01 4:41 Initial draft of musl documentation/manual Rich Felker 2013-09-01 9:43 ` Luca Barbato 2013-09-01 16:57 ` Rich Felker 2013-09-02 0:08 ` Luca Barbato 2013-09-02 7:12 ` Ivan Kanakarakis 2013-09-01 13:45 ` Vasily Kulikov 2013-09-01 16:54 ` Rich Felker 2013-09-02 7:11 ` Christian Wiese 2013-09-06 1:12 ` Second " Rich Felker 2013-09-06 2:41 ` Szabolcs Nagy 2013-09-06 2:57 ` Rich Felker 2013-09-06 4:20 ` Third " Rich Felker 2013-09-06 15:14 ` Szabolcs Nagy 2013-09-06 15:47 ` Rich Felker 2013-09-07 14:50 ` Szabolcs Nagy 2013-09-08 19:22 ` Ivan Kanakarakis 2013-09-09 11:42 ` AW: " Bortis Kevin 2013-09-09 12:09 ` Bortis Kevin 2013-09-09 15:49 ` Rich Felker
Code repositories for project(s) associated with this public inbox https://git.vuxu.org/mirror/musl/ This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).