ideas for build system changes, bits refactoring

ideas for build system changes, bits refactoring

[Rich Felker]

I'd like to start a list of ideas and discussion for changes we could
make to the build system and arch bits. This has been on the agenda a
long time but it's going to be more important as we get more ports,
both as a way of managing complexity and as a way of fighting source
tree bloat. Here are some ideas I have so far:

1. Generate all installable include files, even if just by copying, as
part of the build. If nothing else, this will ensure that "make clean"
followed by "make install" overwrites any (non-future-dated)
preexisting files at the install destination; right now, that might
fail to happen if changes were made at the install destination and the
source timestamp is older. I think this will also make it easier to
add out-of-tree build since there won't need to be separate logic for
generated headers in-tree vs out-of-tree. It will also give us the
option (not sure if we should take it) to merge bits into the main
headers rather than having a bits dir under include.

2. Factor arch bits (types, struct layouts, constant values, etc.) as
a sequence of overlays: a generic base, several common families like
32-bit, 64-bit, 64-bit+ILP32, ld-64, ld-80, ld-128, and finally
arch-specific bits. The latter should be nearly empty for most archs.

3. Moving arch-specific build logic out of the configure script and
into a makefile fragment in the arch dir. This should support the
long-term goal of reducing/eliminating the work configure does and
moving it to declarative rules in make. It also isolates
port-specific logic with the port rather than hard-coding it in a
shared script.

4. Eliminate the duplication of SYS_* and __NR_* in syscall.h bits by
auto-generating one from the other as part of the build process.


Rich

Re: ideas for build system changes, bits refactoring

[Bobby Bingham]

[-- Attachment #1: Type: text/plain, Size: 2788 bytes --]

On Wed, Mar 11, 2015 at 11:59:54PM -0400, Rich Felker wrote:
> I'd like to start a list of ideas and discussion for changes we could
> make to the build system and arch bits. This has been on the agenda a
> long time but it's going to be more important as we get more ports,
> both as a way of managing complexity and as a way of fighting source
> tree bloat. Here are some ideas I have so far:
>
> 1. Generate all installable include files, even if just by copying, as
> part of the build. If nothing else, this will ensure that "make clean"
> followed by "make install" overwrites any (non-future-dated)
> preexisting files at the install destination; right now, that might
> fail to happen if changes were made at the install destination and the
> source timestamp is older. I think this will also make it easier to
> add out-of-tree build since there won't need to be separate logic for
> generated headers in-tree vs out-of-tree. It will also give us the
> option (not sure if we should take it) to merge bits into the main
> headers rather than having a bits dir under include.
>
> 2. Factor arch bits (types, struct layouts, constant values, etc.) as
> a sequence of overlays: a generic base, several common families like
> 32-bit, 64-bit, 64-bit+ILP32, ld-64, ld-80, ld-128, and finally
> arch-specific bits. The latter should be nearly empty for most archs.

Do you mean that a final bits header would come from one of these
overlays that hadn't been overridden by another overlay, or would be a
composite built from pieces provided by the different overlays?

For example, powerpc/bits/errno.h is nearly identical to the file used
on most other architectures, just with a different value of EDEADLOCK.
Would the powerpc arch-specific errno.h need to redefine all of the
values, or would it be able to inherit most of the values from the
generic base and just override EDEADLOCK?

>
> 3. Moving arch-specific build logic out of the configure script and
> into a makefile fragment in the arch dir. This should support the
> long-term goal of reducing/eliminating the work configure does and
> moving it to declarative rules in make. It also isolates
> port-specific logic with the port rather than hard-coding it in a
> shared script.
>
> 4. Eliminate the duplication of SYS_* and __NR_* in syscall.h bits by
> auto-generating one from the other as part of the build process.
>

5. This may be orthogonal to the rest of these ideas, but what about
providing thin wrappers for the syscalls used by musl itself, similar to
the __sys_open* macros already provided by internal/syscall.h?  I think
it would help for bare metal targets to be able to implement one
individual function per syscall, rather than needing to implement a
whole syscall dispatcher.

>
> Rich

--
Bobby Bingham

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: ideas for build system changes, bits refactoring

[Rich Felker]

On Sun, Mar 15, 2015 at 06:47:56PM -0500, Bobby Bingham wrote:
> > 2. Factor arch bits (types, struct layouts, constant values, etc.) as
> > a sequence of overlays: a generic base, several common families like
> > 32-bit, 64-bit, 64-bit+ILP32, ld-64, ld-80, ld-128, and finally
> > arch-specific bits. The latter should be nearly empty for most archs.
> 
> Do you mean that a final bits header would come from one of these
> overlays that hadn't been overridden by another overlay, or would be a
> composite built from pieces provided by the different overlays?
> 
> For example, powerpc/bits/errno.h is nearly identical to the file used
> on most other architectures, just with a different value of EDEADLOCK.
> Would the powerpc arch-specific errno.h need to redefine all of the
> values, or would it be able to inherit most of the values from the
> generic base and just override EDEADLOCK?

I don't have a plan in mind for this yet. Obviously there are
simplifying/deduplication benefits to being able to make small
replacements like this, but in order to make sense there needs to be a
net savings still even after we implement whatever part of the build
script would be responsible for merging them. I'm not sure how that
would work.

This could of course be done in multiple phases, where the first phase
is just cp'ing the 'best match' overlay and we later add fancier
merging.

> 5. This may be orthogonal to the rest of these ideas, but what about
> providing thin wrappers for the syscalls used by musl itself, similar to
> the __sys_open* macros already provided by internal/syscall.h?  I think
> it would help for bare metal targets to be able to implement one
> individual function per syscall, rather than needing to implement a
> whole syscall dispatcher.

Yes. I'd still like to explore what changes users wanting to do
bare-metal stuff with musl would like to see. I agree it's sort of
orthogonal to the above.

Rich