Design idea for subarchs/abivariants

Design idea for subarchs/abivariants

[Rich Felker]

The problem: how to have subarch variants on the bits headers and asm
for a given arch without ugly build system complications or massive
duplication of files.

Proposed solution:

1. $(ARCH)$(SUBARCH)/%.s is searched in addition to $(ARCH)/%.s for
arch-specific files replacing general %.c files in the src tree. This
would allow src/fenv/arm-hf/%.s and src/setjmp/mips-sf/%.s files, for
example.

2. Making bits/endian.h, bits/fenv.h, and any other bits headers that
need to vary per-subarch into generated headers. I think this could be
accomplished with a single rule for generating bits/%.h from
bits/%.h.sh the way alltypes.h is generated now. The full
$(ARCH)$(ENDIAN)$(SUBARCH) could be passed to the script as $1,
allowing simple shell logic to choose the right version to output.

3. Having configure derive separate $(ARCH), $(ENDIAN), and $(SUBARCH)
from the gcc-style machine strings or combined musl-format arch string.

This sounds like it will be minimally invasive and meet the
requirements for supporting subarch/abi-variants.

Comments?

Rich

Re: Design idea for subarchs/abivariants

[Isaac Dunham]

On Wed, 28 Nov 2012 14:26:18 -0500
Rich Felker <dalias@aerifal.cx> wrote:

> The problem: how to have subarch variants on the bits headers and asm
> for a given arch without ugly build system complications or massive
> duplication of files.
> 
> Proposed solution:
> 
> 1. $(ARCH)$(SUBARCH)/%.s is searched in addition to $(ARCH)/%.s for
> arch-specific files replacing general %.c files in the src tree. This
> would allow src/fenv/arm-hf/%.s and src/setjmp/mips-sf/%.s files, for
> example.

> 2. Making bits/endian.h, bits/fenv.h, and any other bits headers that
> need to vary per-subarch into generated headers. 
<snip>
> 3. Having configure derive separate $(ARCH), $(ENDIAN), and $(SUBARCH)
> from the gcc-style machine strings or combined musl-format arch string.
> Comments?
Seems fairly clean to me, though I'd been thinking of a variant of 1: $(ARCH)/$(SUBARCH)/%.s (on the premise that anything that gets used for an arch should be in one place).
But now that I think about it, I'm not certain that's a meaningful advantage, and with your approach, */<file>.s is going to get every appropriate file.


By the way, while we're on the makefile:
in the past, I've run into issues with cycles like:
make
<forget to check for builds>
git pull # or git checkout ....
make clean # doesn't handle files that were moved/deleted
make # link breaks, due to remaining oject files

I usually do this to cleanup in such cases: 
find src arch -name '*.*o' |xargs rm
Would adding some similar logic be sensible (at least for the distclean target)?


-- 
Isaac Dunham <idunham@lavabit.com>

Re: Design idea for subarchs/abivariants

[Szabolcs Nagy]

* Isaac Dunham <idunham@lavabit.com> [2012-11-28 12:03:41 -0800]:
> I usually do this to cleanup in such cases: 
> find src arch -name '*.*o' |xargs rm
> Would adding some similar logic be sensible (at least for the distclean target)?
> 

rm -f src/*/*o


actually i have

all:
mclean:
	rm -f src/math/*o

in my config.mak so i can easily clean math
and then rebuild with different flags etc

Re: Design idea for subarchs/abivariants

[Rich Felker]

On Wed, Nov 28, 2012 at 12:03:41PM -0800, Isaac Dunham wrote:
> On Wed, 28 Nov 2012 14:26:18 -0500
> Rich Felker <dalias@aerifal.cx> wrote:
> 
> > The problem: how to have subarch variants on the bits headers and asm
> > for a given arch without ugly build system complications or massive
> > duplication of files.
> > 
> > Proposed solution:
> > 
> > 1. $(ARCH)$(SUBARCH)/%.s is searched in addition to $(ARCH)/%.s for
> > arch-specific files replacing general %.c files in the src tree. This
> > would allow src/fenv/arm-hf/%.s and src/setjmp/mips-sf/%.s files, for
> > example.
> 
> > 2. Making bits/endian.h, bits/fenv.h, and any other bits headers that
> > need to vary per-subarch into generated headers. 
> <snip>
> > 3. Having configure derive separate $(ARCH), $(ENDIAN), and $(SUBARCH)
> > from the gcc-style machine strings or combined musl-format arch string.
> > Comments?
> Seems fairly clean to me, though I'd been thinking of a variant of
> 1: $(ARCH)/$(SUBARCH)/%.s (on the premise that anything that gets
> used for an arch should be in one place).
> But now that I think about it, I'm not certain that's a meaningful
> advantage, and with your approach, */<file>.s is going to get every
> appropriate file.

I don't have a lot of preference one way or the other. I'd probably
use $(ARCH)/$(ARCH)$(SUBARCH) though, since $(SUBARCH) is likely to be
an awkward string like "-sf" that's ugly and error-prone as a
directory name (easily misinterpreted as an option).

> By the way, while we're on the makefile:
> in the past, I've run into issues with cycles like:
> make
> <forget to check for builds>
> git pull # or git checkout ....
> make clean # doesn't handle files that were moved/deleted
> make # link breaks, due to remaining oject files

This is not possible. Spurious *.o files lying around do not get used
whatsoever in the build process, for the exact same reason that they
do not yet deleted by "make clean".

> I usually do this to cleanup in such cases: 
> find src arch -name '*.*o' |xargs rm

There are no .o files under arch. It's purely headers.

> Would adding some similar logic be sensible (at least for the
> distclean target)?

I'm a little bit hesitant to delete files that we didn't create. This
could lead to deleting a file that somebody did not want deleted,
which I really don't want to be responsible for..

Rich

Re: Design idea for subarchs/abivariants

[John Spencer]

On 11/28/2012 08:26 PM, Rich Felker wrote:
> The problem: how to have subarch variants on the bits headers and asm
> for a given arch without ugly build system complications or massive
> duplication of files.
>
> Proposed solution:
>
> 1. $(ARCH)$(SUBARCH)/%.s is searched in addition to $(ARCH)/%.s for
> arch-specific files replacing general %.c files in the src tree. This
> would allow src/fenv/arm-hf/%.s and src/setjmp/mips-sf/%.s files, for
> example.
that sounds good.
> 2. Making bits/endian.h, bits/fenv.h, and any other bits headers that
> need to vary per-subarch into generated headers. I think this could be
> accomplished with a single rule for generating bits/%.h from
> bits/%.h.sh the way alltypes.h is generated now. The full
> $(ARCH)$(ENDIAN)$(SUBARCH) could be passed to the script as $1,
> allowing simple shell logic to choose the right version to output.
as for endian, the current way seems to work well. we can rely on what 
gcc provides
(even if it doesnt supply the endianness directly, it has something like 
__mipsel__ set,
which we currently already use).
and apart from endian.h, there doesn't seem to be any
need for endian-specific code.
> 3. Having configure derive separate $(ARCH), $(ENDIAN), and $(SUBARCH)
> from the gcc-style machine strings or combined musl-format arch string.
>
> This sounds like it will be minimally invasive and meet the
> requirements for supporting subarch/abi-variants.
as long as it works automatically (i.e. by querying gcc), cool.
but having to pass something like --target=musl-armeabi-v4-el oslt is ugly.

Re: Design idea for subarchs/abivariants

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2012-11-28 14:26:18 -0500]:
> 2. Making bits/endian.h, bits/fenv.h, and any other bits headers that
> need to vary per-subarch into generated headers. I think this could be
> accomplished with a single rule for generating bits/%.h from
> bits/%.h.sh the way alltypes.h is generated now. The full
> $(ARCH)$(ENDIAN)$(SUBARCH) could be passed to the script as $1,
> allowing simple shell logic to choose the right version to output.

i'm not sure about having many generated headers..

it could be a single generated bits/config.h with

#define __FPU 1
#define __BYTEORDER 1234

and other bits/*.h could use these macros

this way the generator is probably simpler,
the drawback is that the cpp may need to do more
work if bits/*.h have many conditional #if __FPU etc

Re: Design idea for subarchs/abivariants

[Rich Felker]

On Wed, Nov 28, 2012 at 11:38:02PM +0100, Szabolcs Nagy wrote:
> * Rich Felker <dalias@aerifal.cx> [2012-11-28 14:26:18 -0500]:
> > 2. Making bits/endian.h, bits/fenv.h, and any other bits headers that
> > need to vary per-subarch into generated headers. I think this could be
> > accomplished with a single rule for generating bits/%.h from
> > bits/%.h.sh the way alltypes.h is generated now. The full
> > $(ARCH)$(ENDIAN)$(SUBARCH) could be passed to the script as $1,
> > allowing simple shell logic to choose the right version to output.
> 
> i'm not sure about having many generated headers..
> 
> it could be a single generated bits/config.h with
> 
> #define __FPU 1
> #define __BYTEORDER 1234
> 
> and other bits/*.h could use these macros

Yes, I thought about this too and even started writing the email that
way before deleting what I wrote and rewriting it. I was trying to
avoid more nested and conditional inclusion, but I don't think it
would actually be too costly to do it the way you proposed.

> this way the generator is probably simpler,
> the drawback is that the cpp may need to do more
> work if bits/*.h have many conditional #if __FPU etc

I would at least use macro names that are less likely to conflict with
things a compiler might predefine.

Rich

Re: Design idea for subarchs/abivariants

[Kurt H Maier]

On Wed, Nov 28, 2012 at 07:10:42PM -0500, Rich Felker wrote:
> Yes, I thought about this too and even started writing the email that
> way before deleting what I wrote and rewriting it. I was trying to
> avoid more nested and conditional inclusion, but I don't think it
> would actually be too costly to do it the way you proposed.

I'm asking this to be educated, rather than to make a point:  why is a
system like this preferable to the way plan9 libc is written, which as
far as I can tell uses no ifdef at all?

Re: Design idea for subarchs/abivariants

[Rich Felker]

On Wed, Nov 28, 2012 at 07:22:51PM -0500, Kurt H Maier wrote:
> On Wed, Nov 28, 2012 at 07:10:42PM -0500, Rich Felker wrote:
> > Yes, I thought about this too and even started writing the email that
> > way before deleting what I wrote and rewriting it. I was trying to
> > avoid more nested and conditional inclusion, but I don't think it
> > would actually be too costly to do it the way you proposed.
> 
> I'm asking this to be educated, rather than to make a point:  why is a
> system like this preferable to the way plan9 libc is written, which as
> far as I can tell uses no ifdef at all?

Right now, musl uses separate bits headers for each arch rather than
trying to share them between archs with #ifdef hackery. I agree this
is much cleaner. But it's been polluted slightly by the fact that a
few archs have both little and big endian variants, and the desire to
avoid duplicating the whole arch for just that one variation.

Now, the need to support more variants for some of these (like mips
softfloat) is making the situation even worse.

Using generated headers like I originally proposed would of course
allow us to avoid #ifdef hackery, and even allow us to eliminate what
already exists in endian.h.

Rich

Re: Design idea for subarchs/abivariants

[Szabolcs Nagy]

* Kurt H Maier <khm-lists@intma.in> [2012-11-28 19:22:51 -0500]:
> I'm asking this to be educated, rather than to make a point:  why is a
> system like this preferable to the way plan9 libc is written, which as
> far as I can tell uses no ifdef at all?

i looked at this

it seems the plan9 and musl approach is similar
in the sense that they avoid unnecessary ifdefs
(putting arch specific stuff in separate dir etc, plan9
has union mount, musl uses a bits symlink in case of
headers)

they both use ifdefs for feature test macros
(obviously it's much less needed in plan9 but see the
ape implementation and it has some, musl needs a lot)

they both use ifdefs for typedef and include guards
(plan9 only uses them in ape, this is unavioidable in a
posix libc)

musl has to use ifdefs for some compiler specific features
(plan9 don't need this, they have their own compiler)

now musl will need to use ifdefs for some arch parameters
(endianness, soft vs hard float, long double,..)
plan9 libc does not need these because they don't implement
such things (there is no soft float or long double support,
no fenv, no endian dependent api etc)

but plan9 has ifdefs for arch specific options as well
(eg in driver code) just not in the libc