Build system adjustments for subarchs

Build system adjustments for subarchs

[Rich Felker]

Hi,

I'm looking for some ideas on a problem in the build system: how to
cleanly share asm between subarchs. The problem is this: ARM, and
possibly other archs in the future, has multiple dimensions of
subarch: little-endian(default) vs big-endian, and
fpu-agnostic(default) vs hardfloat. The asm requirements are:

- MOST asm is shared by all subarchs.

- A small amount of asm (for now, just memcpy) is endian-specific but
  has nothing to do with floating point ABI.

- A fairly significant amount of asm in the future will be hard-float
  specific (optimized math functions and floating point environment)
  but has nothing to do with endianness.

Until commit 90d77722511ad5e9b748f69f42c5b2a8467fa049, asm was shared
by all subarchs; there was no ability to have subarch-specific
overrides. At least one person (I can't remember who right off)
suggested that instead of more build system logic, we should switch to
preprocessed asm files (.S) so that a single asm file could include
the logic to support all subarchs, but that was not a viable solution,
because what I needed was a way to write asm for one subarch that
doesn't interfere with the fallback C source file getting used for
other subarchs.

The current system searches for arch-specific asm in this order:

1. $(ARCH)$(ASMSUBARCH)/%.s, where ASMSUBARCH for the default subarch
   is not blank but rather a unique suffix (for plain arm, it's "el").
   This allows having asm that applies only to the default subarch but
   not others.

2. $(ARCH)/%.s, for shared asm used by all subarchs.

3. %.c, the C fallback (which is empty for code that cannot be
   implemented at all in C).

Unfortunately, this still provides no way to include asm that's used
by both soft and hardfloat little-endian, or both little- and
big-endian hardfloat, without having, for example:

- armel/%.s and armhf/%.s as duplicate files or symlinked
- armhf/%.s and armebhf/%.s as duplicate files or symlinked

Duplicating code is ugly, and from what I've gathered, symlinks in git
repos are frowned upon. (They don't work at all on Windows, even
modern Windows versions that have symlinks, nor do they work on
Android if your source tree is on a FAT32-formatted SD card).

So, I'm looking for a better solution.

One idea is to replace ASMSUBARCH with 3 different variables, and the
makefile rules with 3 rules, allowing a fallback order of:

1. Matches in both dimensions
2. Matches in dimension 1
3. Matches in dimension 2
4. Generic asm for the arch
5. Default C code

What I don't like about this approach is that the logic for the names
to use in steps 1-3 has to go either in the configure script or the
makefile. The makefile is not intended to have this sort of arch
logic, but instead to derive the logic from the source tree structure.
The configure script is intended to have some arch logic, but it
should remain practical to use a hand-generated config.mak file, which
becomes more burdensome if you have to define multiple arcane subarch
vars.

Another idea, using the filesystem, would be to have .sub files in the
asm directories containing the filename of a different asm file to
substitute. This is essentially emulating symlinks, but I don't see a
good way to get make to do automated dependency tracking so that
modifying the pointed-to asm file causes the object file to be
rebuilt. I am exploring this approach further however in hopes that
there might be a way.

Any other ideas?

Rich

Re: Build system adjustments for subarchs

[Rich Felker]

On Tue, Aug 13, 2013 at 09:06:17PM -0400, Rich Felker wrote:
> Another idea, using the filesystem, would be to have .sub files in the
> asm directories containing the filename of a different asm file to
> substitute. This is essentially emulating symlinks, but I don't see a
> good way to get make to do automated dependency tracking so that
> modifying the pointed-to asm file causes the object file to be
> rebuilt. I am exploring this approach further however in hopes that
> there might be a way.
> 
> Any other ideas?

One further idea I sort of like:

-include arch/$(ARCH)/subarchs.mak

and coming up with a halfway-clean way, in subarchs.mak, to put rules
that override the asm files that get used for particular modules.
However, I'm not seeing a really clean way to do that. I don't want to
end up duplicating the commands for assembling into files other than
the main makefile.

Rich

Re: Build system adjustments for subarchs

[Luca Barbato]

On 14/08/13 03:06, Rich Felker wrote:
> Hi,
> 
> I'm looking for some ideas on a problem in the build system: how to
> cleanly share asm between subarchs. The problem is this: ARM, and
> possibly other archs in the future, has multiple dimensions of
> subarch: little-endian(default) vs big-endian, and
> fpu-agnostic(default) vs hardfloat. The asm requirements are:
> 
> - MOST asm is shared by all subarchs.
> 
> - A small amount of asm (for now, just memcpy) is endian-specific but
>   has nothing to do with floating point ABI.
> 
> - A fairly significant amount of asm in the future will be hard-float
>   specific (optimized math functions and floating point environment)
>   but has nothing to do with endianness.
> 
> Until commit 90d77722511ad5e9b748f69f42c5b2a8467fa049, asm was shared
> by all subarchs; there was no ability to have subarch-specific
> overrides. At least one person (I can't remember who right off)
> suggested that instead of more build system logic, we should switch to
> preprocessed asm files (.S) so that a single asm file could include
> the logic to support all subarchs, but that was not a viable solution,
> because what I needed was a way to write asm for one subarch that
> doesn't interfere with the fallback C source file getting used for
> other subarchs.
> 
> The current system searches for arch-specific asm in this order:
> 
> 1. $(ARCH)$(ASMSUBARCH)/%.s, where ASMSUBARCH for the default subarch
>    is not blank but rather a unique suffix (for plain arm, it's "el").
>    This allows having asm that applies only to the default subarch but
>    not others.
> 
> 2. $(ARCH)/%.s, for shared asm used by all subarchs.
> 
> 3. %.c, the C fallback (which is empty for code that cannot be
>    implemented at all in C).

> - armel/%.s and armhf/%.s as duplicate files or symlinked
> - armhf/%.s and armebhf/%.s as duplicate files or symlinked

I'd just explicitly group files by the most selective so

ARMEL+=${list of objects that work on el doesn't matter what}
ARMHF+=${list of objects that work on hf doesn't matter what}
ARMEB+=${list of objects that work on eb doesn't matter what}

ARMHFEL+=$(ARMEL) $(ARMHF) ${other stuff}
ARMHFEB+=$(ARMEB) $(ARMHF) ${different stuff}

OBJS+=$($(targetarch))

And give up on having automagic fallbacks.

lu

Re: Build system adjustments for subarchs

[Rich Felker]

On Wed, Aug 14, 2013 at 03:42:48AM +0200, Luca Barbato wrote:
> I'd just explicitly group files by the most selective so
> 
> ARMEL+=${list of objects that work on el doesn't matter what}
> ARMHF+=${list of objects that work on hf doesn't matter what}
> ARMEB+=${list of objects that work on eb doesn't matter what}
> 
> ARMHFEL+=$(ARMEL) $(ARMHF) ${other stuff}
> ARMHFEB+=$(ARMEB) $(ARMHF) ${different stuff}
> 
> OBJS+=$($(targetarch))
> 
> And give up on having automagic fallbacks.

The problem is that it's hard to integrate this with the existing
source-tree-based arch logic, which needs to know whether to build or
not to build the corresponding .c file. Just adding arch-specific
files won't do anything to turn off building of those .c files. It
also doesn't seem to scale well to multiple archs that need this sort
of logic...

Rich

Re: Build system adjustments for subarchs

[Rich Felker]

On Tue, Aug 13, 2013 at 09:06:17PM -0400, Rich Felker wrote:
> Another idea, using the filesystem, would be to have .sub files in the
> asm directories containing the filename of a different asm file to
> substitute. This is essentially emulating symlinks, but I don't see a
> good way to get make to do automated dependency tracking so that
> modifying the pointed-to asm file causes the object file to be
> rebuilt. I am exploring this approach further however in hopes that
> there might be a way.

Actually, this would work great if I could write an abstract rule that
says roughly: for each .o target %.o, add dependencies on %.c and
$(wildcard */%.s). This would make it so files with asm always get
rebuilt if either the base C file or any of the asm versions, for any
arch or subarch, has been modified. Obviously in some cases the
rebuild would be spurious, but such files are sufficiently small that
I don't care; the simplicity of a universal rule with no
arch-dependent dependency data to maintain is much more valuable.

Unfortunately, I don't know a way to do this with make, but I believe
there should be a way...

Rich

Re: Build system adjustments for subarchs

[Rich Felker]

On Tue, Aug 13, 2013 at 10:25:59PM -0400, Rich Felker wrote:
> On Tue, Aug 13, 2013 at 09:06:17PM -0400, Rich Felker wrote:
> > Another idea, using the filesystem, would be to have .sub files in the
> > asm directories containing the filename of a different asm file to
> > substitute. This is essentially emulating symlinks, but I don't see a
> > good way to get make to do automated dependency tracking so that
> > modifying the pointed-to asm file causes the object file to be
> > rebuilt. I am exploring this approach further however in hopes that
> > there might be a way.
> 
> Actually, this would work great if I could write an abstract rule that
> says roughly: for each .o target %.o, add dependencies on %.c and
> $(wildcard */%.s). This would make it so files with asm always get
> rebuilt if either the base C file or any of the asm versions, for any
> arch or subarch, has been modified. Obviously in some cases the
> rebuild would be spurious, but such files are sufficiently small that
> I don't care; the simplicity of a universal rule with no
> arch-dependent dependency data to maintain is much more valuable.
> 
> Unfortunately, I don't know a way to do this with make, but I believe
> there should be a way...

This seems to do it, though maybe there's a more elegant way:

define archrule =
$(dir $(patsubst %/,%,$(dir $(s))))$(notdir $(s:.s=.o)): $(s)
endef

$(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval $(call archrule,$(s))))

What this does is take every .s file src/foo/arch/bar.s which is under
any arch or subarch dir for the target arch, and manufacture rules of
the form:

src/foo/bar.o: src/foo/arch/bar.s

In particular, you might get:

src/string/memcpy.o: src/string/armel/memcpy.s
src/string/memcpy.o: src/string/armeb/memcpy.s

so that memcpy.o "depends" on all versions of the asm. Then, a
corresponding memcpy.sub file can control which version of the asm
actually gets compiled; the rule which compiles it can be based on the
.sub file rather than on any particular .s file.

I'm going to run some tests on arm, and it if works, I'll commit.

Rich

Re: Build system adjustments for subarchs

[Rich Felker]

On Tue, Aug 13, 2013 at 11:42:26PM -0400, Rich Felker wrote:
> > Actually, this would work great if I could write an abstract rule that
> > says roughly: for each .o target %.o, add dependencies on %.c and
> > $(wildcard */%.s). This would make it so files with asm always get
> > rebuilt if either the base C file or any of the asm versions, for any
> > arch or subarch, has been modified. Obviously in some cases the
> > rebuild would be spurious, but such files are sufficiently small that
> > I don't care; the simplicity of a universal rule with no
> > arch-dependent dependency data to maintain is much more valuable.
> > 
> > Unfortunately, I don't know a way to do this with make, but I believe
> > there should be a way...
> 
> This seems to do it, though maybe there's a more elegant way:
> 
> define archrule =
> $(dir $(patsubst %/,%,$(dir $(s))))$(notdir $(s:.s=.o)): $(s)
> endef
> 
> $(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval $(call archrule,$(s))))
> 
> [...]
> 
> I'm going to run some tests on arm, and it if works, I'll commit.

I think it should be $(1) rather than $(s) in the archrule, but for
some reason, the $(eval ...) is not doing _anything_ on make 3.81 on
my Debian box. With my usual make 3.82, it works fine. This is not
really a show-stopper, since no errors occur; it just means
dependencies aren't getting honored on some versions of make that
might still be out there in the wild. But I would very much appreciate
some insight on why this is happening, from any GNU make experts...

Rich

Re: Build system adjustments for subarchs

[Szabolcs Nagy]

* Rich Felker <dalias@aerifal.cx> [2013-08-14 00:02:26 -0400]:
> > 
> > define archrule =
> > $(dir $(patsubst %/,%,$(dir $(s))))$(notdir $(s:.s=.o)): $(s)
> > endef
> > 
> > $(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval $(call archrule,$(s))))
> > 
> > [...]
> > 
> > I'm going to run some tests on arm, and it if works, I'll commit.
> 
> I think it should be $(1) rather than $(s) in the archrule, but for
> some reason, the $(eval ...) is not doing _anything_ on make 3.81 on
> my Debian box. With my usual make 3.82, it works fine. This is not
> really a show-stopper, since no errors occur; it just means
> dependencies aren't getting honored on some versions of make that
> might still be out there in the wild. But I would very much appreciate
> some insight on why this is happening, from any GNU make experts...

i had to fight this fight for the test repo

define foo =
a:b
endef

does not work for some reason, but

define foo
a:b
endef

does

Re: Build system adjustments for subarchs

[Rich Felker]

On Wed, Aug 14, 2013 at 05:09:50PM +0200, Szabolcs Nagy wrote:
> * Rich Felker <dalias@aerifal.cx> [2013-08-14 00:02:26 -0400]:
> > > 
> > > define archrule =
> > > $(dir $(patsubst %/,%,$(dir $(s))))$(notdir $(s:.s=.o)): $(s)
> > > endef
> > > 
> > > $(foreach s,$(wildcard src/*/$(ARCH)*/*.s),$(eval $(call archrule,$(s))))
> > > 
> > > [...]
> > > 
> > > I'm going to run some tests on arm, and it if works, I'll commit.
> > 
> > I think it should be $(1) rather than $(s) in the archrule, but for
> > some reason, the $(eval ...) is not doing _anything_ on make 3.81 on
> > my Debian box. With my usual make 3.82, it works fine. This is not
> > really a show-stopper, since no errors occur; it just means
> > dependencies aren't getting honored on some versions of make that
> > might still be out there in the wild. But I would very much appreciate
> > some insight on why this is happening, from any GNU make experts...
> 
> i had to fight this fight for the test repo
> 
> define foo =
> a:b
> endef
> 
> does not work for some reason, but
> 
> define foo
> a:b
> endef
> 
> does

Indeed, I figured this out too from reading GNU make's changelog. The
documentation should really document any feature that hasn't been
supported for at least 10 years with a note about which version/year
it was added in, so users can make informed decisions about which
features to depend on, and the examples in the documentation should
refrain from using the new, incompatible syntax for something that has
a more compatible variant. It's really unexpected to find examples
depending on extremely new features in the manual when the adjacent
pages are talking about gcc 1.3 as if it were current...

Rich

Re: Build system adjustments for subarchs

[Luca Barbato]

On 14/08/13 17:15, Rich Felker wrote:

> Indeed, I figured this out too from reading GNU make's changelog. The
> documentation should really document any feature that hasn't been
> supported for at least 10 years with a note about which version/year
> it was added in, so users can make informed decisions about which
> features to depend on, and the examples in the documentation should
> refrain from using the new, incompatible syntax for something that has
> a more compatible variant. It's really unexpected to find examples
> depending on extremely new features in the manual when the adjacent
> pages are talking about gcc 1.3 as if it were current...

gnumake (as bash) lacks proper release policies so I wouldn't expect to
have a non-misleading documentation.

lu

Re: Build system adjustments for subarchs

[Strake]

On 13/08/2013, Rich Felker <dalias@aerifal.cx> wrote:
> The current system searches for arch-specific asm in this order:
>
> 1. $(ARCH)$(ASMSUBARCH)/%.s, where ASMSUBARCH for the default subarch
>    is not blank but rather a unique suffix (for plain arm, it's "el").
>    This allows having asm that applies only to the default subarch but
>    not others.
>
> 2. $(ARCH)/%.s, for shared asm used by all subarchs.
>
> 3. %.c, the C fallback (which is empty for code that cannot be
>    implemented at all in C).

Another option:

Each arch has properties, which each take a value in a set; these sets
are mutually exclusive, and all the values in each set have the same
length. The properties have a well-defined order. The build system
parses each directory name as an arch name and a list of properties,
and chooses the most specific match; if no match, it uses the C code.

> Unfortunately, this still provides no way to include asm that's used
> by both soft and hardfloat little-endian, or both little- and
> big-endian hardfloat, without having, for example:
>
> - armel/%.s and armhf/%.s as duplicate files or symlinked
> - armhf/%.s and armebhf/%.s as duplicate files or symlinked

Thus arm would have 2 properties:
byte order ∈ { "eb", "el" }
float hardness ∈ { "hf", "sf" }

asm used by both soft- and hard-float little-endian: armel/%.s
asm used by both little- and big-endian hard-float: armhf/%.s
asm used by little-endian hard-float: armelhf/%.s

Re: Build system adjustments for subarchs

[Rich Felker]

On Wed, Aug 14, 2013 at 07:55:25PM -0500, Strake wrote:
> On 13/08/2013, Rich Felker <dalias@aerifal.cx> wrote:
> > The current system searches for arch-specific asm in this order:
> >
> > 1. $(ARCH)$(ASMSUBARCH)/%.s, where ASMSUBARCH for the default subarch
> >    is not blank but rather a unique suffix (for plain arm, it's "el").
> >    This allows having asm that applies only to the default subarch but
> >    not others.
> >
> > 2. $(ARCH)/%.s, for shared asm used by all subarchs.
> >
> > 3. %.c, the C fallback (which is empty for code that cannot be
> >    implemented at all in C).
> 
> Another option:
> 
> Each arch has properties, which each take a value in a set; these sets
> are mutually exclusive, and all the values in each set have the same
> length. The properties have a well-defined order. The build system
> parses each directory name as an arch name and a list of properties,
> and chooses the most specific match; if no match, it uses the C code.

This would be nice, but as far as I can tell there's no simple way of
doing it without hard-coding the number of possible properties and
including a rule for each one in the makefile.

> > Unfortunately, this still provides no way to include asm that's used
> > by both soft and hardfloat little-endian, or both little- and
> > big-endian hardfloat, without having, for example:
> >
> > - armel/%.s and armhf/%.s as duplicate files or symlinked
> > - armhf/%.s and armebhf/%.s as duplicate files or symlinked
> 
> Thus arm would have 2 properties:
> byte order ∈ { "eb", "el" }
> float hardness ∈ { "hf", "sf" }
> 
> asm used by both soft- and hard-float little-endian: armel/%.s
> asm used by both little- and big-endian hard-float: armhf/%.s
> asm used by little-endian hard-float: armelhf/%.s

The system I ended up committing is not quite this logical, but it
admits a simple implementation and it's very flexible, allowing
arbitrary sharing and substitutions between subarchs.

Rich

Re: Build system adjustments for subarchs

[Rob Landley]

On 08/13/2013 08:06:17 PM, Rich Felker wrote:
> Hi,
> 
> I'm looking for some ideas on a problem in the build system: how to
> cleanly share asm between subarchs. The problem is this: ARM, and
> possibly other archs in the future, has multiple dimensions of
> subarch: little-endian(default) vs big-endian, and
> fpu-agnostic(default) vs hardfloat. The asm requirements are:
> 
> - MOST asm is shared by all subarchs.
> 
> - A small amount of asm (for now, just memcpy) is endian-specific but
>   has nothing to do with floating point ABI.
> 
> - A fairly significant amount of asm in the future will be hard-float
>   specific (optimized math functions and floating point environment)
>   but has nothing to do with endianness.

If there's no actual overlap it sounds like you're describing search  
paths you stick stuff at the front of. Have hardfloat specific .S bits  
in one dir, differently endian stuff in another dir, and the truly  
agnostic stuff with default versions of everything in a third, then  
thread a search path through 'em.

> What I don't like about this approach is that the logic for the names
> to use in steps 1-3 has to go either in the configure script or the
> makefile. The makefile is not intended to have this sort of arch
> logic, but instead to derive the logic from the source tree structure.

How you get a makefile to do _anything_ is not my department. (Horrible  
language, make. Imperative or declarative, PICK ONE.)

Rob