What if we put the *.o files for arch asm in the arch dirs rather than
in the main src dirs. Is there some clean way we could write a rule to
remove the base .o files from OBJS when the arch-specific .o files are
there?

That's what I had in mind and I've already implemented it, but I couldn't get the *.sub files to work in that setup. However, if we replace them with Makefile fragments as you suggested it might it might solve that problem. Let me try that out and I'll let you know whether this is a way to go.