runit not ready for cross-compilation

runit not ready for cross-compilation

[Radek Podgorny]

[-- Attachment #1: Type: text/plain, Size: 1142 bytes --]

Hi,

I'm trying to make runit run on my arm-based PDA. There's a problem that I 
have to cross-compile runit on my i686 host and it doesn't work. To be more 
specific, it works until the point where it tries to run some compiled 
program...

...
./compile sig_pause.c
./compile wait_nohang.c
./compile wait_pid.c
./makelib byte.a byte_chr.o byte_copy.o byte_cr.o byte_diff.o \
byte_rchr.o fmt_uint.o fmt_uint0.o fmt_ulong.o scan_ulong.o str_chr.o \
str_diff.o str_len.o str_start.o
./chkshsgr || ( cat warn-shsgr; exit 1 )
/bin/sh: ./chkshsgr: cannot execute binary file
./compile fifo.c
...

The root of this problem is that you have to distinguish between host (build) 
machine and target machine arch. Only single conf-cc and stuff simply isn't 
enough. The same for ./compile script and so on.

Could you please fix this to compile all build-time executables (for checking 
stuff) using compiler1 and the target executables using compiler2? I would do 
it myself but I got lost in your build system... :-(

Thanks
Radek Podgorny

-- 
GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x98E56D84

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: runit not ready for cross-compilation

[Alejandro Mery]

Radek Podgorny wrote:
> Hi,
> 
> I'm trying to make runit run on my arm-based PDA. There's a problem that I 
> have to cross-compile runit on my i686 host and it doesn't work. To be more 
> specific, it works until the point where it tries to run some compiled 
> program...

yes, runit has the same bad habit djb software has. compiling it's own 
build timers helpers in the same way that the runtime programs.

what's the reason to use binary helpers instead of simple sh scripts for 
those tasks?

Alejandro Mery

Re: runit not ready for cross-compilation

[Charlie Brady]

On Sun, 11 Dec 2005, Alejandro Mery wrote:

> yes, runit has the same bad habit djb software has. compiling it's own build 
> timers helpers in the same way that the runtime programs.
>
> what's the reason to use binary helpers instead of simple sh scripts for 
> those tasks?

I think djb's rationale is that he can create his own portability, and 
doesn't depend on the portable implementation of features by the platform 
provider. Except for the compiler's C implementation, that is :-)

Re: runit not ready for cross-compilation

[Radek Podgorny]

[-- Attachment #1: Type: text/plain, Size: 805 bytes --]

I though runit is a daemontools fork so we can give s... what djb thinks :-)

Are you gonna implement my cross-compilation stuff? Do want me anything to do 
so you can actually do so?

Radek Podgorny


> On Sun, 11 Dec 2005, Alejandro Mery wrote:
> > yes, runit has the same bad habit djb software has. compiling it's own
> > build timers helpers in the same way that the runtime programs.
> >
> > what's the reason to use binary helpers instead of simple sh scripts for
> > those tasks?
>
> I think djb's rationale is that he can create his own portability, and
> doesn't depend on the portable implementation of features by the platform
> provider. Except for the compiler's C implementation, that is :-)

-- 
GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x98E56D84

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: runit not ready for cross-compilation

[Charlie Brady]

On Sat, 17 Dec 2005, Radek Podgorny wrote:

> I though runit is a daemontools fork so we can give s... what djb thinks :-)

Alejandro said that runit "has the same bad habit djb software has". Then 
asked "what's the reason"? I simply made an attempt to answer that 
question.

I don't think that runit is a daemontools fork. My understanding is it's a 
reimplementation of the same design pattern.

> Are you gonna implement my cross-compilation stuff?

Me? No. Why me? Are you?

Re: runit not ready for cross-compilation

[Radek Podgorny]

[-- Attachment #1: Type: text/plain, Size: 746 bytes --]

Oh yeah, not I got it.

By "you" I meant "you, the mailing list guys". I don't know who should I write 
to to implement it... :-(

Radek P.


> On Sat, 17 Dec 2005, Radek Podgorny wrote:
> > I though runit is a daemontools fork so we can give s... what djb thinks
> > :-)
>
> Alejandro said that runit "has the same bad habit djb software has". Then
> asked "what's the reason"? I simply made an attempt to answer that
> question.
>
> I don't think that runit is a daemontools fork. My understanding is it's a
> reimplementation of the same design pattern.
>
> > Are you gonna implement my cross-compilation stuff?
>
> Me? No. Why me? Are you?

-- 
GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x98E56D84

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: runit not ready for cross-compilation

[Gerrit Pape]

On Mon, Dec 12, 2005 at 01:50:00AM +0100, Radek Podgorny wrote:
> The root of this problem is that you have to distinguish between host (build) 
> machine and target machine arch. Only single conf-cc and stuff simply isn't 
> enough. The same for ./compile script and so on.

Erm.  To me it doesn't make much sense to do runtime tests on the
build system for programs that actually will run on the target system.
Surely I may be wrong though.

Gerrit.

Re: runit not ready for cross-compilation

[Radek Podgorny]

[-- Attachment #1: Type: text/plain, Size: 798 bytes --]

Well, let's get rid of all of this, then...

In general there is no way to do runtime tests when building for different 
architecture. Are these tests even needed? No other software I know of 
doesn't do such things...

Radek P.


> On Mon, Dec 12, 2005 at 01:50:00AM +0100, Radek Podgorny wrote:
> > The root of this problem is that you have to distinguish between host
> > (build) machine and target machine arch. Only single conf-cc and stuff
> > simply isn't enough. The same for ./compile script and so on.
>
> Erm.  To me it doesn't make much sense to do runtime tests on the
> build system for programs that actually will run on the target system.
> Surely I may be wrong though.
>
> Gerrit.

-- 
GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x98E56D84

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: runit not ready for cross-compilation

[Charlie Brady]

On Tue, 3 Jan 2006, Radek Podgorny wrote:

> Well, let's get rid of all of this, then...
>
> In general there is no way to do runtime tests when building for different
> architecture. Are these tests even needed? No other software I know of
> doesn't do such things...

As has earlier been pointed out, other software does do such 
compile-link-run testing - most or all of Dan Bernstein's software does 
it, as does software derived from Dan's software. I think you'll also find 
that some software configured via ./configure will also do such tests, 
just not as explicitly.

Testing for available features and extant bugs this way is quick and 
reliable. It just doesn't work for cross compilation.

Here's something for you to try:

make clean
make clean compile load choose
cat iopause.h1 > iopause.h
cat uint64.h1 > uint64.h
make chkshsgr
cat hasshsgr.h1 > hasshsgr.h
make

You might need to use h2 in place of h1, depending on your target. Mine is 
little end MIPS.

Re: runit not ready for cross-compilation

[Charles Duffy]

Charlie Brady wrote:
> As has earlier been pointed out, other software does do such 
> compile-link-run testing - most or all of Dan Bernstein's software does 
> it, as does software derived from Dan's software. I think you'll also 
> find that some software configured via ./configure will also do such 
> tests, just not as explicitly.

Autoconf, however, has provisions for cross-compiling: The packager can 
specify test results for the given target via environment variables; 
defaults can be provided for use in cross-compilation environments; and 
as many tests as possible are done *without* requiring the use of a 
target-compiled binary.

Re: runit not ready for cross-compilation

[Charlie Brady]

On Thu, 5 Jan 2006, Charles Duffy wrote:

> Charlie Brady wrote:
>> As has earlier been pointed out, other software does do such 
>> compile-link-run testing - most or all of Dan Bernstein's software does 
>> it, as does software derived from Dan's software. I think you'll also 
>> find that some software configured via ./configure will also do such 
>> tests, just not as explicitly.
>
> Autoconf, however, has provisions for cross-compiling:

Sure, and sometimes it's broken. And always it's overcomplicated.

> The packager can specify test results for the given target ...

Which is essentially what I'm doing.

> via environment variables; defaults can be provided for use in 
> cross-compilation environments; and as many tests as possible are done 
> *without* requiring the use of a target-compiled binary.

But let's not talk too much about autoconf. Would anyone care to comment 
about my workaround? Is it "good enough" (so that the build system doesn't 
need to be rewritten)? Would someone care to document the correct h1/h2 
choices for various platforms?

Re: runit not ready for cross-compilation

[Alejandro Mery]

Charles Duffy wrote:
> Charlie Brady wrote:
> 
>> As has earlier been pointed out, other software does do such 
>> compile-link-run testing - most or all of Dan Bernstein's software 
>> does it, as does software derived from Dan's software. I think you'll 
>> also find that some software configured via ./configure will also do 
>> such tests, just not as explicitly.
> 
> 
> Autoconf, however, has provisions for cross-compiling: The packager can 
> specify test results for the given target via environment variables; 
> defaults can be provided for use in cross-compilation environments; and 
> as many tests as possible are done *without* requiring the use of a 
> target-compiled binary.

what?? noone needs something like autoconf, just using simple sh scripts 
and a simple Makefile (instead of .c helpers) is _enough_ to make 
software cross compilable. and of course you wont run make test when 
cross compiling, and there is no need to do that.

Alejandro Mery

Re: runit not ready for cross-compilation

[Radek Podgorny]

[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]

Yeah, that's exactly what I think, too...

Radek P.

> Charles Duffy wrote:
> > Charlie Brady wrote:
> >> As has earlier been pointed out, other software does do such
> >> compile-link-run testing - most or all of Dan Bernstein's software
> >> does it, as does software derived from Dan's software. I think you'll
> >> also find that some software configured via ./configure will also do
> >> such tests, just not as explicitly.
> >
> > Autoconf, however, has provisions for cross-compiling: The packager can
> > specify test results for the given target via environment variables;
> > defaults can be provided for use in cross-compilation environments; and
> > as many tests as possible are done *without* requiring the use of a
> > target-compiled binary.
>
> what?? noone needs something like autoconf, just using simple sh scripts
> and a simple Makefile (instead of .c helpers) is _enough_ to make
> software cross compilable. and of course you wont run make test when
> cross compiling, and there is no need to do that.
>
> Alejandro Mery

-- 
GnuPG key: http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x98E56D84

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: runit not ready for cross-compilation

[Charles Duffy]

Alejandro Mery wrote:
> what?? noone needs something like autoconf, just using simple sh
> scripts and a simple Makefile (instead of .c helpers) is _enough_ to
> make software cross compilable. and of course you wont run make test
> when cross compiling, and there is no need to do that. 
Who said anything about "make test"?

I was referring to tests which determine details of the target platform.

OT: cross compilation and autoconf [was: Re: runit not ready for cross-compilation]

[Charles Duffy]

My apologies for dragging this thread on further. I've changed the 
subject so as to not interfere with more topical discussion.

Alejandro Mery wrote:
> what?? noone needs something like autoconf, just using simple sh scripts 
> and a simple Makefile (instead of .c helpers) is _enough_ to make 
> software cross compilable. and of course you wont run make test when 
> cross compiling, and there is no need to do that.

The software in question here may not need autoconf -- but there are 
cases where its more esoteric functionalities do indeed come in handy. 
I'm not arguing that autoconf makes sense here -- I'm just disputing 
that "no one needs something like autoconf".

In a former life I spent 2-3 years porting and packaging 3rd-party 
(mostly Free) software to be cross compiled to the 5 or 6 target 
architectures and 2 host platforms my employer supported, and in that 
time I ran across a *lot* of different compile-time tests. The strong 
majority of them could be resolved by attempting target compile and link 
steps (to validate headers and library contents) without actually 
needing to run anything on the target, but there were a few more obscure 
tests which did inded care about things that weren't really available 
without access to the target. (For instance: Are you on one of the buggy 
platforms where ferror() returns -1 instead of 0 on a file handle which 
does not in fact have its error flag set? How can you tell without 
actually trying?) [1]

Additionally, it's better to have something that's *consistent* and 
written to make cross compilation reasonably straightforward than for 
every package to have its own custom configure/build system (in which 
case maybe 10% of them have had ample thought put into allowing cross 
compilation, and fewer than that have actually had such functionality 
actually *tested* by the maintainers). To be sure, autoconf is a beast 
-- but it's a beast which has picked up much of its size in satisfying 
esoteric requirements. Every once in a while, someone *needs* those 
capabilities, and having them around is darned handy.


[1] - Okay, this isn't a great example -- one can just treat negative 
return values from ferror() the same way as one treats zero values, and 
it becomes moot. There are similar cases, however, where a method may be 
defined the same way in the headers of two platforms but have material 
behavioral differences between them such that a runtime test is 
necessary. No, I don't remember any -- this was 5 or 6 years ago.

Re: OT: cross compilation and autoconf [was: Re: runit not ready for cross-compilation]

[Laurent Bercot]

> Additionally, it's better to have something that's *consistent* and 
> written to make cross compilation reasonably straightforward than for 
> every package to have its own custom configure/build system

 Yes. That's why it's best to have ONE custom configure/build system
designed once and for all. :)
 If you're experienced in portability problems, could you please check
out the configuration scripts in skalibs (derived from DJB scripts,
but more complete) and tell me if there's some obvious sysdep that I
have forgotten ?
 The beast is at http://www.skarnet.org/software/skalibs/
 and I use it to avoid EVER having to learn or rely on autoconf.

 Thanks for your kind help. :)

-- 
 Laurent

Re: OT: cross compilation and autoconf [was: Re: runit not ready for cross-compilation]

[Charlie Brady]

On Fri, 6 Jan 2006, Laurent Bercot wrote:

> If you're experienced in portability problems, could you please check
> out the configuration scripts in skalibs (derived from DJB scripts,
> but more complete) and tell me if there's some obvious sysdep that I
> have forgotten ?
> The beast is at http://www.skarnet.org/software/skalibs/
> and I use it to avoid EVER having to learn or rely on autoconf.

And see in particular:

http://www.skarnet.org/software/skalibs/crosscompile.html