[9fans] Modularizing plan9port

[9fans] Modularizing plan9port

[Enrico Weigelt]

Hi folks,


I'm going to modularize plan9port a bit.

Ideally, each component (library, app, etc) should become it's
completely own package. This gets tricky because of the boostrapping
circle (mk requires lots libs, etc). In the current structure this
doesnt harm as mk is built directly with these libs' sources - but
an splitted tree won't this way.

So I intent do write some script which creates Makefile's from
mkfile's and maybe even does some build-time configuration
(sort of ./configure ;-)). That script(s) could be packet along
with some other fundamental p9p build utils, and this package
then would be the very first in depedency chain. Taking cross-
builds into account, this would be an TOOL or HOST dependency,
since it runs on the building host, not the target - an compiler-
less target wouldn't ever need it (it plays in the same liga as
tools like lex, make, autoconf, etc).

What do you think about this approach ?


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Modularizing plan9port

[erik quanstrom]

> So I intent do write some script which creates Makefile's from
> mkfile's and maybe even does some build-time configuration
> (sort of ./configure ;-)). That script(s) could be packet along
> with some other fundamental p9p build utils, and this package
> then would be the very first in depedency chain. Taking cross-
> builds into account, this would be an TOOL or HOST dependency,
> since it runs on the building host, not the target - an compiler-
> less target wouldn't ever need it (it plays in the same liga as
> tools like lex, make, autoconf, etc).
>
> What do you think about this approach ?

aren't you skipping a step?  how to modularize p9p is not an
interesting question until one has decided that it worth doing.

why is modularizing p9p a good idea?

as you point out, the process creates a number of problems without
adding any functionality.

- erik

Re: [9fans] Modularizing plan9port

[Jeff Sickel]

On Jun 11, 2008, at 7:40 AM, Enrico Weigelt wrote:

> I'm going to modularize plan9port a bit.

I'd recommend just leaving it as it is--you can easily pull out and
use the pieces you want based off of the current build tree.

...

> What do you think about this approach ?

Way to much like GNU based software and thus another reason not to
make those modifications.    A better approach is to examine the
current INSTALL and surrounding documentation to use the pieces of
that that are pertinent to the bits you desire.

-jas

Re: [9fans] Modularizing plan9port

[Russ Cox]

> I'm going to modularize plan9port a bit.
>
> Ideally, each component (library, app, etc) should become it's
> completely own package.

I fail to see what is ideal about this.

Ask yourself whether you're doing this because it would
actually make your life easier or because of some
pre-conceived notion that software packaging should
be complex.

> So I intent do write some script which creates Makefile's from
> mkfile's and maybe even does some build-time configuration
> (sort of ./configure ;-)). That script(s) could be packet along
> with some other fundamental p9p build utils, and this package
> then would be the very first in depedency chain. Taking cross-
> builds into account, this would be an TOOL or HOST dependency,
> since it runs on the building host, not the target - an compiler-
> less target wouldn't ever need it (it plays in the same liga as
> tools like lex, make, autoconf, etc).
>
> What do you think about this approach ?

I think you are wasting your time.

There's no need to fiddle with the build structure:
you could still require the whole tree to build things
and then just split up the post-build tree.
Then you don't have to worry about rewriting Makefiles
or adding your own configure scripts or other horrors.
I certainly won't take any of that back into the main tree.

If you just want to create a package for the Linux du jour,
just build the tree and then make it one big package.
If that would be too large a package, you could reasonably
split it into four:

	28/12	font	$PLAN9/font
	68/27	bin 	$PLAN9/bin and $PLAN9/lib/lib*.a
	20/4 	src 	$PLAN9/src
	18/6 	base	everything else

The numbers are MB, uncompressed and compressed.
Splitting out bin/lib makes it easy to swap in binaries for
a different architecture.  Splitting out src (or font) makes
those files easy to drop if you need to scrimp on disk space.
There's not much point to anything more fine-grained.

Russ

Re: [9fans] Modularizing plan9port

[Enrico Weigelt]

* Russ Cox <rsc@swtch.com> wrote:

Hi,

> Ask yourself whether you're doing this because it would
> actually make your life easier or because of some

It *does* make my life easier!

I'm not just using it for personal stuff, but for lots of highly
customized production systems, where careful maintenance is
very important.

Disk space is not the issue, but the amount of code to be
maintained (source and binary). So the target systems *always*
should only contain exactly what's needed - nothing more.

> pre-conceived notion that software packaging should be complex.

Actually, I want to make it simpler. You probably can't see this
since you don't know what happens behind the scenes at my site ;-P

One essential constraint is, that everything's built through an
sysroot'ed cross-toolchain. Right after compile several checks
run on the output, packages are then trimmed-down (eg. removing
all build-time stuff) and then it goes to the testing system.
Only after the whole pipe ran through properly, the binary
package is committed to the production systems.

> There's no need to fiddle with the build structure:
> you could still require the whole tree to build things
> and then just split up the post-build tree.

The current approach already fails with crosscompiling.
I *can not* use the in-tree built mk for further building
and I *must* make sure that imports are strictly coming
from within sysroot.

> Then you don't have to worry about rewriting Makefiles
> or adding your own configure scripts or other horrors.
> I certainly won't take any of that back into the main tree.

You shouldn't generally declare this approach as horror,
just because autoconf is a horrible example.


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Modularizing plan9port

[Uriel]

If you want to cross-compile why don't you use Plan 9? or at least the
port of the plan9 compilers to lunix[1], where cross compiling is the
only way to compile.

Cross-compiling in Gnu/land is a nightmare not worth going into.

uriel

[1] http://gsoc.cat-v.org/projects/kencc/

On Wed, Jun 11, 2008 at 5:30 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Russ Cox <rsc@swtch.com> wrote:
>
> Hi,
>
>> Ask yourself whether you're doing this because it would
>> actually make your life easier or because of some
>
> It *does* make my life easier!
>
> I'm not just using it for personal stuff, but for lots of highly
> customized production systems, where careful maintenance is
> very important.
>
> Disk space is not the issue, but the amount of code to be
> maintained (source and binary). So the target systems *always*
> should only contain exactly what's needed - nothing more.
>
>> pre-conceived notion that software packaging should be complex.
>
> Actually, I want to make it simpler. You probably can't see this
> since you don't know what happens behind the scenes at my site ;-P
>
> One essential constraint is, that everything's built through an
> sysroot'ed cross-toolchain. Right after compile several checks
> run on the output, packages are then trimmed-down (eg. removing
> all build-time stuff) and then it goes to the testing system.
> Only after the whole pipe ran through properly, the binary
> package is committed to the production systems.
>
>> There's no need to fiddle with the build structure:
>> you could still require the whole tree to build things
>> and then just split up the post-build tree.
>
> The current approach already fails with crosscompiling.
> I *can not* use the in-tree built mk for further building
> and I *must* make sure that imports are strictly coming
> from within sysroot.
>
>> Then you don't have to worry about rewriting Makefiles
>> or adding your own configure scripts or other horrors.
>> I certainly won't take any of that back into the main tree.
>
> You shouldn't generally declare this approach as horror,
> just because autoconf is a horrible example.
>
>
> cu
> --
> ----------------------------------------------------------------------
>  Enrico Weigelt, metux IT service -- http://www.metux.de/
>
>  cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
> ----------------------------------------------------------------------
>  Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
> ----------------------------------------------------------------------
>
>

Re: [9fans] Modularizing plan9port

[Uriel]

By the way, silly question, but what would it take to have the kencc
port accepted as part of p9p?

And a port of of plan9's awk (trivial to do)? It would be nice to be
able to rely on a decent utf-8 enabled awk when writing scripts for
p9p without worrying about what broken awk does this or that *nix have
installed.

uriel

On Wed, Jun 11, 2008 at 5:43 PM, Uriel <uriel99@gmail.com> wrote:
> If you want to cross-compile why don't you use Plan 9? or at least the
> port of the plan9 compilers to lunix[1], where cross compiling is the
> only way to compile.
>
> Cross-compiling in Gnu/land is a nightmare not worth going into.
>
> uriel
>
> [1] http://gsoc.cat-v.org/projects/kencc/
>
> On Wed, Jun 11, 2008 at 5:30 PM, Enrico Weigelt <weigelt@metux.de> wrote:
>> * Russ Cox <rsc@swtch.com> wrote:
>>
>> Hi,
>>
>>> Ask yourself whether you're doing this because it would
>>> actually make your life easier or because of some
>>
>> It *does* make my life easier!
>>
>> I'm not just using it for personal stuff, but for lots of highly
>> customized production systems, where careful maintenance is
>> very important.
>>
>> Disk space is not the issue, but the amount of code to be
>> maintained (source and binary). So the target systems *always*
>> should only contain exactly what's needed - nothing more.
>>
>>> pre-conceived notion that software packaging should be complex.
>>
>> Actually, I want to make it simpler. You probably can't see this
>> since you don't know what happens behind the scenes at my site ;-P
>>
>> One essential constraint is, that everything's built through an
>> sysroot'ed cross-toolchain. Right after compile several checks
>> run on the output, packages are then trimmed-down (eg. removing
>> all build-time stuff) and then it goes to the testing system.
>> Only after the whole pipe ran through properly, the binary
>> package is committed to the production systems.
>>
>>> There's no need to fiddle with the build structure:
>>> you could still require the whole tree to build things
>>> and then just split up the post-build tree.
>>
>> The current approach already fails with crosscompiling.
>> I *can not* use the in-tree built mk for further building
>> and I *must* make sure that imports are strictly coming
>> from within sysroot.
>>
>>> Then you don't have to worry about rewriting Makefiles
>>> or adding your own configure scripts or other horrors.
>>> I certainly won't take any of that back into the main tree.
>>
>> You shouldn't generally declare this approach as horror,
>> just because autoconf is a horrible example.
>>
>>
>> cu
>> --
>> ----------------------------------------------------------------------
>>  Enrico Weigelt, metux IT service -- http://www.metux.de/
>>
>>  cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
>> ----------------------------------------------------------------------
>>  Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
>> ----------------------------------------------------------------------
>>
>>
>

Re: [9fans] Modularizing plan9port

[Enrico Weigelt]

* Uriel <uriel99@gmail.com> wrote:

Hi,

> If you want to cross-compile why don't you use Plan 9?

I'm not using (native) Plan9 - just a few tools from p9p.
For my targets I need to integrate them into Gnu'ish platforms.
So, an p9p-compile'd app has to run just like any other app,
w/o additionally porting.

> Cross-compiling in Gnu/land is a nightmare not worth going into.

No, it isn't - as long as you've got a proper toolchain and
get around autoshit. (eg. I've got my own libtool implementation ;-p)


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Modularizing plan9port

[Russ Cox]

"I need a way to cross-compile plan9port" would
have been a much more productive opening statement
than "I'm going to modularize plan9port".

I think you should be able to cross-compile it
pretty easily if you already have a cross-compilation
environment set up.

1. Build a local plan9port tree, put it in the chroot
  environment, and put it at the end of your chrooted $PATH.
2. Edit that tree's 9 script to override $PLAN9 with
  the path to the local tree (1 line change).
3. Extract a fresh tree somewhere else.
4. run SYSNAME=xxx OBJTYPE=xxx INSTALL in the fresh tree.
  (assumes
I haven't tried this (I find it much easier just to set up
a dedicated machine of the right OS and architecture
and use its native tools), but having the local copy of
the tools in the path before running INSTALL should
make it use those instead of the ones it builds.

That sounds like a lot less work than your original proposal.

Good luck.

Russ

Re: [9fans] Modularizing plan9port

[Roman Shaposhnik]

On Wed, 2008-06-11 at 14:23 -0400, Russ Cox wrote:
> I find it much easier just to set up
> a dedicated machine of the right OS and architecture
> and use its native tools

Speaking of which: am I the only one betraying the true
cross-compiling in favor of virtualized copies of the
OS/platform? I mean, with Qemu and abundance of images
it just seems way easier these days.

Thanks,
Roman.

Re: [9fans] Modularizing plan9port

[Enrico Weigelt]

* Russ Cox <rsc@swtch.com> wrote:

> I think you should be able to cross-compile it
> pretty easily if you already have a cross-compilation
> environment set up.
>
> 1. Build a local plan9port tree, put it in the chroot
>   environment, and put it at the end of your chrooted $PATH.

Note: sysroot != chroot.
(sysroot *only* applies to the toolchain's search pathes)

> 2. Edit that tree's 9 script to override $PLAN9 with
>   the path to the local tree (1 line change).
> 3. Extract a fresh tree somewhere else.
> 4. run SYSNAME=xxx OBJTYPE=xxx INSTALL in the fresh tree.
>   (assumes

Okay, how can I make sure the right toolchain is used, *all*
imports come from within sysroot and *only* code from the
(building) HOST system is executed - *never* from TARGET ?

> I haven't tried this (I find it much easier just to set up
> a dedicated machine of the right OS and architecture
> and use its native tools), but having the local copy of

No, I can't have one dedicated machine per target right here.
And most of my targets aren't even capable of compiling.

> That sounds like a lot less work than your original proposal.

Even if I could go this way, it would require *much* more
manual works. Simply too expensive.


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Modularizing plan9port

[William Josephson]

On Wed, Jun 11, 2008 at 10:48:34PM +0200, Enrico Weigelt wrote:
> * Russ Cox <rsc@swtch.com> wrote:
> No, I can't have one dedicated machine per target right here.
> And most of my targets aren't even capable of compiling.

Why not just use a VM for each target?

> > That sounds like a lot less work than your original proposal.
>
> Even if I could go this way, it would require *much* more
> manual works. Simply too expensive.

Re: [9fans] Modularizing plan9port

[Russ Cox]

> Okay, how can I make sure the right toolchain is used, *all*
> imports come from within sysroot and *only* code from the
> (building) HOST system is executed - *never* from TARGET ?

I would have expected your host system to refuse
to run binaries for the target architecture.

You are clearly doing far more complex things
than my simple brain can fathom.

Good luck.
Russ

Re: [9fans] Modularizing plan9port

[Enrico Weigelt]

* Russ Cox <rsc@swtch.com> wrote:

Hi,

> > Okay, how can I make sure the right toolchain is used, *all*
> > imports come from within sysroot and *only* code from the
> > (building) HOST system is executed - *never* from TARGET ?
>
> I would have expected your host system to refuse
> to run binaries for the target architecture.

targets can differ in much more things than just arch.
When arch and platform are the same, often the code can run
somehow - but not reliable. So there's a risk that really
strange things happen.

> You are clearly doing far more complex things
> than my simple brain can fathom.

Might be ;-P


cu
--
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 cellphone: +49 174 7066481   email: info@metux.de   skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------

Re: [9fans] Modularizing plan9port

[Iruata Souza]

On Wed, Jun 11, 2008 at 2:53 PM, Enrico Weigelt <weigelt@metux.de> wrote:
> * Uriel <uriel99@gmail.com> wrote:
>
>> Cross-compiling in Gnu/land is a nightmare not worth going into.
>
> No, it isn't - as long as you've got a proper toolchain and
> get around autoshit. (eg. I've got my own libtool implementation ;-p)
>

no, it isn't - as long it is.

Re: [9fans] Modularizing plan9port

[tlaronde]

Hello,

On Wed, Jun 11, 2008 at 02:40:38PM +0200, Enrico Weigelt wrote:
>
> [...]
> So I intent do write some script which creates Makefile's from
> mkfile's and maybe even does some build-time configuration
> (sort of ./configure ;-)). That script(s) could be packet along
> with some other fundamental p9p build utils, and this package
> then would be the very first in depedency chain. Taking cross-
> builds into account, this would be an TOOL or HOST dependency,
> since it runs on the building host, not the target - an compiler-
> less target wouldn't ever need it (it plays in the same liga as
> tools like lex, make, autoconf, etc).

FWIW, and this has nothing to do with what should or should not be done
with plan9port, for this kind of stuff you can give a look at:

http://downloads.kergis.com/risk_comp_1.0C.0.1.tar.gz

(I'm the author, this is a BSD licence with public domain configuration
files. If some people find this usefull and are uncomfortable with the
licence, I can put in public domain the whole stuff. R.I.S.K. was for:
Reduced Instructions Set toolKit)

This is precisely (for POSIX right now) some simple sh(1) scripts, and
a set of sh(1) files setting variables to do this. It embeds processing
to use several parsers and scanners (lex(1) and yacc(1)) in a same
object file (I don't use this anymore, since I have understood that I
needed this due to engineering faults...)

This is not rocket science. This is simple (and limited and faulty). And
this was done to allow not using any autoHell stuff.

It has been used to successfully compile (cross-compile too) and
distribute under the *BSD, some flavors of Linux (I lost tracks of all
the variations), and Mac OSX.

The documentation is the commented scripts by themselves.

But as others more gifted have spent less time understanding than I
spent myself, putting the emphasis on these sort of tools, is like
looking to the finger when plan9 shows you the moon.

I put now together: consistancy and concision:

"hbrw, an hmg to Kn Thmpsn".

Cheers,
--
Thierry Laronde (Alceste) <tlaronde +AT+ polynum +dot+ com>
                 http://www.kergis.com/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C