Re: [9fans] Standalone unix port of the original rc shell

[Russ Cox <russcox@gmail.com>]

"Porting" rc is not the most representative test case for
porting things from p9p, because rc, more than anything
else in the suite, is already a Unix program.  Secstore
would be a better test case.  It's still a simple program,
but it uses a lot more of the infrastructure.

> Just have a look into bin/9* and I ask why this
> extra-abstraction is needed... 

There are a lot of files matching 9* in bin.
	9  9a  9ar  9c	9fs  9install  9l  9p  9pserve	9.rc  9term
I assume you mean 9a, 9ar, 9c, 9l.

First and foremost, these scripts provide a standard
interface for the mkfiles to use.  9c(1) defines most
of what they guarantee to provide.  If you want to run
on a system with its own set of odd conventions, you
can accomodate them here and not change the rest of the
system.  This does happen.  (SunOS and OS X come to mind.)
If the underlying system tools need extra options (they
always do) or disagree on exactly what the flags are,
that is all hidden in these scripts instead of exposed
to every mkfile in the system.  9c invokes gcc or cc
or whatever the right compiler is -- gcc isn't the only
compiler in the world.  9a invokes the correct assembler.
9ar invokes ar, following it with ranlib if needed (some
OS X versions).  9l links the files, automatically linking
the necessary libraries.  9l alone removes a huge burden
from the mkfile writers.

Aesthetically, the scripts also help to hide the fact that
Unix has grown ridiculously long-winded in its old age.
To begin with, when I type "mk", it's a lot nicer to see

	x40=; mk
	9c  acme.c
	9c  addr.c
	9c  buff.c
	9c  cols.c
	9c  disk.c
	...

than

	x40=; mk 
	gcc -DPLAN9PORT -I/usr/local/plan9/include -O2 -c -Wall
-Wno-parentheses -Wno-missing-braces -Wno-switch -Wno-comment
-Wno-sign-compare -Wno-unknown-pragmas -fno-omit-frame-pointer -ggdb
-D__Linux26__  addr.c
	gcc -DPLAN9PORT -I/usr/local/plan9/include -O2 -c -Wall
-Wno-parentheses -Wno-missing-braces -Wno-switch -Wno-comment
-Wno-sign-compare -Wno-unknown-pragmas -fno-omit-frame-pointer -ggdb
-D__Linux26__  buff.c
	gcc -DPLAN9PORT -I/usr/local/plan9/include -O2 -c -Wall
-Wno-parentheses -Wno-missing-braces -Wno-switch -Wno-comment
-Wno-sign-compare -Wno-unknown-pragmas -fno-omit-frame-pointer -ggdb
-D__Linux26__  cols.c
	gcc -DPLAN9PORT -I/usr/local/plan9/include -O2 -c -Wall
-Wno-parentheses -Wno-missing-braces -Wno-switch -Wno-comment
-Wno-sign-compare -Wno-unknown-pragmas -fno-omit-frame-pointer -ggdb
-D__Linux26__  disk.c

because then you might actually have a chance of
noticing when the compiler prints a warning or error.
9c also reformats the output of the compiler, to put
line references in the standard line:number form on Suns,
or to increase the signal to noise ratio of the gcc errors:

	x40=; cat x.c
	int f(void) { return g; }
	x40=; 9c x.c
	x.c:1: error: `g' undeclared
	x40=; gcc -Wall x.c
	x.c: In function `f':
	x.c:1: error: `g' undeclared (first use in this function)
	x.c:1: error: (Each undeclared identifier is reported only once
	x.c:1: error: for each function it appears in.)
	x40=; 

After the first 1000 times you've seen those parenthetical
remarks by gcc, you don't need to be told again.

> src/mk* - pretty much stuff, isn't one general inclusion
> sufficient?

Mkone and mkmany might be combined (with the loss of the
default target for mkone), and mklib and mksyslib might
be combined (at the cost of every system library mkfile
knowing where the lib directory is), but all four serve
different purposes and the common bits *are* factored out
into mkcommon.  I think the situation would be worse with
just one.

See http://plan9.bell-labs.com/sys/doc/mkfiles.pdf for more
on why things are the way they are. 

> Why are shell scripts (like INSTALL), make and mk
> needed? Isn't make for bootstrapping enough and afterwards
> only mk?

Isn't this how it works now?  I admit that I could remove
the reliance on make pretty easily, but everyone has make.
Once the system is installed, only mk is used.  INSTALL is
just a convenience to get you started.

I agree with your original comment that if all you want
is rc, then working hard to build mk might be overkill.
But since my goal was to build everything anyway, making
mk as the first step isn't such a big deal.

> I didn't said that the system is bad, but it can be done
> simplier. For the monolithic character, the main reason to
> me is, that all headers reside in $PLAN9/include, which
> could be done in two places, that it is easier to just
> have a bunch of independent libs with their headers and
> a central place (poorly there is no union mount in *NIX)
> where all headers get together... Currently it is very much
> effort to determine which headers are needed/provided by
> which lib in src/lib*...

Grep for AUTOLIB in the headers and you'll find out which
libraries they need to be linked with.  In fact, 9l will
do the linking for you.  As for which headers are provided
by which library, the usual rule is that foo.h or libfoo.h
goes with $PLAN9/src/libfoo.  There are a few exceptions
but that is by far the norm.  An annotated list of the
header files is below.  In any event, the header situation
is far simpler than it is in Unix.

I think the main tension here is that you're trying
to view Plan 9 from User Space as a new piece of Unix
software, and you're expecting it to conform to the
Unix conventions.  This isn't always a bad thing to do,
but it isn't my first priority.  My first priority is to
have all the software I missed from Plan 9 building on
the many Unix systems that I am now forced to use, and to
create an environment where bringing new software over is
easy to trivial.  On this count, I think I've succeeded.
In fact, a week or two ago, instead of trying to set up
an lpr or cups server on my Debian system, I ported Plan
9's lp, which took under an hour.

> That makes sense, but with some modularization in
> mind, this could be separated into some dev-wrapper
> package/directory.

I don't know what a dev-wrapper package/directory is,
but it sounds scary.

> The main idea is to modularize the whole p9p into
> sub-pieces, that one don't needs to install all 420kSLOC
> if one wants only depend on some p9 libs or on some bits
> of the whole thing, ie acme only.

Yes, please, try it.  I'm be interested to see what you
come up with.

I would like to see more libraries separated out so
that other projects can use them without the whole tree.
I think (but am not sure) that the best way to do this
is to separate them explicitly (look in $PLAN9/unix for
the scripts that pull out utf, fmt, bio, regexp, and mk)
rather than change the main tree.  Lib9 in particular
is the main sticking point, since libc.h #defines a lot
of standard Unix symbols (like open and dup and pipe),
making it not suitable for use as a regular Unix library.

> > Well, isn't programs compiled with p9p statically linked?
> 
> No, they are dynamically linked by default.

You're both right.  The Plan 9 libraries are static
libraries but the system libraries underneath (-lc -lm
-lpthread) are linked the default way, which usually
dynamically.

Russ

	9pclient.h       lib9pclient
	9p.h             lib9p
	ar.h             anyone who cares (mk, libmach)
	auth.h           libauth
	authsrv.h        libauthsrv
	bin.h            libbin
	bio.h            libbio
	complete.h       libcomplete
	cursor.h         libdraw
	diskfs.h         libdiskfs
	disk.h           libdisk
	draw.h           libdraw
	event.h          libevent
	fcall.h          lib9
	flate.h          libflate
	fmt.h            libfmt (source in lib9/fmt)
	frame.h          libframe
	geometry.h       libgeometry
	html.h           libhtml
	httpd.h          libhttpd
	ip.h             libip
	keyboard.h       libdraw
	lib9.h           lib9
	libc.h           lib9 (sic)
	libsec.h         libsec
	libString.h      libString
	mach.h           libmach
	memdraw.h        libmemdraw
	memlayer.h       libmemlayer
	mouse.h          libdraw
	mp.h             libmp
	mux.h            libmux
	ndb.h            libndb
	nfs3.h           libsunrpc
	plumb.h          libplumb
	regexp9.h        libregexp
	regexp.h         libregexp
	sunrpc.h         libsunrpc
	thread.h         libthread
	u.h              (no library, system setup)
	utf.h            libutf (source in lib9/utf)
	venti.h          libventi