[9fans] getting sdC0/nvram right in /bootdir

[9fans] getting sdC0/nvram right in /bootdir

[ron minnich]

Recap: I am loading Plan 9 direct from linuxbios, no 9load. Factotum, run
out of bootdir, is not seeing disk partitions, in particular #sdC0/nvram,
as no one has set them up to be seen. Normally, 9load sets them up; 9load
is not there.  Something needs to do that setup. Options are:

- rip the code out of 9load to do this
- rip the code out of disk/fdisk and disk/prep to do this
- something else

I looked at 9load, to see how it gets disk partitions set up for factotum
to find. Messy, it has to go dig around the disk looking for things. I
looked at ripping out bits of fdisk and prep to do the job, essentially
having a fdiskprep program in bootdir. Messy, got to rip them here and
there and put them back together. Frankencode.  Although I must say the
way fdisk and prep work and share code is quite slick.

So, it seems to me: once I load plan 9 from linuxbios from flash, and
we're coming up, we've got all the Plan 9 power there, including the
ability to put arbitrary programs into bootdir and run them. What I have
found is that writing the program in C that is equivalent to that 'set up
the partitions' script in termrc and cpurc is about 20 lines of C code.
The code looks like this:

- fd = open /dev/sdC0/ctl
- if fails, exits
- fork, dup(fd, 1),  exec disk/fdisk -p /dev/sdC0/data
- fails, exits
- fork, dup(fd, 1), exec disk/prep -p /dev/sdC0/plan9

That's it. Itsy C program, tested, works fine.

So why not:
- put disk/fdisk in bootdir, disk/prep in bootdir, and the little program
  in bootdir
- have the little program start right before factotum, and call disk/fdisk
  and disk/prep

Sure, it's more memory. My $8 CF parts are 32MB. Who cares. Is there any
real reason not to do it this way? It's easy, that's for sure. Making it
iterate over all the devices in nvramtab (in readnvram.c) is trivial.

The other option, not taken, is the new linux approach of actually running
shell scripts in bootdir (which in linux is called initrd). I think I'd
rather wait to do this ... do we really want /bin/rc in bootdir?

ron

Re: [9fans] getting sdC0/nvram right in /bootdir

[Charles Forsyth]

>>The other option, not taken, is the new linux approach of actually running
>>shell scripts in bootdir (which in linux is called initrd). I think I'd
>>rather wait to do this ... do we really want /bin/rc in bootdir?

that's how the venti+fossil contraption was bootstrapped at first.

Re: [9fans] getting sdC0/nvram right in /bootdir

[Richard Miller]

> The other option, not taken, is the new linux approach of actually running
> shell scripts in bootdir (which in linux is called initrd). I think I'd
> rather wait to do this ... do we really want /bin/rc in bootdir?

It's already been done. Look at /sys/src/9/pc/pcfl

bootdir
	/386/bin/rc
	/rc/lib/rcmain
	/386/bin/bind
	/386/bin/cat
	/386/bin/cp
	/386/bin/echo
	/386/bin/mount
	/386/bin/sleep
	... etc etc

-- Richard

Re: [9fans] getting sdC0/nvram right in /bootdir

[ron minnich]

On Sat, 8 Nov 2003, Richard Miller wrote:

> > The other option, not taken, is the new linux approach of actually running
> > shell scripts in bootdir (which in linux is called initrd). I think I'd
> > rather wait to do this ... do we really want /bin/rc in bootdir?
>
> It's already been done. Look at /sys/src/9/pc/pcfl
>
> bootdir
> 	/386/bin/rc
> 	/rc/lib/rcmain
> 	/386/bin/bind
> 	/386/bin/cat
> 	/386/bin/cp
> 	/386/bin/echo
> 	/386/bin/mount
> 	/386/bin/sleep
> 	... etc etc
>

well then I don't even need my silly program! I'll just run the shell
script.

Thanks. It is interesting to see how Plan 9 and Linux are doing pretty
much the same thing for booting nowadays.

ron

Re: [9fans] getting sdC0/nvram right in /bootdir

[Russ Cox]

> I looked at 9load, to see how it gets disk partitions set up for factotum
> to find. Messy, it has to go dig around the disk looking for things.

Where else would it look?

> I looked at ripping out bits of fdisk and prep to do the job, essentially
> having a fdiskprep program in bootdir. Messy, got to rip them here and
> there and put them back together. Frankencode.

That's what 9load's part.c *is*.  The work has been done for you.

I appreciate that shoving fdisk and prep into the kernel is
a quick fix, but I do not believe it is the right fix,
just like I don't believe that putting rc into the kernel
boot is good practice for a production system.  (It's great
for quick prototyping, but you can be a lot smarter about
errors and such from inside C.  Otherwise you have to toss
in oodles of helper programs for the rc script to use.)

The kernel is too big and bloated as it is.  I regret
having put fossil and venti into the pcf kernels, and
I'd rather not throw in yet another 150kB just to partition
the disk, when I can do it in 4kB by reusing existing code.
This is how we've ended up with such enormous kernels.
Ken's file server kernel is 560kB.  9pcf is over 2MB.
This is progress?  We're approaching Linux.

Here's the 4kB of code.  Like I said, it's almost just cut
and paste from 9load, and it will work in /boot just fine.

#include <u.h>
#include <libc.h>

#define	GSHORT(p)	(((p)[1]<<8)|(p)[0])
#define	GLONG(p)	((GSHORT(p+2)<<16)|GSHORT(p))
#define	GLSHORT(p)	(((p)[0]<<8)|(p)[1])
#define	GLLONG(p)	(((ulong)GLSHORT(p)<<16)|GLSHORT(p+2))

typedef struct Dosboot	Dosboot;
typedef struct Dos	Dos;
typedef struct Dosdir	Dosdir;
typedef struct Dosfile	Dosfile;
typedef struct Dospart	Dospart;

struct Dospart
{
	uchar flag;		/* active flag */
	uchar shead;		/* starting head */
	uchar scs[2];		/* starting cylinder/sector */
	uchar type;		/* partition type */
	uchar ehead;		/* ending head */
	uchar ecs[2];		/* ending cylinder/sector */
	uchar start[4];		/* starting sector */
	uchar len[4];		/* length in sectors */
};

#define FAT12	0x01
#define FAT16	0x04
#define EXTEND	0x05
#define FATHUGE	0x06
#define FAT32	0x0b
#define FAT32X	0x0c
#define EXTHUGE	0x0f
#define DMDDO	0x54
#define PLAN9	0x39
#define LEXTEND 0x85

struct Dosfile{
	Dos	*dos;		/* owning dos file system */
	char	name[8];
	char	ext[3];
	uchar	attr;
	long	length;
	long	pstart;		/* physical start cluster address */
	long	pcurrent;	/* physical current cluster address */
	long	lcurrent;	/* logical current cluster address */
	long	offset;
};

struct Dos{
	long	start;		/* start of file system */
	int	sectsize;	/* in bytes */
	int	clustsize;	/* in sectors */
	int	clustbytes;	/* in bytes */
	int	nresrv;		/* sectors */
	int	nfats;		/* usually 2 */
	int	rootsize;	/* number of entries */
	int	volsize;	/* in sectors */
	int	mediadesc;
	int	fatsize;	/* in sectors */
	int	fatclusters;
	int	fatbits;	/* 12 or 16 */
	long	fataddr;	/* sector number */
	long	rootaddr;
	long	rootclust;
	long	dataaddr;
	long	freeptr;
};

uchar *mbrbuf, *partbuf;
#define trace 0

#define Secsize ((vlong)512)

int
tsdbio(char *name, int fd, ulong dstart, void *a, ulong off, int mbr)
{
	uchar *b;

	if(pread(fd, a, Secsize, (dstart+off)*Secsize) != Secsize){
		if(trace)
			print("%s: read %llud at %llud failed\n", name,
				Secsize, (dstart+off)*Secsize);
		return -1;
	}
	b = a;
	if(mbr && (b[0x1FE] != 0x55 || b[0x1FF] != 0xAA)){
		if(trace)
			print("%s: bad magic %.2ux %.2ux at %lld\n",
				name, b[0x1FE], b[0x1FF],
				(dstart+off)*Secsize);
		return -1;
	}
	return 0;
}

/*
 *  read partition table.  The partition table is just ascii strings.
 */
#define MAGIC "plan9 partitions"
static void
oldp9part(char *name, int fd, ulong dsize)
{
	char *field[3], *line[100];
	ulong n, start, end;
	int i;

	/*
	 * We prefer partition tables on the second to last sector,
	 * but some old disks use the last sector instead.
	 */
	start = dsize-2;
	end = start+1;
	if(tsdbio(name, fd, 0, partbuf, start, 0) < 0)
		return;

	if(strncmp((char*)partbuf, MAGIC, sizeof(MAGIC)-1) != 0) {
		/* not found on 2nd last sector; look on last sector */
		start++;
		end++;
		if(tsdbio(name, fd, 0, partbuf, start, 0) < 0)
			return;
		if(strncmp((char*)partbuf, MAGIC, sizeof(MAGIC)-1) != 0)
			return;
		print("%s: using old plan9 partition table on last sector\n", name);
	}else
		print("%s: using old plan9 partition table on 2nd-to-last sector\n", name);

	print("part partition %lud %lud\n", start, end);

	partbuf[Secsize-1] = '\0';

	/*
	 * parse partition table
	 */
	n = getfields((char*)partbuf, line, nelem(line), 1, "\n");
	if(n && strncmp(line[0], MAGIC, sizeof(MAGIC)-1) == 0){
		for(i = 1; i < n; i++){
			if(getfields(line[i], field, 3, 1, " ") != 3)
				break;
			start = strtoul(field[1], 0, 0);
			end = strtoul(field[2], 0, 0);
			if(start >= end || end > dsize)
				break;
			print("part %s %lud %lud\n", field[0], start, end);
		}
	}
}

static void
p9part(char *name, int fd, ulong dstart, ulong dsize)
{
	char *field[4], *line[100];
	ulong start, end;
	int i, n;

	if(tsdbio(name, fd, dstart, partbuf, 1, 0) < 0)
		return;
	partbuf[Secsize-1] = '\0';

	if(strncmp((char*)partbuf, "part ", 5) != 0)
		return;

	n = getfields((char*)partbuf, line, nelem(line), 1, "\n");
	if(n == 0)
		return;
	for(i = 0; i < n; i++){
		if(strncmp(line[i], "part ", 5) != 0)
			break;
		if(getfields(line[i], field, 4, 1, " ") != 4)
			break;
		start = strtoul(field[2], 0, 0);
		end = strtoul(field[3], 0, 0);
		if(start >= end || end > dsize)
			break;
		print("part %s %lud %lud\n", field[1], dstart+start, dstart+end);
	}
}

int
isdos(int t)
{
	return t==FAT12 || t==FAT16 || t==FATHUGE || t==FAT32 || t==FAT32X;
}

int
isextend(int t)
{
	return t==EXTEND || t==EXTHUGE || t==LEXTEND;
}

/*
 * Fetch the first dos and all plan9 partitions out of the MBR partition table.
 * We return -1 if we did not find a plan9 partition.
 */
static int
mbrpart(char *name, int fd)
{
	ulong mbroffset;
	Dospart *dp;
	ulong start, end;
	ulong epart, outer, inner;
	int havedos, i, nplan9;
	char nname[10];

	if(tsdbio(name, fd, 0, mbrbuf, 0, 1) < 0)
		return -1;

	mbroffset = 0;
	dp = (Dospart*)&mbrbuf[0x1BE];
	for(i=0; i<4; i++, dp++)
		if(dp->type == DMDDO) {
			mbroffset = 63*512;
			if(trace)
				print("DMDDO partition found\n");
			if(tsdbio(name, fd, 0, mbrbuf, mbroffset, 1) < 0)
				return -1;
			i = -1;	/* start over */
		}

	nplan9 = 0;
	havedos = 0;
	epart = 0;
	dp = (Dospart*)&mbrbuf[0x1BE];
	if(trace)
		print("%s mbr ", name);
	for(i=0; i<4; i++, dp++) {
		if(trace)
			print("dp %d...", dp->type);
		start = mbroffset+GLONG(dp->start);
		end = start+GLONG(dp->len);

		if(dp->type == PLAN9) {
			if(nplan9 == 0)
				strcpy(nname, "plan9");
			else
				sprint(nname, "plan9.%d", nplan9);
			print("part %s %lud %lud\n", nname, start, end);
			p9part(name, fd, start, end-start);
			nplan9++;
		}

		/*
		 * We used to take the active partition (and then the first
		 * when none are active).  We have to take the first here,
		 * so that the partition we call ``dos'' agrees with the
		 * partition disk/fdisk calls ``dos''.
		 */
		if(havedos==0 && isdos(dp->type)){
			havedos = 1;
			print("part dos %lud %lud\n", start, end);
		}
		if(isextend(dp->type)){
			epart = start;
			if(trace)
				print("link %lud...", epart);
		}
	}
	if(trace)
		print("\n");

	/*
	 * Search through the chain of extended partition tables.
	 */
	outer = epart;
	while(epart != 0) {
		if(trace)
			print("%s ext %lud ", name, epart);
		if(tsdbio(name, fd, 0, mbrbuf, epart, 1) < 0)
			break;
		inner = epart;
		epart = 0;
		dp = (Dospart*)&mbrbuf[0x1BE];
		for(i=0; i<4; i++, dp++) {
			if(trace)
				print("dp %d...", dp->type);
			start = GLONG(dp->start);
			if(dp->type == PLAN9){
				start += inner;
				end = start+GLONG(dp->len);
				if(nplan9 == 0)
					strcpy(nname, "plan9");
				else
					sprint(nname, "plan9.%d", nplan9);
				print("part %s %lud %lud\n", nname, start, end);
				p9part(name, fd, start, end-start);
				nplan9++;
			}
			if(havedos==0 && isdos(dp->type)){
				start += inner;
				end = start+GLONG(dp->len);
				havedos = 1;
				print("part dos %lud %lud\n", start, end);
			}
			if(isextend(dp->type)){
				epart = start + outer;
				if(trace)
					print("link %lud...", epart);
			}
		}
		if(trace)
			print("\n");
	}

	return nplan9 ? 0 : -1;
}

enum {
	NEW = 1<<0,
	OLD = 1<<1,
};

void
partition(char *name)
{
	int fd;
	int type;
	ulong dend;
	Dir *d;

	if((fd = open(name, OREAD)) < 0)
		sysfatal("open %s: %r", name);
	if((d = dirfstat(fd)) == nil)
		sysfatal("dirfstat %s: %r", name);
	dend = d->length/Secsize;
	free(d);

	type = NEW|OLD;

	mbrbuf = malloc(Secsize);
	partbuf = malloc(Secsize);

	if((type & NEW) && mbrpart(name, fd) >= 0)
		{}
	else if(type & OLD)
		oldp9part(name, fd, dend);
}

void
usage(void)
{
	fprint(2, "usage: part disk\n");
	exits("usage");
}

void
main(int argc, char **argv)
{
	ARGBEGIN{
	default:
		usage();
	}ARGEND

	if(argc != 1)
		usage();

	partition(argv[0]);
	exits(0);
}

Re: [9fans] getting sdC0/nvram right in /bootdir

[ron minnich]

On Sat, 8 Nov 2003, Russ Cox wrote:

> Where else would it look?

Yep, and I can easily take that code, as you just did, and now we've got
two chunks of code to think about anytime anything changes; precisely why
I did not want to do that. I don't like it. Two is one too many.

That's the problem we've got right now with 9load/plan9. Just to get 9load
at all to work on some of my nodes I had to grab stuff out of plan 9 and
put it back into 9load. It was a real pain. That's code bloat --
duplicating capability in lots of different places when it should only be
in one place. Growing a ram disk with stuff like fdisk/prep is not code
bloat. It's ram disk bloat, maybe, but not code bloat. I just can't get
excited about 560KB kernels vs 2 MB kernels when my CF parts are so cheap.
The time difference to read it in is, realistically, 0.  bootdir size is
the wrong metric.

Thanks for the fixed up part.c, it's neat, and I'll probably just use it
and ignore my own arguments I just made :-)

thanks

ron
p.s. All opinions my own, and doubtless wrong :-)