[9fans] now for something completely devfs :)

[9fans] now for something completely devfs :)

[andrey mirtchovski]

Now I have a venti, I'm keeping score, emailing it to myself every morning
(thanx to nemo's suggestions!) and am ready to embark on something that'll
break the machine completely! I decide to create a devfs mirror.

The fossil installation page on the wiki gives some instructions, but the
main part is missing -- how to write the first disk to the second one?

In the hardware RAID world, there are utilities provided for copying a hard
drive 'en masse' to others to create the mirror. With devfs I'm supposed to
do it myself.

But what do I copy? The entire drive (dd -if /dev/sdC0/data -of /dev/sdD0/data)?
Partition by partition? The wiki says:

	If you already have a fossil and want to add another disk to
	mirror it, you can setup the mirror as said before, start your
	fossil and while you are using it, use dd(1) to copy the first
	device to the new one. Since writes go to all devices but reads
	are performed just by the first device as long as it works it
	can be done this way.

but what does 'other one' mean here? am I supposed to do something like:

	dd -if /dev/sdC0/isect -of /dev/fs/isect

? (ok, I try that -- I echo a file to sdD0's isect, run the command, it
takes much longer than normal, completes, I run cmp on the two drives
-- they're not the same.)

Another thing: I should *not*, repeat *not* mirror 9fat with devfs, right?
Because if I do the machine becomes unbootable. MBR... shows up, but 9load
is nowhere to be found.

also: how do I know if one of my /dev/fs/ mirrored disks has crashed? I
mean, without rebooting :)

andrey

Re: [9fans] now for something completely devfs :)

[andrey mirtchovski]

forgot to add that at the end I did something like:

	for (i in nvram 9fat isect fossil arenas) {
		dd -if /dev/sdC0/$i -of /dev/sdD0/$i -bs 8192
		if(cmp /dev/sdC0/$i /dev/sdD0/$i) { end=success }
		if not{ end=failure }
		echo $end $i | mail mirtchov@cpsc
	}

naturally the only one that was a failure was fossil, since things like cron
write to /sys/log every 10 minutes (and 'dd' takes longer than that).

hopefully devfs knows how to miraculously fix that, otherwise I don't see
how I can ever have two identical mirrored disks...

of course, all this was before I found out that 9fat shouldn't be mirrored..

:)

ps: OT note -- it's interesting how trust is distributed among various
operating systems, at least for me:

	- i'd never trust a linux system on an open network. i'm very paranoid
	  about this box here, run firewalls and what have you
	- i'm fine with a freebsd box on the network -- it's much less chaotic,
	  and i know that once fixed, the set of utilities that open ports to
	  the internet will not change
	- i'm fine with a plan9 'out-of-the-box' install on the internet.. it
	  instills trust in me that so far nobody has broken. i can just leave
	  the box and never worry about it.

on the other hand:

	- i couldn't care less if the disk on this linux box crashed.. i know i
	  have important information on it, but somehow i can't be bothered to
	  worry about backups
	- same data, but on freebsd -- I'd back that up several times a week,
	  i don't know why, just seems reasonable, since the setup on the box is
	  so nice and thorough and andrey-appeasing... it took time to arrive
	  there, i don't want to do it again...
	- plan9 -- here i'm paranoid, i can set up a file server in less than
	  half a day now, but I have so many personal scripts and modifications
      to the system, that losing them would feel like I'm completely
	  naked...

i'll go fix that 9fat partition now -- I keep a spare kfs disk just for that
purpose :)

Re: [9fans] now for something completely devfs :)

[Fco.J.Ballesteros, nemo]

[-- Attachment #1: Type: text/plain, Size: 423 bytes --]

I did setup the mirror before formatting venti/fossil partitions.

Otherwise, you'd need to copy one disk to another while the partitions
are idle.

Let's say you have /dev/fs/mirror out of /dev/sdC0/part and /dev/sdC1/part

If you add /dev/sdD0/part you could dd from either /dev/fs/mirror or
from /dev/sdC[01]/part to /dev/sdD0/part. You should also add the device
to devfs.

I can help somehow, let me know.

[-- Attachment #2: Type: message/rfc822, Size: 3659 bytes --]

From: andrey mirtchovski <mirtchov@cpsc.ucalgary.ca>
To: 9fans@cse.psu.edu
Subject: [9fans] now for something completely devfs :)
Date: Wed, 9 Jul 2003 09:42:02 -0600 (MDT)
Message-ID: <Pine.LNX.4.44.0307090906010.8112-100000@fbsd.cpsc.ucalgary.ca>

Now I have a venti, I'm keeping score, emailing it to myself every morning
(thanx to nemo's suggestions!) and am ready to embark on something that'll
break the machine completely! I decide to create a devfs mirror.

The fossil installation page on the wiki gives some instructions, but the
main part is missing -- how to write the first disk to the second one?

In the hardware RAID world, there are utilities provided for copying a hard
drive 'en masse' to others to create the mirror. With devfs I'm supposed to
do it myself.

But what do I copy? The entire drive (dd -if /dev/sdC0/data -of /dev/sdD0/data)?
Partition by partition? The wiki says:

	If you already have a fossil and want to add another disk to
	mirror it, you can setup the mirror as said before, start your
	fossil and while you are using it, use dd(1) to copy the first
	device to the new one. Since writes go to all devices but reads
	are performed just by the first device as long as it works it
	can be done this way.

but what does 'other one' mean here? am I supposed to do something like:

	dd -if /dev/sdC0/isect -of /dev/fs/isect

? (ok, I try that -- I echo a file to sdD0's isect, run the command, it
takes much longer than normal, completes, I run cmp on the two drives
-- they're not the same.)

Another thing: I should *not*, repeat *not* mirror 9fat with devfs, right?
Because if I do the machine becomes unbootable. MBR... shows up, but 9load
is nowhere to be found.

also: how do I know if one of my /dev/fs/ mirrored disks has crashed? I
mean, without rebooting :)

andrey

Re: [9fans] now for something completely devfs :)

[andrey mirtchovski]

My fossil is working, but I still want to know if /dev/sdC0/fossil (for
example) changes while I'm dd-ing it to /dev/sdD0/fossil, will devfs take
care of that discrepancy?

The other question -- is there a way to know whether one or more of the
mirrored disks have crashed and take measures? Watch the HDD light on the
front panel? (A RAID controller we have here actually says that the only way
to know if a disk has crashed is to reboot the machine -- then the
controller BIOS will detect it.)

andrey

On Wed, 9 Jul 2003, Fco.J.Ballesteros wrote:

> I did setup the mirror before formatting venti/fossil partitions.
>
> Otherwise, you'd need to copy one disk to another while the partitions
> are idle.
>
> Let's say you have /dev/fs/mirror out of /dev/sdC0/part and /dev/sdC1/part
>
> If you add /dev/sdD0/part you could dd from either /dev/fs/mirror or
> from /dev/sdC[01]/part to /dev/sdD0/part. You should also add the device
> to devfs.
>
> I can help somehow, let me know.
>

Re: [9fans] now for something completely devfs :)

[Fco.J.Ballesteros, nemo]

[-- Attachment #1: Type: text/plain, Size: 544 bytes --]

if you have several partitions configure in devfs, them all are
updated. If you config the mirror before dding, then dd takes
care of everything not updated during the dd, and devfs takes
care of things updated while dding. Since devfs services reads
from the first device (as long as it works), there's no problem
in using it.

There's a race of course, but you can probably check later with cmp
(I did that a couple of times and the race did not hurt me). I'd try
as hard as feasible to keep the partitions untouched, just in case...

[-- Attachment #2: Type: message/rfc822, Size: 3220 bytes --]

From: andrey mirtchovski <mirtchov@cpsc.ucalgary.ca>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] now for something completely devfs :)
Date: Wed, 9 Jul 2003 11:19:46 -0600 (MDT)
Message-ID: <Pine.LNX.4.44.0307091113390.8525-100000@fbsd.cpsc.ucalgary.ca>

My fossil is working, but I still want to know if /dev/sdC0/fossil (for
example) changes while I'm dd-ing it to /dev/sdD0/fossil, will devfs take
care of that discrepancy?

The other question -- is there a way to know whether one or more of the
mirrored disks have crashed and take measures? Watch the HDD light on the
front panel? (A RAID controller we have here actually says that the only way
to know if a disk has crashed is to reboot the machine -- then the
controller BIOS will detect it.)

andrey

On Wed, 9 Jul 2003, Fco.J.Ballesteros wrote:

> I did setup the mirror before formatting venti/fossil partitions.
>
> Otherwise, you'd need to copy one disk to another while the partitions
> are idle.
>
> Let's say you have /dev/fs/mirror out of /dev/sdC0/part and /dev/sdC1/part
>
> If you add /dev/sdD0/part you could dd from either /dev/fs/mirror or
> from /dev/sdC[01]/part to /dev/sdD0/part. You should also add the device
> to devfs.
>
> I can help somehow, let me know.
>

Re: [9fans] now for something completely devfs :)

[Geoff Collyer]

I think the paragraph on the wiki would be clearer like this:

	If you already have a fossil and want to add another disk to mirror
	it, you can setup the mirror as said before, with the original fossil
	device (possibly a partition) being mirrored by the new disk, start
	your fossil and while you are using it, use dd(1) to copy the fossil
	device to the mirror device or partition.  Since writes go to all
	devices in the mirror but reads are performed just by the first device
	(as long as it has no read errors), the copy can be made this way with
	relative safety.  Races are still possible, since dd might read a
	block from the fossil device, fossil might change it and write it to
	both devices (original fossil and mirror) and then dd might write the
	now-stale block to the mirror, so it's safer to make the copy when
	fossil (and venti, if you mirror it) are idle.  This might require
	booting your file server off another standalone Plan 9 file & cpu
	server.

How do you know if a disk in a mirror has died?  The kernel should
print I/O-error messages on the console.  devfs should cope until you
can shut the system down and replace the dying disk.  With modern
disks, I/O errors are generally a sign of degradation: the disk has
forwarded as many bad sectors as it can and now the reserved bad
sector areas are full.

Re: [9fans] now for something completely devfs :)

[Geoff Collyer]

Andrey, I can't get mail to you; it bounces with this:
---
Your request ``smtp /net.alt/tcp!cpsc.ucalgary.ca mirtchov '' failed (code smtp 685: Permanent Failure).
The symptom was:

Wed Jul  9 15:58:59 PDT 2003 connect to /net.alt/tcp!cpsc.ucalgary.ca:
554 fsa.cpsc.ucalgary.ca ESMTP not accepting messages
---

Anyway, yes, I'll update the wiki.

Re: [9fans] now for something completely devfs :)

[Scott Schwartz]

| How do you know if a disk in a mirror has died?  The kernel should
| print I/O-error messages on the console.

Maybe /dev/fs/status could do the job, too.

Re: [9fans] now for something completely devfs :)

[boyd, rounin]

> | How do you know if a disk in a mirror has died?  The kernel should
> | print I/O-error messages on the console.
>
> Maybe /dev/fs/status could do the job, too.

death of a mirror/raid disk is something you want to know about.

those Digital StorageWorks raid arrays were something, once
you had them set up.  i had disks dying every month but the array
would pick a free disk and just continue.

iirc, the RZ-2[89]'s were incredibly unreliable.  good thing i had
them in mirrors or raid 5.

the interface was a bit VMS-ey, but it walked all over those horrible
EMC2 fiber channel horrors ...

Re: [9fans] now for something completely devfs :)

[Fco.J.Ballesteros, nemo]

[-- Attachment #1: Type: text/plain, Size: 255 bytes --]

The kernel does print messages. Doesn't it?
from devfs.c:
			if (waserror()){
				// if a read fails we let the user know and try
				// another device.
				print("#k: mread: (%llx %d): %s\n",
					c->qid.path, i, up->errstr);
				continue;
			}

[-- Attachment #2: Type: message/rfc822, Size: 1958 bytes --]

From: Scott Schwartz <schwartz@bio.cse.psu.edu>
To: 9fans@cse.psu.edu
Subject: Re: [9fans] now for something completely devfs :)
Date: Wed, 09 Jul 2003 19:32:15 -0400
Message-ID: <20030709233215.25363.qmail@g.bio.cse.psu.edu>

| How do you know if a disk in a mirror has died?  The kernel should
| print I/O-error messages on the console.

Maybe /dev/fs/status could do the job, too.