[9fans] cwfs

[9fans] cwfs

[cinap_lenrek]

Have converted my fossil/venti filesystem to cwfs (with history) and
made boot code so that i can start off from it. On the way i had to change
cwfs a bit so that it can cope with the long filenames from my mp3
collection and as it was supposed to run on my cpu server, i reduced
the cache memory size to be a quarter of all free userspace memory.

converting fossil to cwfs:

used the cphist program from http://www.quanstro.net/plan9/history.pdf
and wrote a /dev/(bin)time faking fileserver
(/n/sources/contrib/cinap_lenrek/delorean) that fools cwfs and replica
about the current time, but does not confuse cron and fossil.  you
have to manualy convert fossils /adm/users to cwfs format.  the
sequence is this: start cwfs type "allow" and "users default".  then
mount it and copy a users file in the root.  then type "users /users"
and remove the file again before doing the replica.  after replica
copy that file again to /adm/users so that on next startup the owners
and groups get loaded correctly.  for the replica you better make a
script and check the fossil dumps for corrupted directories.  i had
several metadata corruptions in the root direcotiry of fossil dumps
that caused replica to remove anything just to recreate it on the next
round. in the end i made a list of bad dumps that got skpied in
the script. the good news is that you can do all this while your fossil
cpuserver is running so if you find problems you can start over again
and to crazy things while anything is still running.

booting from cwfs:

/n/sources/contrib/cinap_lenrek/local.c (copy to /sys/src/9/boot/local.c)

the important part of the kernel configuration looks like this:

boot cpu boot w0
	tcp
	local

bootdir
	bootpccpuc.out boot
	/386/bin/ip/ipconfig
	/386/bin/auth/factotum
	/386/bin/cwfs

cwfs as the original stand alone fileserver has its own device names. cwfs
needs a device mapping file that maps plan9 files to cwfs devices.
connectlocalcwfs() builds that mapping dynamicly from plan9.ini keys
in the form wN=file. so my plan9.ini looks like this:

bootfile=sdE0!9fat!9pccpuc
bootargs=local!w0
bootdisk=local!w0
w0=#S/sdE1/data

on boot, cwfs is started in config mode so one can recover from errors. after
typing "end" on the cwfscmd promt stdin of cwfs is put in /srv/cwfscmd
and the filesystem gets mounted.

misc:

dont forget to change fshalt to do something like this:

f=...
k=...
c=`{ls /srv/cwfs*cmd >[2]/dev/null}

for (i in $c){
	echo -n $i...
	echo halt >>$i
}

cwfs needs to be halted correctly as it has no jurnaling or softupdates.

--
cinap

Re: [9fans] cwfs

[erik quanstrom]

> used the cphist program from http://www.quanstro.net/plan9/history.pdf
> and wrote a /dev/(bin)time faking fileserver
> (/n/sources/contrib/cinap_lenrek/delorean) that fools cwfs and replica
> about the current time, but does not confuse cron and fossil.  you

very sneaky!  i love it.
thanks for the write up.

> cwfs needs to be halted correctly as it has no jurnaling or softupdates.

mmm.  i have not had any trouble with unexpected shutdowns.
and i have had a few.  maybe not enough.

i could have just been lucky, but i suspect that ken was much
craftier than that.  i suspect, but haven't taken the time to verify,
that ken rolls up to the root so while one may write unfinished
garbage to the cache device, you won't see corruption.  unless
your fancy disk has reorderd writes in a bad way. :-)

you may wish to take the change that the diskless fs code has
to the cache on-disk layout.  by going column wise one can
eliminate the seek between every successive block.  you will
need a cache big enough to never resize.  what you give up
is the ability to add cache during startup.  (is there even a
console at startup with cwfs?)

- erik

Re: [9fans] cwfs

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 1102 bytes --]

yes, here is a console... buts quite hacky:

	strcpy(buf, "end");
	for(;;){
		outin("cwfscmd", buf, sizeof buf);
		if(strlen(buf) > 0)
			fprint(p[0], "%s\n", buf);
		if(strcmp(buf, "end")==0)
			break;
		buf[0] = 0;
	}

p[0] is our end of the pipe to cwfs stdin.  if you do nothing or hit
enter it just writes "end" and starts to connect to the fs.  but you
could type cwfs command then this becomes something like a console
input loop.  stdout and stderr of cwfs go to the cpuservers console.

hmmm...  as i'm thinking of it...  this is broken...  if for example
you want to do a devopy, you always have to type end before it starts
doing its work right?  but init will continue and try to mount it wich
will fail in this case panic the kernel.

maybe a better way is this:

	/* ask for cwfs config mode, will timeout in 15 seconds and continue mounting */
	strcpy(buf, "No");
	outin("config mode", buf, sizeof buf);
	if(buf[0]=='Y' || buf[0]=='y'){
		for(;;)
			if((i = read(0, buf, sizeof buf)) > 0)
				write(p[0], buf, i);
	}
	fprint(p[0], "end\n");

--
cinap

[-- Attachment #2: Type: message/rfc822, Size: 3077 bytes --]

From: erik quanstrom <quanstro@quanstro.net>
To: 9fans@9fans.net
Subject: Re: [9fans] cwfs
Date: Fri, 8 May 2009 12:39:31 -0400
Message-ID: <1814230b714601cd6e55fdc37232f6a3@quanstro.net>

> used the cphist program from http://www.quanstro.net/plan9/history.pdf
> and wrote a /dev/(bin)time faking fileserver
> (/n/sources/contrib/cinap_lenrek/delorean) that fools cwfs and replica
> about the current time, but does not confuse cron and fossil.  you

very sneaky!  i love it.
thanks for the write up.

> cwfs needs to be halted correctly as it has no jurnaling or softupdates.

mmm.  i have not had any trouble with unexpected shutdowns.
and i have had a few.  maybe not enough.

i could have just been lucky, but i suspect that ken was much
craftier than that.  i suspect, but haven't taken the time to verify,
that ken rolls up to the root so while one may write unfinished
garbage to the cache device, you won't see corruption.  unless
your fancy disk has reorderd writes in a bad way. :-)

you may wish to take the change that the diskless fs code has
to the cache on-disk layout.  by going column wise one can
eliminate the seek between every successive block.  you will
need a cache big enough to never resize.  what you give up
is the ability to add cache during startup.  (is there even a
console at startup with cwfs?)

- erik

Re: [9fans] cwfs

[erik quanstrom]

> hmmm...  as i'm thinking of it...  this is broken...  if for example
> you want to do a devopy, you always have to type end before it starts
> doing its work right?  but init will continue and try to mount it wich
> will fail in this case panic the kernel.

i'm not sure why devcopy would be useful with cwfs.
dd should work just fine outside the fs.

the diskless fs doesn't have a config-time devcopy but you
can run it at the normal console prompt.  this is good for
updating your kernel.

- erik

Re: [9fans] cwfs

[cinap_lenrek]

[-- Attachment #1: Type: text/plain, Size: 343 bytes --]

you are right... just did it... dd'ed the worm partition to another
disk rebooted... changed main configuration to mirror the
worm and added other filesystem. halt... reboot... done.

i updated the local.c in my contrib so that config mode never
ends or timeouts after it was selected.

i'm happy now. look ma! no fossil!

--
cinap

[-- Attachment #2: Type: message/rfc822, Size: 2417 bytes --]

From: erik quanstrom <quanstro@quanstro.net>
To: 9fans@9fans.net
Subject: Re: [9fans] cwfs
Date: Fri, 8 May 2009 15:10:50 -0400
Message-ID: <061ac3481db4fdd77bb856033bbd6697@quanstro.net>

> hmmm...  as i'm thinking of it...  this is broken...  if for example
> you want to do a devopy, you always have to type end before it starts
> doing its work right?  but init will continue and try to mount it wich
> will fail in this case panic the kernel.

i'm not sure why devcopy would be useful with cwfs.
dd should work just fine outside the fs.

the diskless fs doesn't have a config-time devcopy but you
can run it at the normal console prompt.  this is good for
updating your kernel.

- erik

[9fans] cwfs

[arisawa]

Hello,

 I have been playing with cwfs64x on 9front since last summer as my home system.
My current chief concern is, how to restore data when the disk is corrupted.
I have a copy of /dev/sdC0/fsworm on  /dev/sdD0/fsworm.
However, I don't understand how to ream fscache based on the existing fsworm.
Cwfs64x complains "badtag" in fscache and exits...

I believe there is no much difference in administration between two release: Geoff's and 9front.

Kenji Arisawa.

Re: [9fans] cwfs

[cinap_lenrek]

see the recover command from the fsconfig(8) manual. to enter
config mode, you have to pass the -c option to cwfs. on boot, you
can do it on the bootargs line like local!/dev/sdXX/fscache -c
(thats the only 9front specific part, afaik theres no local cwfs root
filesystem support by the standard distribution)

then enter:

recover main
end

that should ream the cache and reinitialize the superblock from
the last dump.

also note, that config mode and the normal fileserver console are
different and accept different command.

i dont know to what degree your cache partition got damaged. how
did this happen? whats the tag error? is the the contents of the
configuration block on the cache still intact? if not, then you
would have to retype the filesystem configuration prior the recover
or it would have no idea what you mean with "main" filesystem name.

be carefull and good luck.

--
cinap

Re: [9fans] cwfs

[arisawa]

Thanks Cinap,

that is merely an exercise for disk corruption.
I assumed /dev/sdC0 is corrupted.
the back up is on /dev/SD0 with partitions 9fat, nvram, fscache, fsworm, other,...
9fat, nvram on D0 is same as those of C0.
cwfs config in the first block of /dev/sdC0/fscache is copied to that of D0.
blocks in fsworm on C0 are copied to fsworm on D0 up to last super block (snext in console).
and then the D0 disk is moved to C0.

on reboot, I passed the -c option for the selection prompt.
the followings are my input for config prompt:
service cwfs
filsys main c(/dev/sdC0/fscache)(/dev/sdC0/fsworm)
filsys dump o
filsys other (/dev/sdC0/other)
noauth
newcache
recover
end

the response were:
check tag PC=12336 w"/dev/sdC0/fscache"(0) tag/path=<badtag> /15.....36: expected <nil>
and then returned back to the selection prompt.

why tag/path in fscache is the problem in recovering?
they must be cleaned based on fsworm.

anyone did similar exercise as me?

On 2013/02/22, at 17:55, cinap_lenrek@gmx.de wrote:

> see the recover command from the fsconfig(8) manual. to enter
> config mode, you have to pass the -c option to cwfs. on boot, you
> can do it on the bootargs line like local!/dev/sdXX/fscache -c
> (thats the only 9front specific part, afaik theres no local cwfs root
> filesystem support by the standard distribution)
> 
> then enter:
> 
> recover main
> end
> 
> that should ream the cache and reinitialize the superblock from
> the last dump.
> 
> also note, that config mode and the normal fileserver console are
> different and accept different command.
> 
> i dont know to what degree your cache partition got damaged. how
> did this happen? whats the tag error? is the the contents of the
> configuration block on the cache still intact? if not, then you
> would have to retype the filesystem configuration prior the recover
> or it would have no idea what you mean with "main" filesystem name.
> 
> be carefull and good luck.
> 
> --
> cinap
> 

Re: [9fans] cwfs

[arisawa]

sorry

/dev/SD0 is typo.
please replace by /dev/sdD0

Kenji Arisawa

Re: [9fans] cwfs

[erik quanstrom]

> the response were:
> check tag PC=12336 w"/dev/sdC0/fscache"(0) tag/path=<badtag> /15.....36: expected <nil>
> and then returned back to the selection prompt.
>
> why tag/path in fscache is the problem in recovering?
> they must be cleaned based on fsworm.

on the real file server, the command is

	recover main

note the fs name.  and one would expect a stream of
"dump %lld is good; %lld next\n" messages.

peeking at the source, i would expect similar from
cwfs.

- erik

Re: [9fans] cwfs

[arisawa]

Thanks, erik and cinap,

I have believed that tag/path of first block in fscache is to be created by config mode.
but not.
I retried complete copy (16KB copy) of the first block of fscache.
as erik pointed me.
	recover main
	end
is enough.

thanks again

Kenji Arisawa

On 2013/02/22, at 19:40, erik quanstrom <quanstro@quanstro.net> wrote:

>> the response were:
>> check tag PC=12336 w"/dev/sdC0/fscache"(0) tag/path=<badtag> /15.....36: expected <nil>
>> and then returned back to the selection prompt.
>> 
>> why tag/path in fscache is the problem in recovering?
>> they must be cleaned based on fsworm.
> 
> on the real file server, the command is
> 
> 	recover main
> 
> note the fs name.  and one would expect a stream of
> "dump %lld is good; %lld next\n" messages.
> 
> peeking at the source, i would expect similar from
> cwfs.
> 
> - erik
> 

Re: [9fans] cwfs

[erik quanstrom]

> I retried complete copy (16KB copy) of the first block of fscache.
> as erik pointed me.
> 	recover main
> 	end
> is enough.

great!  i've done that twice to recover something that wasn't
quite right.  in both cases help from hardware, and operational
mistakes both were required to get to that point.  (using aoe,
lose ethernet connection to mirror of worm while initiating
dump and rebooting instead of fixing the network connection.)

even so, i keep meaning to find the time to fix this bug.  the
fs should not care when it goes down.

i've also found that the fworm device was more trouble than
it was worth to me.  :-)  especially in those cases, you just
rewrite the new superblock on next dump and nobody is
wiser.

- erik