[9fans] dirty blocks in cwfs

[9fans] dirty blocks in cwfs

[arisawa]

Hello,

It seems my cwfs has too many dirty blocks.
cfws command ("dump", "check tag" and etc) is no help to decrease these blocks.
is this only to me?

cwstats main
	filesys main
		maddr  =        3
		msize  =    10313
		caddr  =     1035
		csize  =  1392255
		sbaddr =  2129493
		craddr =  2129540  2129540
		roaddr =  2129543  2129543
		fsize  =  2129545  2129545  0+30%
		slast  =           2129437
		snext  =           2129544
		wmax   =  2129543           0+30%
		wsize  =  6972701           1+ 0%
		 46964 none
		202107 dirty
		     0 dump
		1143005 read
		   179 write
		     0 dump1
		cache 16% full

Re: [9fans] dirty blocks in cwfs

[erik quanstrom]

On Tue Mar  5 01:03:23 EST 2013, arisawa@ar.aichi-u.ac.jp wrote:
> Hello,
>
> It seems my cwfs has too many dirty blocks.
> cfws command ("dump", "check tag" and etc) is no help to decrease these blocks.
> is this only to me?

i have not seen this with kenfs.  is it possible that you
are using the "t" bit that 9front added?  if so, that would
explain all the dirty blocks that never get dumped.
one also ends up with dirty superblocks that can't get
dumped because they're already written, about 1 per
day in the dump.

one thing i notice about your cache-worm is that the
cache is very large.  this is not advisable because your
cache buckets will take up too many memory buffers,
leaving little for actual data.  20G cache has been
enough.  the first big file server i set up had 750G
cache, and it thrashed the heck out of its disks.

- erik

Re: [9fans] dirty blocks in cwfs

[arisawa]

thanks erik,

> the "t" bit that 9front added
how to do?
I don't know this one.

I analyzed map area in fscache and then found:
corresponding worm blocks of dirty cache blocks are garbage.

On 2013/03/05, at 15:22, erik quanstrom <quanstro@quanstro.net> wrote:

> On Tue Mar  5 01:03:23 EST 2013, arisawa@ar.aichi-u.ac.jp wrote:
>> Hello,
>> 
>> It seems my cwfs has too many dirty blocks.
>> cfws command ("dump", "check tag" and etc) is no help to decrease these blocks.
>> is this only to me?
> 
> i have not seen this with kenfs.  is it possible that you
> are using the "t" bit that 9front added?  if so, that would
> explain all the dirty blocks that never get dumped.
> one also ends up with dirty superblocks that can't get
> dumped because they're already written, about 1 per
> day in the dump.
> 
> one thing i notice about your cache-worm is that the
> cache is very large.  this is not advisable because your
> cache buckets will take up too many memory buffers,
> leaving little for actual data.  20G cache has been
> enough.  the first big file server i set up had 750G
> cache, and it thrashed the heck out of its disks.
> 
> - erik
> 

Re: [9fans] dirty blocks in cwfs

[arisawa]

Hello,

I reconsidered my problem. I said:
> corresponding worm blocks of dirty cache blocks are garbage.
now I think, this is not a problem. this is a state that it should be.
the problem is only in the fact: my dirty blocks wouldn't be dumped.

Re: [9fans] dirty blocks in cwfs

[erik quanstrom]

On Tue Mar  5 09:02:18 EST 2013, arisawa@ar.aichi-u.ac.jp wrote:
> Hello,
>
> I reconsidered my problem. I said:
> > corresponding worm blocks of dirty cache blocks are garbage.
> now I think, this is not a problem. this is a state that it should be.
> the problem is only in the fact: my dirty blocks wouldn't be dumped.

have you ever seen information about "%lld blocks dumped"
in cwfs' log?

you might consider using acid to see if any processes are stuck
doing i/o.  though on second thought, that seems iffy.

- erik

Re: [9fans] dirty blocks in cwfs

[arisawa]

Hello,

I have created dirty blocks that is not cleared by dump command.
the process is as follows:
(1) make a directory tree in somewhere.
	then we can observe these fscache blocks are in dirty state, 
	as mentioned Geoff's documents.
(2) remove the tree using clri command.
        then we can confirm these fscache blocks still exist with dirty state.
(3) dump
(4) reboot
(5) we can again confirm these fscache blocks still exist with dirty state.

I have believed the dump command clears all dirty block in fscache,
however, it seems the command clears only dirty blocks that is referenced
by main tree.

FS(8) says:
 A subsequent check free will place the abandoned storage in the free list.

I did't "check free" immediately after clri.

Re: [9fans] dirty blocks in cwfs

[cinap_lenrek]

this is correct. in general, the filesystem repairing aside, dump
will always just visit blocks that are referenced in the filesystem.

for example, if you have 0 dirty blocks and then temporarily create
100MB file and then delete it *before* it is dumped. then you got
100MB of dirty blocks in the cache. these blocks will not be cleaned
out by dump as the blocks are not referneced in the filesystem. these
blocks belong to no files but are chained in the free list. these
blocks are not lost. they will be used when allocating storage for
files. if these files get dumped, the dirty pool shrinks.

--
cinap

Re: [9fans] dirty blocks in cwfs

[erik quanstrom]

> for example, if you have 0 dirty blocks and then temporarily create
> 100MB file and then delete it *before* it is dumped. then you got
> 100MB of dirty blocks in the cache. these blocks will not be cleaned
> out by dump as the blocks are not referneced in the filesystem. these
> blocks belong to no files but are chained in the free list. these
> blocks are not lost. they will be used when allocating storage for
> files. if these files get dumped, the dirty pool shrinks.

these blocks may not be lost in the cache, but they are lost to the worm.

- erik