Re: [9fans] The creepy WORM. (was: Re: Thinkpad T61 Installation Experience)

[Francisco J Ballesteros <nemo@lsub.org>]

[-- Attachment #1: Type: text/plain, Size: 3665 bytes --]




using ipad keyboard. excuse any typos.

>> 
> 
> Just curious.
> If the tree doesn't fit in memory, how do you decide who to
> kick out? LRU? Sounds much like a cache fs. What does it buy
> you over existing cache filesystems? Speaking more generally,
> not just in the plan9 context.
> 
> 

lru for clean blocks. but you really have the tree you use in memory, all if it fits.
what it buys is simplicity, thus reliability, and speed.
instead of a single program doing everything, you have several trying to use
their memory and to avoid copying blocks in the main server.
plus, it's going to be modified to exploit the upcoming nix zero copy framework.


>>          The disk is organized as a log of blocks. When a new version
>>          of the tree must be written to disk, all blocks that changed
>>          are given disk addresses and are appended to the log. Once
>>          written, they are frozen.  If new changes are made to the
>>          tree, blocks are melted and forget their previous addresses:
>>          each time they are written again, they are assigned new
>>          ones.
> 
> I don't understand use of the words frozen & melted here.  How
> is this different from how things work now? Something worse
> than what venti or zfs do, which is to leave the old blocks
> alone and allocate new space for new blocks.
> 

it's not cow. you reuse the memory of a frozen block instead of copying.
you just melt it and reuse it. 

all this is in memory. cow happens only on the disk, but you don't wait for that.
that's the main difference wrt others.



>>          When the disk gets full, all reachable blocks are marked and
>>          all other blocks are considered available for growing the
>>          log (this is a description of semantics, not of the imple-
>>          mentation). Thus, the log is circular but jumps to the next
>>          available block each time it grows.  If, after the mark pro-
>>          cess, the disk is still full, the file system becomes read
>>          only but for removing files.
> 
> Why does circularity matter? It would make more sense to allocate
> new blocks for a given file near its existing blocks regardless of
> writing order.
> 

for simplicity, I removed most of the fanciest things I had before in place in
previous versions that could be a source of bugs. there are no ref. counters,
for example. it's designed to operate on
main memory, and it seems it does well even though the disk algorithms are
naive.


> Why not just use venti or some existing FS underneath than
> come up with a new disk format?
> 

to avoid complexity, latency, and bugs.

it's now a set of tools, you can archive creepy into venti if you want, or archive
fossil into a creepy rip (it's the same program, actually).

for archival, you are going to use a pipe, and not a tcp connection.

you have a program half the size, or 1/4 depending on how you wc.

it takes half the time fossil takes in the silly tests I made, and you can understand
the code the first time you read it, which is not trivial with the others, but for Ken's.

last, it's expected not to give you corrupted files despite power failures, which we
had in both fossil and venti (I'm not saying its their fault, our environment is bumpy).

that was the motivation, exploiting large main memories and keeping things simple
and reliable. Time will tell if we managed to achieve that or not :)

sorry I wrote in Sioux this time. its been a long day here :)

> Sounds like a fun project but it would be nice to see the
> rationale for it.
> 
> Thanks!

[-- Attachment #2: Type: text/html, Size: 6233 bytes --]