[9fans] The creepy WORM. (was: Re: Thinkpad T61 Installation Experience)

[Nemo <nemo@lsub.org>]

Because I noticed Ken's worm fs was being discussed in this thread, I thought
I might just drop here the man page for a new alternate file server that we wrote
for nix.

It's not yet ready for use (I'm using it, but it's still under testing, and the version
in the main nix tree is now out of date, btw; will send the new one soon).

But I thought it might be of interest to 9fans, if only to get comments.


     CREEPY(4)                                               CREEPY(4)

     NAME
          9pix, fmt, rip, arch - creepy file server and WORM archive

     SYNOPSIS
          creepy/9pix [ -DFLAGS ] [ -ra ] [ -A addr ] [ -S srv ] disk

          creepy/fmt [ -DFLAGS ] [ -wy ] disk

          creepy/rip [ -DFLAGS ] [ -a ] [ -c n ] [ -A addr ] [ -S srv
          ] disk

          creepy/arch [ -DFLAGS ] [ dir ]

     DESCRIPTION
          Creepy is a prototype file server for Nix. It maintains a
          mutable file tree with unix semantics, kept in main memory,
          served through 9P, see intro(5), and through IX.

          Creepy/9pix is the main file server program. It serves a
          file tree through 9P and IX.  The tree kept in memory is
          mutable. But frozen versions of the tree are written to
          disk, both upon request and also on a periodic basis, to
          survive power outages and other problems, and to be able to
          operate on trees that do not fit on main memory.  The
          tree(s) stored on disk are frozen and cannot be changed once
          written.

          By default the program listens for 9P in the standard TCP
          port and posts a connection that can be mounted at
          /srv/9pix.  Flags -A and -S may be used to specify an alter-
          nate network address and/or srv(3) file to post. Using these
          flags makes the program not to listen and not to post to
          srv(3) unless a flag indicates so.  Flag -r makes the pro-
          gram operate in read-only mode, and flag -a starts the pro-
          gram without requiring authentication to mount the file
          tree.

          The disk is organized as a log of blocks. When a new version
          of the tree must be written to disk, all blocks that changed
          are given disk addresses and are appended to the log. Once
          written, they are frozen.  If new changes are made to the
          tree, blocks are melted and forget their previous addresses:
          each time they are written again, they are assigned new
          ones.

          When the disk gets full, all reachable blocks are marked and
          all other blocks are considered available for growing the
          log (this is a description of semantics, not of the imple-
          mentation). Thus, the log is circular but jumps to the next
          available block each time it grows.  If, after the mark pro-
          cess, the disk is still full, the file system becomes read
          only but for removing files.

          Before using a disk it must be formatted using creepy/fmt.
          This initializes blocks used to store marks for the mark and
          sweep process and also initializes a super block and an
          empty root directory. Flag -y forces the format even if the
          disk seems to contain a fossil (4) or creepy (4) partition.
          Flag -w is used to format the partition for a WORM
          (described later) and not for a main file tree.

          Creepy/rip is the same file server program, but operates in
          WORM mode. In this case, no mark and sweep for free blocks
          will ever happen. Blocks are consumed from the disk until it
          becomes full.  The file tree served is always read-only.

          Operating the WORM to in administrative mode to add more
          trees or new versions of the archived trees does not require
          any protocol: it can be done using the standard file inter-
          face used to operate on any other tree, by mounting and
          changing it.

          An alternate mount specifier, wormwr, can be used to mount
          the tree in read-write mode, to update the archive.  Updat-
          ing the archive is performed by creating new trees with the
          conventional /treename/year/mmdd paths on the WORM. But note
          that such paths are not enforced by the program at all.
          Before updating a tree in the archive, for a new day, a con-
          trol request described later can be used to link the direc-
          tory for the previous version of the archive to the new one.
          After that, changes made to files would in effect copy on
          write all blocks affected, and refer to old ones when they
          did not change.

          Creepy/arch is a program started by creepy/9pix to archive
          snapshots of the tree into a directory provided by
          creepy/rip (or by any other file server).  The program is
          not expected to be run by users, and speaks a simple proto-
          col through its standard input: A block address is written
          and the block contents are read from it.  Usually, the stan-
          dard input is a pipe to creepy/9pix. The program takes the
          address of the root directory as kept on disk and then asks
          the file server program for block contents, to archive them
          into a mounted directory. Thus, the tree archived is always
          consistent because only consistent trees can be read from
          disk (the mutable file tree is kept in memory).

          The file tree provided by 9pix and rip has the following
          layout:
               /
               /root
               /cons
               /stats
          And, by convention, archives kept in the WORM are expected
          to have names like:
               /root/treename/2012/0425
               /root/treename/2012/0425.1
               /root/treename/2012/0426

          Using the main attach specifier yields the tree as just
          shown. Using an empty specifier, or root, or main/active
          yields the tree found at /root.  In creepy/rip the attach
          specifier wormwr yields the root of the tree in read-write
          mode.

          ∙    The root directory is never found on disk. It is a
               placeholder for the contained files.

          ∙    Cons is a synthesized file used as a console for file
               system administration, and it is not found on disk. It
               is still subject to file permissions, and coordinates
               concurrent access to the console by different users.
               The owner of the file server process is always allowed
               to access the console.

          ∙    Stats is another synthesized file used to report
               statistics.

          ∙    Root contains the actual  file tree. It corresponds to
               the file tree as seen at this moment by file system
               users. It's semantics are similar to those of a UNIX or
               Plan 9 file system.

          Users and groups can be added by creating and editing
          /root/users . See an example shown later.

          The console accepts the following commands:

          allow [uid]
               Lets uid bypass permissions, or show allowed users if
               none is given.

          disallow [uid]
               undoes the effect of the previous command for the given
               uid or for everyone if none is given.

          halt Halts the file system and terminates the processes.

          stats
               Reports usage statistics.

          sync Synchronizes the disk by writing a new version of the
               tree to the log.

          uname [uid]
               Reports information about user uid.

          users
               Prints the list of users.

          who  Reports the list of network addresses and users cur-
               rently using the file system.

          arch name hour [path]
               Schedules an archival into the WORM with the given tree
               name and at the given hour. By default, the WORM is
               expected at /n/rip in administrative mode. An optional
               path may be given to use a different directory.

          arch now
               archives the tree. The previous form must be used
               before to configure archival.

          arch none
               disables archival (this is the default).

          link src dst
               is used in the WORM to initialize a new archive by
               linking the one of the previous day to a new name,
               before updating the tree.

          ?    Prints the list of known commands. More than those
               describe here will be shown, mostly for debugging.

     EXAMPLES
          Format a partition and a WORM
               creepy/fmt /dev/sdC0/creepy
               creepy/fmt -w /dev/sdC0/rip

          Run the file system, mount it, initialize the user list with
          one from fossil(4), and populate it
               creepy/9pix /dev/sdC0/creepy main
               mount -c /srv/9pix /n/creepy
               cp /adm/users /n/creepy/root/users
               echo allow $user >/n/creepy/cons
               dircp /n/dist /n/creepy/root
          Run the worm (the previously file server would mount it if
          it finds its file posted in srv(3) and is asked to archive a
          tree.
               creepy/rip /dev/sdC0/rip

     SOURCE
          /sys/src/cmd/creepy

     SEE ALSO
          fossil(4), venti(8)

     BUGS
          Yes. But hopefully, with time and testing, less than those
          still waiting under the covers in other popular file server
          programs we use.

          Do not use this for your own files yet. It is experimental
          and still under testing.