From mboxrd@z Thu Jan 1 00:00:00 1970 Mime-Version: 1.0 (Apple Message framework v1257) Content-Type: text/plain; charset=utf-8 From: Nemo In-Reply-To: Date: Fri, 18 May 2012 23:13:54 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: <9F03A819-F521-407C-A6BD-13A04A3AC877@lsub.org> References: To: Fans of the OS Plan 9 from Bell Labs <9fans@9fans.net> Subject: [9fans] The creepy WORM. (was: Re: Thinkpad T61 Installation Experience) Topicbox-Message-UUID: 92607bfc-ead7-11e9-9d60-3106f5b1d025 Because I noticed Ken's worm fs was being discussed in this thread, I = thought I might just drop here the man page for a new alternate file server that = we wrote for nix. It's not yet ready for use (I'm using it, but it's still under testing, = and the version in the main nix tree is now out of date, btw; will send the new one = soon). But I thought it might be of interest to 9fans, if only to get comments. CREEPY(4) CREEPY(4) NAME 9pix, fmt, rip, arch - creepy file server and WORM archive SYNOPSIS creepy/9pix [ -DFLAGS ] [ -ra ] [ -A addr ] [ -S srv ] disk creepy/fmt [ -DFLAGS ] [ -wy ] disk creepy/rip [ -DFLAGS ] [ -a ] [ -c n ] [ -A addr ] [ -S srv ] disk creepy/arch [ -DFLAGS ] [ dir ] DESCRIPTION Creepy is a prototype file server for Nix. It maintains a mutable file tree with unix semantics, kept in main memory, served through 9P, see intro(5), and through IX. Creepy/9pix is the main file server program. It serves a file tree through 9P and IX. The tree kept in memory is mutable. But frozen versions of the tree are written to disk, both upon request and also on a periodic basis, to survive power outages and other problems, and to be able to operate on trees that do not fit on main memory. The tree(s) stored on disk are frozen and cannot be changed once written. By default the program listens for 9P in the standard TCP port and posts a connection that can be mounted at /srv/9pix. Flags -A and -S may be used to specify an alter- nate network address and/or srv(3) file to post. Using these flags makes the program not to listen and not to post to srv(3) unless a flag indicates so. Flag -r makes the pro- gram operate in read-only mode, and flag -a starts the pro- gram without requiring authentication to mount the file tree. The disk is organized as a log of blocks. When a new version of the tree must be written to disk, all blocks that changed are given disk addresses and are appended to the log. Once written, they are frozen. If new changes are made to the tree, blocks are melted and forget their previous addresses: each time they are written again, they are assigned new ones. When the disk gets full, all reachable blocks are marked and all other blocks are considered available for growing the log (this is a description of semantics, not of the imple- mentation). Thus, the log is circular but jumps to the next available block each time it grows. If, after the mark pro- cess, the disk is still full, the file system becomes read only but for removing files. Before using a disk it must be formatted using creepy/fmt. This initializes blocks used to store marks for the mark and sweep process and also initializes a super block and an empty root directory. Flag -y forces the format even if the disk seems to contain a fossil (4) or creepy (4) partition. Flag -w is used to format the partition for a WORM (described later) and not for a main file tree. Creepy/rip is the same file server program, but operates in WORM mode. In this case, no mark and sweep for free blocks will ever happen. Blocks are consumed from the disk until it becomes full. The file tree served is always read-only. Operating the WORM to in administrative mode to add more trees or new versions of the archived trees does not require any protocol: it can be done using the standard file inter- face used to operate on any other tree, by mounting and changing it. An alternate mount specifier, wormwr, can be used to mount the tree in read-write mode, to update the archive. Updat- ing the archive is performed by creating new trees with the conventional /treename/year/mmdd paths on the WORM. But note that such paths are not enforced by the program at all. Before updating a tree in the archive, for a new day, a con- trol request described later can be used to link the direc- tory for the previous version of the archive to the new one. After that, changes made to files would in effect copy on write all blocks affected, and refer to old ones when they did not change. Creepy/arch is a program started by creepy/9pix to archive snapshots of the tree into a directory provided by creepy/rip (or by any other file server). The program is not expected to be run by users, and speaks a simple proto- col through its standard input: A block address is written and the block contents are read from it. Usually, the stan- dard input is a pipe to creepy/9pix. The program takes the address of the root directory as kept on disk and then asks the file server program for block contents, to archive them into a mounted directory. Thus, the tree archived is always consistent because only consistent trees can be read from disk (the mutable file tree is kept in memory). The file tree provided by 9pix and rip has the following layout: / /root /cons /stats And, by convention, archives kept in the WORM are expected to have names like: /root/treename/2012/0425 /root/treename/2012/0425.1 /root/treename/2012/0426 Using the main attach specifier yields the tree as just shown. Using an empty specifier, or root, or main/active yields the tree found at /root. In creepy/rip the attach specifier wormwr yields the root of the tree in read-write mode. =E2=88=99 The root directory is never found on disk. It is = a placeholder for the contained files. =E2=88=99 Cons is a synthesized file used as a console for = file system administration, and it is not found on disk. It is still subject to file permissions, and coordinates concurrent access to the console by different users. The owner of the file server process is always allowed to access the console. =E2=88=99 Stats is another synthesized file used to report statistics. =E2=88=99 Root contains the actual file tree. It = corresponds to the file tree as seen at this moment by file system users. It's semantics are similar to those of a UNIX or Plan 9 file system. Users and groups can be added by creating and editing /root/users . See an example shown later. The console accepts the following commands: allow [uid] Lets uid bypass permissions, or show allowed users if none is given. disallow [uid] undoes the effect of the previous command for the given uid or for everyone if none is given. halt Halts the file system and terminates the processes. stats Reports usage statistics. sync Synchronizes the disk by writing a new version of the tree to the log. uname [uid] Reports information about user uid. users Prints the list of users. who Reports the list of network addresses and users cur- rently using the file system. arch name hour [path] Schedules an archival into the WORM with the given tree name and at the given hour. By default, the WORM is expected at /n/rip in administrative mode. An optional path may be given to use a different directory. arch now archives the tree. The previous form must be used before to configure archival. arch none disables archival (this is the default). link src dst is used in the WORM to initialize a new archive by linking the one of the previous day to a new name, before updating the tree. ? Prints the list of known commands. More than those describe here will be shown, mostly for debugging. EXAMPLES Format a partition and a WORM creepy/fmt /dev/sdC0/creepy creepy/fmt -w /dev/sdC0/rip Run the file system, mount it, initialize the user list with one from fossil(4), and populate it creepy/9pix /dev/sdC0/creepy main mount -c /srv/9pix /n/creepy cp /adm/users /n/creepy/root/users echo allow $user >/n/creepy/cons dircp /n/dist /n/creepy/root Run the worm (the previously file server would mount it if it finds its file posted in srv(3) and is asked to archive a tree. creepy/rip /dev/sdC0/rip SOURCE /sys/src/cmd/creepy SEE ALSO fossil(4), venti(8) BUGS Yes. But hopefully, with time and testing, less than those still waiting under the covers in other popular file server programs we use. Do not use this for your own files yet. It is experimental and still under testing.