From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: From: "Russ Cox" To: 9fans@cse.psu.edu Subject: Re: [9fans] replica (was: ipv6) MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Date: Tue, 28 May 2002 11:32:02 -0400 Topicbox-Message-UUID: 9e554a62-eaca-11e9-9e20-41e7f4b1d025 > Your message arrived here while I had replica/pull running. > I don't know whether this already came up in the replica > discussion so far... (or can be R in TFM): > if you push while I'm pulling, do I still get > a 'consistent' set of files? Yes, unless you get extraordinarily unlucky. Pushing consists of two operations: copying the file to sources and updating /n/sources/plan9/dist/replica/plan9.log to reflect the change. Replica/pull consults the log, sees that a new file is available, and copies the new file. First, the log: The log is an append-only file that you copy to your local machine via tail. If more data arrives while you're copying, you'll just get the new data. No harm done. Updatedb uses buffered I/O to write the log, so it is possible that on a really big update (more than 8kb worth of log), you'd get a log ending in a half-entry. Replica/applylog (the guts of replica/pull) would then complain, and if you ran replica/pull again, the rest of the log would come down and you'd be fine. When the log gets far too large, I compress it, removing the old one and writing an equivalent smaller one. If this happens at exactly the right moment, pull (a lowly shell script) could get confused, trying to grab the tail of a file that has changed underfoot. The change always makes the log smaller, though, so if you've updated at all since installing, tail and thus pull will fail. Running replica/pull again will then work. It is possible (though extraordinarily unlikely) that if you install Plan 9 and then do an update just as I replace the log with a compressed version, tail will succeed but leave you with your original log and then some arbitrary suffix of the new log. At this point, the log is probably corrupt, so replica/pull will complain. If the log isn't corrupt but you downloaded less than a kilobyte for the suffix, then replica/pull will download the whole log afresh the next time. If, by chance, you get a well-formed log AND the new suffix you downloaded is more than a kilobyte, then you'll never notice the screwup. This is exceedingly unlikely even if I were compressing the log often. I intend to compress the log only when it grows to about 1.6MB, twice its normal size. If we sustain our current rate of change, I'll have to compress the log every two years. Oh, and you had to install when the log was at its minimal size. Installing with a recent packaging would have gotten you a bigger log and the tail would have failed. I'm just not worried. Second, copying: If a file changes before being mentioned in the log (but is also mentioned earlier in the log) you might copy the new file expecting to get an older version of it. That's okay, replica/pull will notice the unexpectedly new time stamp and record that it has the new file. When (later) it sees the entry in the log for the new version, it won't bother copying the file a second time. When replica/pull downloads a file, it spools it to /tmp and then once it has the whole file copies it from /tmp to its destination. If the copy into /tmp fails (say, because of a network hangup), then an error is printed and the /tmp copy is tossed. Pull stats the file before and after the copy into /tmp. It repeats the copy until the qid stops changing. In summary: Downloading the new log while it changes won't leave you inconsistent unless you install Plan 9 right after I compress the log (an annual or biannual event), and then wait to run replica/pull until the exact instant I'm about to compress the log again. Downloading files while they change won't leave you in an inconsistent state. Ever. Russ