[9fans] Plan 9 wiping itself?

[9fans] Plan 9 wiping itself?

[Pietro Gagliardi]

term% /usr/glenda/bin/rc/pull
d just about every file in the sys/src/lib* directories

What does this mean? am I going to lose my work? Is this happening to  
anyone else?

Re: [9fans] Plan 9 wiping itself?

[Antonin Vecera]

Yes. But in the same time many files and dirs were copied back.

Antonin

On Nov 24, 2007 12:29 AM, Pietro Gagliardi <pietro10@mac.com> wrote:
> term% /usr/glenda/bin/rc/pull
> d just about every file in the sys/src/lib* directories
>
> What does this mean? am I going to lose my work? Is this happening to
> anyone else?
>

Re: [9fans] Plan 9 wiping itself?

[Pietro Gagliardi]

I realized that a few minutes
.I after
sending myself the email. Oh well. What was the point of that - did  
someone rewrite the entire library? Why didn't there appear just cs  
and as?

On Nov 24, 2007, at 12:44 PM, Antonin Vecera wrote:

> Yes. But in the same time many files and dirs were copied back.
>
> Antonin
>
> On Nov 24, 2007 12:29 AM, Pietro Gagliardi <pietro10@mac.com> wrote:
>> term% /usr/glenda/bin/rc/pull
>> d just about every file in the sys/src/lib* directories
>>
>> What does this mean? am I going to lose my work? Is this happening to
>> anyone else?
>>

Re: [9fans] Plan 9 wiping itself?

[Uriel]

Somebody was asking why replica sucks?

uriel

On Nov 24, 2007 6:46 PM, Pietro Gagliardi <pietro10@mac.com> wrote:
> I realized that a few minutes
> .I after
> sending myself the email. Oh well. What was the point of that - did
> someone rewrite the entire library? Why didn't there appear just cs
> and as?
>
>
> On Nov 24, 2007, at 12:44 PM, Antonin Vecera wrote:
>
> > Yes. But in the same time many files and dirs were copied back.
> >
> > Antonin
> >
> > On Nov 24, 2007 12:29 AM, Pietro Gagliardi <pietro10@mac.com> wrote:
> >> term% /usr/glenda/bin/rc/pull
> >> d just about every file in the sys/src/lib* directories
> >>
> >> What does this mean? am I going to lose my work? Is this happening to
> >> anyone else?
> >>
>
>

Re: [9fans] Plan 9 wiping itself?

[erik quanstrom]

On Sat Nov 24 12:47:08 EST 2007, pietro10@mac.com wrote:
> I realized that a few minutes
> .I after
> sending myself the email. Oh well. What was the point of that - did  
> someone rewrite the entire library? Why didn't there appear just cs  
> and as?
> 

Re: [9fans] Plan 9 wiping itself?

[Russ Cox]

On Nov 24, 2007 1:43 PM, Uriel <uriel99@gmail.com> wrote:
> Somebody was asking why replica sucks?

It's a bug in a shell script, which caused sources to look like
some files went away and then got recreated.   This happens
occasionally if the replica update programs that run at Bell Labs
get interrupted or the sources disk fills.  It doesn't cause data loss,
scary though it may look.

In this case, replica actually recreates all the files it deletes,
and it refuses to delete any file that had been locally modified,
so no information would have been lost.  (I tested this earlier
today on my own system.)

I assume that your vague references to replica destroying people's
systems is in fact referring to earlier times that this kind of sequence
has happened.  Again, replica won't touch any file that it didn't create
and it also won't delete any file that has been changed since replica
put it there.  I really don't expect very much of you, Uriel, but I didn't
expect FUDmongering.

For what it's worth, I made some changes to pull today to
eliminate spurious warnings about "locally modified" files in the
case where the local file and the sources copy are identical.
This avoids conflicts in the common case where one does

   9fs sources
   cp /n/sources/plan9/some/file /some/file

and then later run "pull" to update your system.  It also avoids
conflicts in the common case where you make a local change,
submit a patch, the patch gets installed, and then pull wants to
propagate your change back down from sources to your local machine.

I also changed pull to ignore deletions if the supposedly-deleted
file has been recreated.  This will make pull skip over a sequence of
"delete then recreate" entries in the log.

Russ

Re: [9fans] Plan 9 wiping itself?

[sqweek]

On Nov 28, 2007 1:21 PM, Russ Cox <rsc@swtch.com> wrote:
> In this case, replica actually recreates all the files it deletes,
> and it refuses to delete any file that had been locally modified,
> so no information would have been lost.  (I tested this earlier
> today on my own system.)

 Unless of course, replica gets interrupted halfway through on the
client side (not idle speculation, happened to someone on IRC).

> Again, replica won't touch any file that it didn't create
> and it also won't delete any file that has been changed since replica
> put it there.

 Replica not touching any file that it didn't create, while a nice
quality, is not much consolation[1] when it has created the majority
of your system.

[1]ok, it at least means you're not losing any irreplacable files...
unless perhaps you're running on a file server and replica hosed
/bin/^(mount fossilcons venti/sync etc) [2]. in any case, an OS
doesn't earn many brownie points by erasing itself
[2] this is idle speculation

<snip replica changes>
> This will make pull skip over a sequence of
> "delete then recreate" entries in the log.

 The changes sound great, and eliminate my above doomsday scenario...
unless something goes really wrong with the replica log. Is it
replica/updatededb that is the culprit or just the way it is run in
replica/scan? It strikes me that if it is forced to abort for some
reason it should avoid updating the log at all.
 By they way, any idea why the replica/pull -n output is so confusing
in this situation? It lists a lot of "a"s for files which already
exist (eg 386/9pc 386/auth/factotum sys/src/9/boot/boot.h) (in
alphabetic order?), then a lot of "d"s (in reverse alphabetical
order?), and then I have some legitimate c/a/ds (sys/src/games/mp3enc)
on account of I haven't pulled in awhile. But the initial
additions/deletions are disjoint sets as far as I can tell - why don't
I see deletions followed by additions for the same files?
-sqweek

Re: [9fans] Plan 9 wiping itself?

[Russ Cox]

>  The changes sound great, and eliminate my above doomsday scenario...
> unless something goes really wrong with the replica log. Is it
> replica/updatededb that is the culprit or just the way it is run in
> replica/scan? It strikes me that if it is forced to abort for some
> reason it should avoid updating the log at all.

Actually the problem was that updatedb *wasn't* aborting.
The network connection got lost halfway, so a lot of files "disappeared"
so updatedb thought they had been deleted.  And then it found
them the next time it ran.  I just submitted a fix so that it will
notice network errors and stop before inferring file deletions.
This won't happen again (and if it does, pull is smarter anyway).

>  By they way, any idea why the replica/pull -n output is so confusing
> in this situation? It lists a lot of "a"s for files which already
> exist (eg 386/9pc 386/auth/factotum sys/src/9/boot/boot.h) (in
> alphabetic order?), then a lot of "d"s (in reverse alphabetical
> order?), and then I have some legitimate c/a/ds (sys/src/games/mp3enc)
> on account of I haven't pulled in awhile. But the initial
> additions/deletions are disjoint sets as far as I can tell - why don't
> I see deletions followed by additions for the same files?

The log occasionally gets compressed (really, canonicalized)
to remove redundant events.  When that happens, the canonical
replacement form is an "a" event for every file on the system 
followed by a "d" for every file that existed in the past and got deleted.
Putting the "d" events in reverse alphabetical order deletes files
before their parent directories.  The two sets are disjoint because
redundant event sequences have been collapsed into a single "a" or "d".

Geoff compressed the log yesterday so that the big
"delete /sys/src and then recreate it" sequence would go away.
And then after compressing the log a few more entries got added.

There's not a lot of difference between "a" and "c".  If the log says "a"
but the file already exists (and was created by replica) then it's
treated as a "c".  The "a" events you saw were really the "c" events
that had happened before the log got compressed and that you
hadn't yet pulled.  Probably "c" should go away.

Russ

Re: [9fans] Plan 9 wiping itself?

[Douglas A. Gwyn]

Instead of delete and later replace, how about create unique-named temp new file then immediately rename to target name?
If rename is atomic then nothing could ever get lost.

Re: [9fans] Plan 9 wiping itself?

[sqweek]

On Nov 29, 2007 1:29 AM, Douglas A. Gwyn <DAGwyn@null.net> wrote:
> Instead of delete and later replace, how about create unique-named temp new file then immediately rename to target name?
> If rename is atomic then nothing could ever get lost.

 I suspect the submitted patch (ie, don't delete the file in the first
place) is similar enough.
 Thanks a lot Russ, for the fixes and explanation.
-sqweek