9fans - fans of the OS Plan 9 from Bell Labs
 help / color / mirror / Atom feed
* [9fans] silly replica question (repeated m msgs won't go away)
@ 2003-12-02 15:49 Axel Belinfante
  2003-12-02 16:10 ` mirtchov
  0 siblings, 1 reply; 13+ messages in thread
From: Axel Belinfante @ 2003-12-02 15:49 UTC (permalink / raw)
  To: 9fans

Today I updated my fake-worm fs from sources (first time since summer).

every time I replica/pull, I get lots of msgs like:

m sys/lib/antiword/8859-1.txt 644 sys sys 1069793886
m sys/lib/antiword/8859-10.txt 644 sys sys 1069793886
m sys/lib/antiword/8859-13.txt 644 sys sys 1069793886
m sys/lib/antiword/8859-14.txt 644 sys sys 1069793886
m sys/lib/antiword/8859-15.txt 644 sys sys 1069793886

and whenever I now run replica/pull again, I get the same...

adding -s flag to replica/pull did not help
(fair enough, it serves a different purpose)

What am I doing wrong, or overlooking?
I _think_ have the fake-worm fs permissions open enough, or???

Thanks,
Axel.




^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2003-12-02 15:49 [9fans] silly replica question (repeated m msgs won't go away) Axel Belinfante
@ 2003-12-02 16:10 ` mirtchov
  2004-02-10  3:46   ` Adrian Tritschler
  0 siblings, 1 reply; 13+ messages in thread
From: mirtchov @ 2003-12-02 16:10 UTC (permalink / raw)
  To: 9fans

> Today I updated my fake-worm fs from sources (first time since summer).
>
> every time I replica/pull, I get lots of msgs like:
>
> m sys/lib/antiword/8859-1.txt 644 sys sys 1069793886
> m sys/lib/antiword/8859-10.txt 644 sys sys 1069793886
> m sys/lib/antiword/8859-13.txt 644 sys sys 1069793886
> m sys/lib/antiword/8859-14.txt 644 sys sys 1069793886
> m sys/lib/antiword/8859-15.txt 644 sys sys 1069793886
>
> and whenever I now run replica/pull again, I get the same...
>
> adding -s flag to replica/pull did not help
> (fair enough, it serves a different purpose)
>
> What am I doing wrong, or overlooking?
> I _think_ have the fake-worm fs permissions open enough, or???
>
> Thanks,
> Axel.

there were broken /sys/lib/antiword and /sys/src/cmd/aux/antiword with
bad permissions on sources.  looks like you're stuck with a bad copy.
try chmod-ing your own version to something world-readable.

or deleting them altogether and trying the -s flag to pull..

andrey



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2003-12-02 16:10 ` mirtchov
@ 2004-02-10  3:46   ` Adrian Tritschler
  2004-02-10  5:07     ` Geoff Collyer
  0 siblings, 1 reply; 13+ messages in thread
From: Adrian Tritschler @ 2004-02-10  3:46 UTC (permalink / raw)
  To: 9fans

mirtchov@cpsc.ucalgary.ca wrote (sometime last December):

>>Today I updated my fake-worm fs from sources (first time since summer).
>>
>>every time I replica/pull, I get lots of msgs like:
>>
>>m sys/lib/antiword/8859-1.txt 644 sys sys 1069793886
>>m sys/lib/antiword/8859-10.txt 644 sys sys 1069793886
>>m sys/lib/antiword/8859-13.txt 644 sys sys 1069793886
>>m sys/lib/antiword/8859-14.txt 644 sys sys 1069793886
>>m sys/lib/antiword/8859-15.txt 644 sys sys 1069793886
>>
>>and whenever I now run replica/pull again, I get the same...
>>
>>adding -s flag to replica/pull did not help
>>(fair enough, it serves a different purpose)
>>
>>What am I doing wrong, or overlooking?
>>I _think_ have the fake-worm fs permissions open enough, or???
>>
>>Thanks,
>>Axel.

> there were broken /sys/lib/antiword and /sys/src/cmd/aux/antiword with
> bad permissions on sources.  looks like you're stuck with a bad copy.
> try chmod-ing your own version to something world-readable.
> 
> or deleting them altogether and trying the -s flag to pull..
> 
> andrey

I'm suffering the same problem, every pull that I do from sources is 
producing two lines for each file in sys/lib/antiword and 
sys/src/cmd/aux/antiword.  I've tried removing all the files and redoing 
the pull as "pull -s sys/src/cmd/aux", but that then failed because 
/n/kfs/sys/src/cmd/aux/antiword didn't exist.  I created the directory, 
then the next time I did a pull I got
  :
a sys/src/cmd/aux/antiword/xml.c 644 sys sys 1069795877
  :
m sys/src/cmd/aux/antiword/xml.c 664 sys sys 1069795877

Then on every subsequent pull I get the two:
  :
m sys/src/cmd/aux/antiword/xml.c 644 sys sys 1069795877
  :
m sys/src/cmd/aux/antiword/xml.c 644 sys sys 1069795877

Is there any way to convince replica that I really do have all the files 
on my machine, and for it to stop flipping the permissions back and 
forth between 644 and 664?

RTFM accepted, but can some kind sole point me to the right part of the FM?

	Adrian

---------------------------------------------------------------
Adrian Tritschler                          mailto:ajft@ajft.org
Latitude 38°S, Longitude 145°E, Altitude 50m,      Shoe size 44
---------------------------------------------------------------


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10  3:46   ` Adrian Tritschler
@ 2004-02-10  5:07     ` Geoff Collyer
  2004-02-10  5:12       ` Lucio De Re
  2004-02-10 12:22       ` Charles Forsyth
  0 siblings, 2 replies; 13+ messages in thread
From: Geoff Collyer @ 2004-02-10  5:07 UTC (permalink / raw)
  To: 9fans

> RTFM accepted, but can some kind sole point me to the right part of the FM?

I believe that would be the BUGS section.  I've been suffering with
this too and have resorted to grepping out the antiword messages.
replica can flip its bits to its heart's content.



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10  5:07     ` Geoff Collyer
@ 2004-02-10  5:12       ` Lucio De Re
  2004-02-10 12:22       ` Charles Forsyth
  1 sibling, 0 replies; 13+ messages in thread
From: Lucio De Re @ 2004-02-10  5:12 UTC (permalink / raw)
  To: 9fans

On Mon, Feb 09, 2004 at 09:07:38PM -0800, Geoff Collyer wrote:
>
> > RTFM accepted, but can some kind sole point me to the right part of the FM?
>
> I believe that would be the BUGS section.  I've been suffering with
> this too and have resorted to grepping out the antiword messages.
> replica can flip its bits to its heart's content.

I've edited various instances of the .log and .db files (master or
slave) and I have no idea which one did the trick, but the messages
have ceased.

++L


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10  5:07     ` Geoff Collyer
  2004-02-10  5:12       ` Lucio De Re
@ 2004-02-10 12:22       ` Charles Forsyth
  2004-02-10 12:38         ` Lucio De Re
  1 sibling, 1 reply; 13+ messages in thread
From: Charles Forsyth @ 2004-02-10 12:22 UTC (permalink / raw)
  To: 9fans

i think the replica scheme is actually quite pretty,
but there's a flaw that happens to afflict 'm' particularly.

the easiest way to fix this without code change is to use
-c and -s as desired (carefully!)to resolve every other inconsistency,
so that plan9.time will finally be updated, and it won't process the 'm' entries again.
better still, your subsequent updates will be quicker too because it won't replay so much
of the log!  perhaps just do one replica/pull -v ... to get a list of differences to check later,
and fetch anything new, then do replica/pull -vc ... to retain your current client state.
a subsequent replica/pull -v ... shouldn't show the dreaded 'm' entries.

it might be helpful to look at one case to see what went wrong.

the file on sources is
--rw-rw-r-- M 119 rsc sys 10583 Nov 25 20:58 /n/sources/plan9/sys/lib/antiword/8859-15.txt

the m messages are repeated because the log contains several batches of records like the following:

1069794016 8 a sys/lib/antiword/8859-15.txt - 640 sys sys 1069793886 10583
1069799416 4 m sys/lib/antiword/8859-15.txt - 644 sys sys 1069793886 10583
1069801217 4 m sys/lib/antiword/8859-15.txt - 664 sys sys 1069793886 10583

(with other records interspersed).  note that the mtime is the same in each record
(correctly, as it happens).

the 'a' record has the test:
			if(dbd.mtime >= rd.mtime)	/* already created this file; ignore */
				break;
which prevents it from going any further.

the 'm' record has the test:
			if(!(dbd.mode&DMDIR) && dbd.mtime > rd.mtime)		/* have newer version; ignore */
				break;
			if((dbd.mode&DMDIR) && dbd.mtime > now)
				break;
where `now' is the date of the log entry (its first field), not the current time.
the test for `newer version' is > not >= so each 'm' is considered further.

one of the tests is whether the local file mode matches the remote mode,
where the `remote mode' is the mode in the log entry not the state of the current remote file.
they don't match, so the local file's mode is changed by the first 'm', then changed again
by the second 'm', and since it has done something you get a comment each time,
and an update in the database.

the file's mtime is the same in each log entry, because chmod correctly affects the parent directory,
not the file itself.  the -c or -s options have no effect because although they correctly update
the metadata (.db) to set the appropriate state, all the 'm' entries have the same
mtime and thus the newer-version test will continue to be false and they'll
rerun each time.  the trouble is that the .db has got only the mtime to resolve
`newer action', not the log sequence number for that action.

is there a simple change to the code? it can't simply set the mtime in .db
to the log entry time because it needs the correct last-modified time
for the file contents to be in .db to test later 'a'/'c'/'d' requests.  it isn't correct
to use >= in the mtime comparison because that would prevent 'm' from having much effect generally.
it would help to know which log entry it has seen.
there might be another way short of storing the log sequence in .db,
but i couldn't see it in the time i was prepared to spend on it.
i could see several partial/incorrect changes (eg, stat the actual remote file).

i looked at replica's approach for possible use in Inferno, because it
was compact and seemed simpler (in code size) compared to some
alternatives.  as it happens, the replica problem has its subtle
points (even for a problem simpler than tra's).  i quickly resorted to
a case analysis on my own, and wrote Limbo code based on that.
i noticed at least one other difference in
my analysis and applylog's; i must check my notes when i remember
where i filed them.  i ended up deciding to process the log files differently,
more precisely as with a (class of) log-structured file system (i was
doing a lot with them at the time).  there is a particular invariant
for the contents of the client's .log.  i could possibly have used a
modified .db structure (including log sequence numbers), but in my
scheme that's just an optimisation (and there was another way to do that).
of course, something else might be wrong ...  we'll see.



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10 12:22       ` Charles Forsyth
@ 2004-02-10 12:38         ` Lucio De Re
  2004-02-10 15:00           ` David Presotto
  0 siblings, 1 reply; 13+ messages in thread
From: Lucio De Re @ 2004-02-10 12:38 UTC (permalink / raw)
  To: 9fans

On Tue, Feb 10, 2004 at 12:22:28PM +0000, Charles Forsyth wrote:
>
> i think the replica scheme is actually quite pretty,
> but there's a flaw that happens to afflict 'm' particularly.
>
Replica is extremely useful, specially in the role it serves to
most of this community.  Its minor flaws are sufficiently trivial
to be ignored and workarounds applied as the occasion demands.

But replication deserves being done "right" rather than heuristically
and Plan 9 seems to have all the ingredients to provide the final
word in replication.  Further, as I mentioned in another thread,
replication enhances the "distributability" and Plan 9 can only
benefit from such an ugly word.

If I'm right that the dump (in its ISO-9660 guise) and replica can
converge and provide the ideal tool, then I'm willing to make myself
unpopular and challenge the present replica wisdom.  With all due
respect, of course, to those who first trod this path.

If I'm being stupid, then I won't mind acknowledging it once my nose
has been rubbed in it.

++L


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10 12:38         ` Lucio De Re
@ 2004-02-10 15:00           ` David Presotto
  2004-02-10 15:08             ` Fco.J.Ballesteros
                               ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: David Presotto @ 2004-02-10 15:00 UTC (permalink / raw)
  To: 9fans

On Tue Feb 10 07:39:37 EST 2004, lucio@proxima.alt.za wrote:
> But replication deserves being done "right" rather than heuristically

Describe right a little more.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10 15:00           ` David Presotto
@ 2004-02-10 15:08             ` Fco.J.Ballesteros
  2004-02-10 15:28             ` Lucio De Re
  2004-02-11  1:52             ` boyd, rounin
  2 siblings, 0 replies; 13+ messages in thread
From: Fco.J.Ballesteros @ 2004-02-10 15:08 UTC (permalink / raw)
  To: 9fans

>> But replication deserves being done "right" rather than heuristically
>
> Describe right a little more.

tra


PS: Couldn't resist :-)



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10 15:00           ` David Presotto
  2004-02-10 15:08             ` Fco.J.Ballesteros
@ 2004-02-10 15:28             ` Lucio De Re
  2004-02-11  1:52             ` boyd, rounin
  2 siblings, 0 replies; 13+ messages in thread
From: Lucio De Re @ 2004-02-10 15:28 UTC (permalink / raw)
  To: 9fans

On Tue, Feb 10, 2004 at 10:00:21AM -0500, David Presotto wrote:
>
> On Tue Feb 10 07:39:37 EST 2004, lucio@proxima.alt.za wrote:
> > But replication deserves being done "right" rather than heuristically
>
> Describe right a little more.

I do really mean that in the grey area between exact and heuristically
adequate, I'd like to be closer to exact.  My opinion is that
replica has weaknesses that diminish the "faith" factor whereas
"dump" certainly doesn't or doesn't seem to.

What might be a different approach would be to provide a measurement
of the uncertainty in replica, but somehow I can't help believing
that there is a better paradigm implicit in dump9660 that can be
exploited instead.

Does the above help?  "Right" as in as accurate as produced by the
dump9660 procedure just ahead of generating the new release.  For
all I know this is already the case and it's the baseline that
triggers replica errors, but that needs proof.

Sorry if I seem to be pontificating, as I mentioned it's a half
formed idea and I'm looking to give it a little more solidity.  If
it's preferable to continue this in private mail, please don't
hesitate to point it out.

++L


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10 15:00           ` David Presotto
  2004-02-10 15:08             ` Fco.J.Ballesteros
  2004-02-10 15:28             ` Lucio De Re
@ 2004-02-11  1:52             ` boyd, rounin
  2 siblings, 0 replies; 13+ messages in thread
From: boyd, rounin @ 2004-02-11  1:52 UTC (permalink / raw)
  To: 9fans

> Describe right a little more.

wobber/prusker siphon.



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
  2004-02-10 16:10 Charles Forsyth
@ 2004-02-10 17:24 ` Lucio De Re
  0 siblings, 0 replies; 13+ messages in thread
From: Lucio De Re @ 2004-02-10 17:24 UTC (permalink / raw)
  To: 9fans

On Tue, Feb 10, 2004 at 04:10:20PM +0000, Charles Forsyth wrote:
>
> i don't think i'd describe what replica attempts as `heuristic'.
> there's a little graph of possibilities and it either chooses the safe
> path or requests guidance.
>
I really hope I don't sound confrontational, there's much I don't
know and can only guess at, but there's definitely more than a
trivial amount of slipperiness about replica that makes it unsuitable
for the type of replication that Microsoft used (or used to use)
to propagate changes to the users database between the PDC and BDCs
(if you don't know the acronym, you're missing nothing) or what
Lotus Notes uses (this is NetNews in disguise or is it on steroids?)
or, for that matter, the OpenLDAP replicator.

> an SHA1 hash would be helpful; by using mtime+length it can moan about
> insignificant changes but then again, with some things it's possible
> the mtime matters, and perhaps it should moan.
>
I think there's a philosophy behind replica that suggests humans
are involved therefore there's scope for some fuzzy logic.  I'm
not even arguing the merits thereof, it's a perfectly good approach.
What I prefer, however, is the solidity of the dump and a tool to
extract the updates list in a format that makes it possible to
revert them and/or apply them elsewhere.

I'd better be careful, however, I'm beginning to recognise signs
that my attraction for CVS is coming through.

> having mentioned SHA1: there might be an interesting variant that uses
> venti scores.

I won't try to follow your thinking there, it's way beyond my
processing powers :-)

++L


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [9fans] silly replica question (repeated m msgs won't go away)
@ 2004-02-10 16:10 Charles Forsyth
  2004-02-10 17:24 ` Lucio De Re
  0 siblings, 1 reply; 13+ messages in thread
From: Charles Forsyth @ 2004-02-10 16:10 UTC (permalink / raw)
  To: 9fans

i don't think i'd describe what replica attempts as `heuristic'.
there's a little graph of possibilities and it either chooses the safe
path or requests guidance.

an SHA1 hash would be helpful; by using mtime+length it can moan about
insignificant changes but then again, with some things it's possible
the mtime matters, and perhaps it should moan.

having mentioned SHA1: there might be an interesting variant that uses
venti scores.



^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2004-02-11  1:52 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2003-12-02 15:49 [9fans] silly replica question (repeated m msgs won't go away) Axel Belinfante
2003-12-02 16:10 ` mirtchov
2004-02-10  3:46   ` Adrian Tritschler
2004-02-10  5:07     ` Geoff Collyer
2004-02-10  5:12       ` Lucio De Re
2004-02-10 12:22       ` Charles Forsyth
2004-02-10 12:38         ` Lucio De Re
2004-02-10 15:00           ` David Presotto
2004-02-10 15:08             ` Fco.J.Ballesteros
2004-02-10 15:28             ` Lucio De Re
2004-02-11  1:52             ` boyd, rounin
2004-02-10 16:10 Charles Forsyth
2004-02-10 17:24 ` Lucio De Re

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).