9front - general discussion about 9front
 help / color / mirror / Atom feed
* [9front] git lca bug
@ 2022-03-05 19:32 Michael Forney
  2022-03-06  3:44 ` ori
  0 siblings, 1 reply; 4+ messages in thread
From: Michael Forney @ 2022-03-05 19:32 UTC (permalink / raw)
  To: 9front

I recently encountered a bug where git/query returns the wrong
result when calculating LCA(A, B) in the following graph:

A
|  B
|  |
|  C
|  |
|  D
|  |
|  E
|  |
|  F
|  |
|  G
|  |
|  H
| /
|/
I
|
J
|
K
|
L
|\
| \
M  N
| / 
|/
O
|
P
|
...


It returns M rather than I.

The LCA algorithm was rewritten not too long ago in
http://git.9front.org/plan9front/plan9front/c7dcc82b0be805717efbe77c98eaadf3ee1e31af/commit.html

However, I don't understand that commit message. The definition of
LCA(b, g) that I've read is "the lowest node that has both b and g
as descendents". In the graph

    <--a--b--c--d--e--f--g
        \               /
          +-----h-------

the lowest node that fits this definition is b. a is not the LCA,
since b is a descendent of a, and therefore lower.

I'm not sure what is meant by strict LCA, but maybe there is some
other definition of LCA that finds the ancestor whose distances
between the two nodes is minimized in some way? What metric was
used here? Perhaps sum of distances?

It seems to me that what we actually want to find is a common
ancestor such that there is no other common ancestor that descends
from it. I don't think the distances between the LCA arguments and
the ancestor are relevant here, except maybe to break ties when
there are multiple LCAs.

Here's a debug log with commit hashes replaced with the letter in
the problematic graph:

term% git/query -dd A B
init: keep A
init: drop B
finding twixt commits
found best (dist 7 < 1073741824): I
found best (dist 5 < 7): M
found ancestor
M
term%

term% git/query -dd A B
init: keep A
init: drop B
finding twixt commits
	enqueue: keep I
	enqueue: drop C
	enqueue: keep J
	enqueue: keep K
	enqueue: keep L
	enqueue: keep M
	enqueue: keep N
	enqueue: keep O
	enqueue: keep P
	enqueue: keep Q
	enqueue: keep R
	enqueue: keep S
	enqueue: keep T
	enqueue: keep U
	enqueue: keep V
	enqueue: keep W
	enqueue: keep X
	enqueue: keep Y
	enqueue: keep Z
	enqueue: keep AA
	enqueue: keep AB
	enqueue: keep AC
	enqueue: keep AD
	enqueue: drop D
	enqueue: drop E
	enqueue: drop F
	enqueue: drop G
	enqueue: drop H
	enqueue: drop I
found best (dist 7 < 1073741824): I
repaint: blank => drop AD
repaint: blank => drop M
found best (dist 5 < 7): M
found ancestor
M
term%

Here's what I think is going wrong:
- We paint commits reachable from A as Keep, and eventually we get
  to a commit with timestamp earlier than B. The remaining ancestors
  (M and AD) are left on the queue.
- We start painting commits reachable from B as Drop and eventually
  find I, marking that as the current best with distance 7 (from B).
- We repaint the commits reachable from I as Drop, until we get to
  the ancestors not yet marked (M and AD).
- We continue with what was left on the queue, M and AD. M is painted
  Drop, and was queued as Keep, so this is a new common ancestor.
- M's distance (from A) is 5, which is lower than 7, so we use that
  in preference to I (even though I only had distance 1 from A).

Any idea on how best to fix this?

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [9front] git lca bug
  2022-03-05 19:32 [9front] git lca bug Michael Forney
@ 2022-03-06  3:44 ` ori
  2022-03-06  5:21   ` Michael Forney
  0 siblings, 1 reply; 4+ messages in thread
From: ori @ 2022-03-06  3:44 UTC (permalink / raw)
  To: 9front

Quoth Michael Forney <mforney@mforney.org>:
> 
> However, I don't understand that commit message. The definition of
> LCA(b, g) that I've read is "the lowest node that has both b and g
> as descendents". In the graph

How do you define 'lowest' in this case?  especially
without walking the whole graph from the initial
commit, which is very slow in large repos?

It was initially being done by 'smallest number of
steps from the starting points', but that's not
correct, because in the graph above, the shortest sum
of steps goes via 'h' to 'a'.

> It returns M rather than I.

That's definitely not intended.

> Any idea on how best to fix this?

Not yet. Tweaking repainting to sum the weight when
colors meet seems like it may fix it, but I haven't
thought it through yet.


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [9front] git lca bug
  2022-03-06  3:44 ` ori
@ 2022-03-06  5:21   ` Michael Forney
  2022-03-06 18:33     ` ori
  0 siblings, 1 reply; 4+ messages in thread
From: Michael Forney @ 2022-03-06  5:21 UTC (permalink / raw)
  To: 9front

ori@eigenstate.org wrote:
> How do you define 'lowest' in this case?  especially
> without walking the whole graph from the initial
> commit, which is very slow in large repos?

Yeah, it's a bit tricky, but I think it's fair to say that if A is
reachable through parent pointers from B, then B is lower than A
(but the converse is not necessarily true).

Interestingly, I found that when I do `B A @` instead of `A B @`,
it *does* end up walking the whole graph to the initial commit.
What happens here is that the Drop painting finds I first, and then
when we get to I through Keep painting. I is recorded as the best,
but then the loop continues (`if(... || ncolor == Drop) continue;`).
Then we keep painting the rest of the graph as Drop, finally returning
I when we run out of nodes.

> Not yet. Tweaking repainting to sum the weight when
> colors meet seems like it may fix it, but I haven't
> thought it through yet.

I also haven't thought this through fully, but here are some thoughts
I had:
- I don't think any nodes traversed by repaint() should be considered
  as candidates for the LCA. repaint() traverses starting at a
  common ancestor, so any nodes we traverse will be higher
  on the graph.
- Instead of repaint() painting nodes from Keep to Drop, maybe there
  should be a third color (Skip?). It could take a starting point
  of either color, repainting everything it finds as Skip, stopping
  when it repaints Blank nodes. If this happens to find our previous
  best common ancestor, then we use the repaint starting point as
  the new best.
- Then, when paint() pops a node painted Skip, it just skips it and
  continues in the loop, since the node was an ancestor of a common
  ancestor.
- paint() should be symmetric in its arguments. It can continue
  if e.color == ncolor, paint it if ncolor == Blank, and repaint
  to Skip otherwise, potentially choosing a new common ancestor.

I'll try those changes in a bit to see if that approach is promising.
I *think* it should at least guarantee that the node it returns is
not reachable from any other common ancestor of A and B.

There are still a few edge cases like the following that I'm not
sure how it would handle, but I don't know how often those come up
in practice.

A   B
|\ /|
| \ |
|/ \|
C   D
 \  |
  E |
   \|
    F
    |
   ...

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [9front] git lca bug
  2022-03-06  5:21   ` Michael Forney
@ 2022-03-06 18:33     ` ori
  0 siblings, 0 replies; 4+ messages in thread
From: ori @ 2022-03-06 18:33 UTC (permalink / raw)
  To: 9front

Quoth Michael Forney <mforney@mforney.org>:
> ori@eigenstate.org wrote:
> > How do you define 'lowest' in this case?  especially
> > without walking the whole graph from the initial
> > commit, which is very slow in large repos?
> 
> Yeah, it's a bit tricky, but I think it's fair to say that if A is
> reachable through parent pointers from B, then B is lower than A
> (but the converse is not necessarily true).
> 
> Interestingly, I found that when I do `B A @` instead of `A B @`,
> it *does* end up walking the whole graph to the initial commit.
>
> What happens here is that the Drop painting finds I first, and then
> when we get to I through Keep painting. I is recorded as the best,
> but then the loop continues (`if(... || ncolor == Drop) continue;`).
> Then we keep painting the rest of the graph as Drop, finally returning
> I when we run out of nodes.
> 
> > Not yet. Tweaking repainting to sum the weight when
> > colors meet seems like it may fix it, but I haven't
> > thought it through yet.
> 
> I also haven't thought this through fully, but here are some thoughts
> I had:
> - I don't think any nodes traversed by repaint() should be considered
>   as candidates for the LCA. repaint() traverses starting at a
>   common ancestor, so any nodes we traverse will be higher
>   on the graph.

yes, though 

> - Instead of repaint() painting nodes from Keep to Drop, maybe there
>   should be a third color (Skip?). It could take a starting point
>   of either color, repainting everything it finds as Skip, stopping
>   when it repaints Blank nodes. If this happens to find our previous
>   best common ancestor, then we use the repaint starting point as
>   the new best.

We may need to resplit the code, if we do that-- it's
also used for pushing and pulling, where the goal is
somewhat different; it wants to find all nodes that
are in one graph but not the other, so we can put
them into a pack file.

With the current code: In theory, we should always be
returning the node on the boundary between Keep and
Drop with the shortest number of steps from both parents.

Regardless of keep/drop, we'll need to explore more or
less the full boundary, and ideally not much more.

I think that this current approach should be more or
less correct, though possibly not 100% optimal -- there's
just a bug in the stop or selection condition.


> - Then, when paint() pops a node painted Skip, it just skips it and
>   continues in the loop, since the node was an ancestor of a common
>   ancestor.
> - paint() should be symmetric in its arguments. It can continue
>   if e.color == ncolor, paint it if ncolor == Blank, and repaint
>   to Skip otherwise, potentially choosing a new common ancestor.
> 
> I'll try those changes in a bit to see if that approach is promising.
> I *think* it should at least guarantee that the node it returns is
> not reachable from any other common ancestor of A and B.
> 
> There are still a few edge cases like the following that I'm not
> sure how it would handle, but I don't know how often those come up
> in practice.
> 
> A   B
> |\ /|
> | \ |
> |/ \|
> C   D
>  \  |
>   E |
>    \|
>     F
>     |
>    ...

In this case, torvalds git will return a list of
{c,d} as the set of ancestors. merging will be
done recursively, creating a virtual commit that
is merge(c,d) -- and if that creates a cross
branch, then the merge will be repeated.

for us, I'd be happy if we pick an arbitrary parent.

Though, in this case, thinking about it, we can
return *all* minimums in the reachability boundary
if we want to do that.

(I don't think we want to)


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2022-03-06 18:40 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-03-05 19:32 [9front] git lca bug Michael Forney
2022-03-06  3:44 ` ori
2022-03-06  5:21   ` Michael Forney
2022-03-06 18:33     ` ori

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).