git branches and compilation

git branches and compilation

[Bart Schaefer]

On Saturday, May 18, 2013, Phil Pennock wrote:

> On 2013-05-18 at 16:24 -0700, Bart Schaefer wrote:
> > (One of the few things I'm finding annoying about git is that
> > a cloned repository doesn't preserve file timestamps.)
>
> There's a post-checkout hook [...]

Having been annoyed a couple of times by things not recompiling when a
different branch is checked out (especially the texinfo inputs in Doc/,
leading to reams of bad node errors when doing "make info") I dug into
this a bit further and found "metastore".

http://david.hardeman.nu/software.php

It's available as a Debian package and seems so far to do what I want,
although I haven't really been able to test it thoroughly because no one
else has pushed anything in the past couple of days.

The following patch adds a couple of more lines to the .*ignore files
and three functions in Util/, two of which are git hooks and the third
of which creates symlinks in .git/hooks for the first two.  I'm curious
if anyone has any opinions/examples on whether this approach is best,
or if something requiring "git clone --template ..." is preferred.

It might also be ideal to have the metadata stored when stashing or at
least (in the old branch) before checkout (of the new), but there don't
seem to be hooks for either of those.

I won't push this patch without some discussion from the list first.

diff --git a/.cvsignore b/.cvsignore
index 95cdc58..a5f7191 100644
--- a/.cvsignore
+++ b/.cvsignore
@@ -14,3 +14,4 @@ stamp-h.in
 autom4te.cache
 *.swp
 .git
+.metadata
diff --git a/.gitignore b/.gitignore
index d0172e3..b89a5cc 100644
--- a/.gitignore
+++ b/.gitignore
@@ -9,6 +9,7 @@ TAGS
 *~
 .*.sw?
 \#*
+.metadata
 
 /META-FAQ
 /config.cache
diff --git a/Util/add-git-hooks b/Util/add-git-hooks
new file mode 100755
index 0000000..77b2ac2
--- /dev/null
+++ b/Util/add-git-hooks
@@ -0,0 +1,38 @@
+#! /bin/sh
+
+HOOKS="pre-commit post-checkout"
+
+case `pwd` in
+*/Util) cd ..;;
+esac
+
+if ! [ -d .git/hooks ]
+then
+	echo "No directory for git hooks" >&2
+	exit 1
+fi
+
+if metastore -h 2>/dev/null 1>&2
+then
+	echo "Installing in `pwd`/.git/hooks ..." >&2
+	cd .git/hooks
+	for hook in $HOOKS
+	do
+		if [ -e $hook ]
+		then
+			if cmp -s ../../Util/git-$hook $hook
+			then
+				echo "$hook is already installed" >&2
+			else
+				echo "Another $hook already exists" >&2
+			fi
+		else
+			ln -s ../../Util/git-$hook $hook
+			chmod +x $hook
+		fi
+	done
+else
+	echo "Did not find metastore, skipping hooks" >&2
+fi
+
+exit 0
diff --git a/Util/git-post-checkout b/Util/git-post-checkout
new file mode 100755
index 0000000..a15d1d9
--- /dev/null
+++ b/Util/git-post-checkout
@@ -0,0 +1,24 @@
+#!/bin/bash
+#
+# Hook script to restore metadata information using metastore
+# following each checkout.
+
+# Do not restore metadata on a checkout from index
+if [ "$3" -eq 0 ]
+then
+	exit 0
+fi
+
+STOREFILE=".metadata/$(git symbolic-ref HEAD)/.metadata"
+
+if [ ! -e "$STOREFILE" ]; then
+	echo "No .metadata yet" >&2
+	exit 0
+fi
+
+if ! metastore -q -a -f "$STOREFILE"; then
+	echo "Failed to execute metastore -a" >&2
+	exit 1
+fi
+
+exit 0
diff --git a/Util/git-pre-commit b/Util/git-pre-commit
new file mode 100755
index 0000000..6030d12
--- /dev/null
+++ b/Util/git-pre-commit
@@ -0,0 +1,25 @@
+#!/bin/bash
+#
+# Hook script to store metadata information using metastore
+# before each commit.
+
+STOREPATH=".metadata/$(git symbolic-ref HEAD)"
+STOREFILE="$STOREPATH/.metadata"
+
+if [ ! -d "$STOREPATH" ] && ! mkdir -p "$STOREPATH"
+then
+	echo "Cannot create $STOREPATH, committing anyway" >&2
+	exit 0
+fi
+
+if ! metastore -s -f "$STOREFILE"; then
+	echo "Failed to execute metastore -s" >&2
+	exit 1
+fi
+
+if [ ! -e "$STOREFILE" ]; then
+	echo ".metadata missing after metastore -s" >&2
+	exec test \! -x .git/hooks/post-checkout
+fi
+
+exit 0

-- 
Barton E. Schaefer

Re: git branches and compilation

[Mikael Magnusson]

On 15 June 2013 21:46, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Saturday, May 18, 2013, Phil Pennock wrote:
>
>> On 2013-05-18 at 16:24 -0700, Bart Schaefer wrote:
>> > (One of the few things I'm finding annoying about git is that
>> > a cloned repository doesn't preserve file timestamps.)
>>
>> There's a post-checkout hook [...]
>
> Having been annoyed a couple of times by things not recompiling when a
> different branch is checked out (especially the texinfo inputs in Doc/,
> leading to reams of bad node errors when doing "make info") I dug into
> this a bit further and found "metastore".
>
> http://david.hardeman.nu/software.php
>
> It's available as a Debian package and seems so far to do what I want,
> although I haven't really been able to test it thoroughly because no one
> else has pushed anything in the past couple of days.

I don't quite understand the purpose... When you change branches you want
the source files to have an updated mtime, otherwise they will not be newer
than the build files of the previously checked out branch and make will not
rebuild. So what git does is what you usually want for source code -> "the
files on disk changed so rebuild". How does setting the timestamps older
help?

--
Mikael Magnusson

Re: git branches and compilation

[Bart Schaefer]

On Jun 17, 12:57pm, Mikael Magnusson wrote:
}
} > Having been annoyed a couple of times by things not recompiling when a
} > different branch is checked out (especially the texinfo inputs in Doc/,
} > leading to reams of bad node errors when doing "make info") I dug into
} > this a bit further and found "metastore".
} 
} I don't quite understand the purpose... When you change branches you want
} the source files to have an updated mtime, otherwise they will not be newer
} than the build files of the previously checked out branch and make will not
} rebuild. So what git does is what you usually want for source code -> "the
} files on disk changed so rebuild". How does setting the timestamps older
} help?

This is probably because I'm using build trees that are separated from
the source tree (cf. patch in 31474) so I have several different sets
of build files with different timestamps.  When I change brances, only
the files that are different on the new branch get an updated mtime.  In
some case "different" means "reverted to an older version" which is not
really newly modified.

This causes the relative mtimes of source files to one another to be off,
so in some cases the Makefiles themselves need to be rebuilt but are not.
This also affects doc files that have cross-references and all need to be
rebuilt together, but don't have mutual dependency reflected in Makefile.

It's possible that in the latter case I'm just masking the problem and NOT
recompiling things that need to be.  As I said, I haven't had a chance to
test it very thoroughly.

I could believe that (re)storing timestamps of the object files may cause
a problem, so in fact it's likely essential to the process that I'm using
separate build trees.

Re: git branches and compilation

[Richard Hartmann]

On Sat, Jun 15, 2013 at 9:46 PM, Bart Schaefer
<schaefer@brasslantern.com> wrote:

> Having been annoyed a couple of times by things not recompiling when a
> different branch is checked out (especially the texinfo inputs in Doc/,
> leading to reams of bad node errors when doing "make info") I dug into
> this a bit further and found "metastore".

As a word of warning, metastore's binary storage format is _not_ merge
friendly. You must not diverge or branch off this file. David has lost
interest in metastore and does not intend to fix this.

Because of this, I wrote metamonger[1] to scratch this itch and
there's a branch which restores metadata. The master branch can only
store at the moment. Patches welcome; or if I know that someone
besides me wants to use this that may be the kick in the proverbial
behind I need to continue working on this, clean up history a bit, and
merge into master.


RIchard

[1] https://github.com/RichiH/metamonger

Re: git branches and compilation

[Bart Schaefer]

On Jul 24,  4:25pm, Richard Hartmann wrote:
}
} As a word of warning, metastore's binary storage format is _not_ merge
} friendly. You must not diverge or branch off this file.

The .metadata files are not part of the git repository, they're listed
in .gitignore.  Instead there is a separate subdirectory of metadata for
each git branch, which is maintained by the pre-commit and post-checkout
hooks.

As I mentioned, this is probably helpful only because I keep separate
build and source trees.

Thanks for the pointer to metamonger, though; I will look at it.