From mboxrd@z Thu Jan 1 00:00:00 1970 Message-Id: <200205272037.g4RKbkM2002188@ducky.net> To: 9fans@cse.psu.edu From: Mike Haertel Subject: [9fans] some thoughts on replica and irreproducible results Date: Mon, 27 May 2002 13:37:46 -0700 Topicbox-Message-UUID: 9d98c482-eaca-11e9-9e20-41e7f4b1d025 So, I just ran replica again to get this morning's batch of changes. Here's what I saw: term% replica/pull -v /dist/replica/network c 386/bin/dd c sys/src/cmd/dd.c c sys/src/cmd/acme/fsys.c c sys/src/cmd/upas/fs/fs.c c sys/src/9/pc/sd53c8xx.c c sys/src/9/pc/sd53c8xx.n The issue: some of the maintainers are updating the corresponding binaries to updated sources, while others are not. Unfortunately, if I go and update these binaries myself, and then later someone else updates them on sources, replica assumes they are locally modified and stops tracking the version on sources. I can override this with replica/pull -s, but it's not safe to be doing that by default and it's potentially error prone to go and do it manually if I have genuine local changes that I've since forgotten about. (Ok, admittedly the #1 solution to the latter is to keep all my genuine local changes in a nonstandard location...) Now, for an experiment and then a suggestion: term% mkdir /tmp/cmd term% bind -bc /tmp/cmd /sys/src/cmd term% cd /sys/src/cmd term% mk 8.dd 8^c -FVw dd.c 8^l -o 8.dd dd.8 term% cmp 8.dd /386/bin/dd term% Cool, no differences! So nobody is doing anything dumb like putting gratuitous timestamps in binaries. If I build the same binary twice I get the same bytes. This is a feature. Therefore one simple improvement to replica might be: if a file appears to be locally modified, but is actually byte-for-byte identical to the corresponding file on sources, then replica should "update" the file after all (including changing the mtime to match the file on sources). However the problem is wider than it first appears. In the course of investigating this problem I also discovered (picking "ls" at random): term% mk 8.ls 8^c -FVw ls.c 8^l -o 8.ls ls.8 term% cmp 8.ls /386/bin/ls 8.ls /386/bin/ls differ: char 20 This happened on the very first command other than "dd" that I tried to build, and it suggests that quite a few binaries may be out of date with the sources. So perhaps it might also be worthwhile to have a cron job on sources that would periodically rebuild all the binaries and compare with the installed versions. However, some obnoxious programs like "ar" keep timestamps in binaries that are gratuitiously updated even when the object file contents themselves haven't changed. Furthermore The lack of shared libraries would also cause a lot of binary turnover anytime somebody updated a popular routine in the C library. Designing an automatic build that wouldn't produce a huge volume of updates would be hard.