From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from scc-mailout.scc.kit.edu (scc-mailout.scc.kit.edu [129.13.185.202]) by krisdoz.my.domain (8.14.5/8.14.5) with ESMTP id q040TPA2023109 for ; Tue, 3 Jan 2012 19:29:25 -0500 (EST) Received: from hekate.usta.de (asta-nat.asta.uni-karlsruhe.de [172.22.63.82]) by scc-mailout-02.scc.kit.edu with esmtp (Exim 4.72 #1) id 1RiEjM-0005Cr-P6; Wed, 04 Jan 2012 01:29:24 +0100 Received: from donnerwolke.usta.de ([172.24.96.3]) by hekate.usta.de with esmtp (Exim 4.72) (envelope-from ) id 1RiEjM-0006xs-OG for tech@mdocml.bsd.lv; Wed, 04 Jan 2012 01:29:24 +0100 Received: from iris.usta.de ([172.24.96.5] helo=usta.de) by donnerwolke.usta.de with esmtp (Exim 4.72) (envelope-from ) id 1RiEjM-0005r8-N9 for tech@mdocml.bsd.lv; Wed, 04 Jan 2012 01:29:24 +0100 Received: from schwarze by usta.de with local (Exim 4.72) (envelope-from ) id 1RiEjM-0004h3-M6 for tech@mdocml.bsd.lv; Wed, 04 Jan 2012 01:29:24 +0100 Date: Wed, 4 Jan 2012 01:29:24 +0100 From: Ingo Schwarze To: tech@mdocml.bsd.lv Subject: Re: half-atomically rebuild databases Message-ID: <20120104002924.GE2607@iris.usta.de> References: <20120101154538.GA5307@iris.usta.de> <20120101155508.GA4516@britannica.bec.de> <20120101162958.GB5307@iris.usta.de> <4F02F700.1060506@bsd.lv> X-Mailinglist: mdocml-tech Reply-To: tech@mdocml.bsd.lv MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4F02F700.1060506@bsd.lv> User-Agent: Mutt/1.5.21 (2010-09-15) Hi Kristaps, Kristaps Dzonsons wrote on Tue, Jan 03, 2012 at 01:39:28PM +0100: > Ingo Schwarze wrote: >> Creating both files under temporary names in the same directory >> and rename both to the normal file names when both are ready. > I don't like this: no matter how we splice it, at some point the > files will be inconsistent. Agreed. There is no way to atomically replace two existing files with two new files, and combining both into a single file as suggested by Joerg in not an option right now. > I also don't think it's unreasonable to > have a "bogus" database for a few seconds when they're being rebuilt > so long as this is documented. Well, with my patch, it's not a few seconds, but maybe a few milliseconds, or even less. On the other hand, without my patch, it is more than a few seconds. On one of my old i386 servers (still in production use), it is about 45 seconds. On my sparc64, 4 minutes 40 seconds. On the VAX or hp300, i have no idea, except that it will be *much* worse. During that time, apropos(1) is out of service. And then, if the job crashes near the end, you are left without any database, for good. > However, if this is a problem scenario, I don't think the rename-rename race is a problem scenario. But i do think being without any database for several minutes, or more likely many hours on the VAX, is a problem. > we can just use flock(2) and > be upfront that the database is inconsistent (apropos_db.c and > mandocdb.c would need to sync open order) instead of crossing our > fingers and hoping that they don't dbopen(3) between rename(2)s. I don't object to that, but don't consider it an urgent matter either. It can easily be added later, on top of my patch. > Thoughts? I'm aware that my patch doesn't *completely* solve the issue, but it solves it for almost all practical purposes. So, what do you mean by "I don't like this": Do you mean to say that you like the current situation better than the one with the patch? If so, why? Yours, Ingo -- To unsubscribe send an email to tech+unsubscribe@mdocml.bsd.lv