From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from smtp-2.sys.kth.se (smtp-2.sys.kth.se [130.237.32.160]) by krisdoz.my.domain (8.14.5/8.14.5) with ESMTP id q03DjEDY022175 for ; Tue, 3 Jan 2012 08:45:15 -0500 (EST) Received: from mailscan-1.sys.kth.se (mailscan-1.sys.kth.se [130.237.32.91]) by smtp-2.sys.kth.se (Postfix) with ESMTP id E2CBA14C137 for ; Tue, 3 Jan 2012 14:45:08 +0100 (CET) X-Virus-Scanned: by amavisd-new at kth.se Received: from smtp-2.sys.kth.se ([130.237.32.160]) by mailscan-1.sys.kth.se (mailscan-1.sys.kth.se [130.237.32.91]) (amavisd-new, port 10024) with LMTP id fpR5dT380cAQ for ; Tue, 3 Jan 2012 14:45:07 +0100 (CET) X-KTH-Auth: kristaps [193.10.49.5] X-KTH-mail-from: kristaps@bsd.lv X-KTH-rcpt-to: tech@mdocml.bsd.lv Received: from ctime.hhs.se (ctime.hhs.se [193.10.49.5]) by smtp-2.sys.kth.se (Postfix) with ESMTP id 60C7F14C133 for ; Tue, 3 Jan 2012 14:45:05 +0100 (CET) Message-ID: <4F030660.9040704@bsd.lv> Date: Tue, 03 Jan 2012 14:45:04 +0100 From: Kristaps Dzonsons User-Agent: Mozilla/5.0 (X11; OpenBSD amd64; rv:5.0) Gecko/20110805 Thunderbird/5.0 X-Mailinglist: mdocml-tech Reply-To: tech@mdocml.bsd.lv MIME-Version: 1.0 To: tech@mdocml.bsd.lv Subject: Re: half-atomically rebuild databases References: <20120101154538.GA5307@iris.usta.de> <20120101155508.GA4516@britannica.bec.de> <20120101162958.GB5307@iris.usta.de> <20120103124535.GA6689@britannica.bec.de> <4F02FDA4.6070003@bsd.lv> <20120103131927.GA7903@britannica.bec.de> In-Reply-To: <20120103131927.GA7903@britannica.bec.de> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit On 01/03/12 14:19, Joerg Sonnenberger wrote: > On Tue, Jan 03, 2012 at 02:07:48PM +0100, Kristaps Dzonsons wrote: >> On 01/03/12 13:45, Joerg Sonnenberger wrote: >>> On Sun, Jan 01, 2012 at 05:29:58PM +0100, Ingo Schwarze wrote: >>>> Hi Joerg, >>>> >>>> Joerg Sonnenberger wrote on Sun, Jan 01, 2012 at 04:55:08PM +0100: >>>>> On Sun, Jan 01, 2012 at 04:45:38PM +0100, Ingo Schwarze wrote: >>>> >>>>>> This is not perfect because the mandocdb process might get -KILLed >>>>>> between the two renames, but i don't see a syscall anywhere to >>>>>> atomically rename *two* files. >>>> >>>>> Why do you have to? >>>> >>>> There are two files to rename(2): whatis.{db,index}. >>> >>> Well, let me repeat the question, that is the point of having two >>> databases. >> >> Joerg---you mean to ask why I originally used two separate database files? > > Yes. Joerg, Duplication. By having the keyword database point to the index database entries, I avoided having duplicate index data everywhere---and this data can be huge since it includes titles, descriptions, and so on. Considering that the number of keywords is expected to grow as we add more values (full-text search?), this is quite important. Since libdb has only one table per file... It's still debatable whether using recno(3) for the index is a good idea. The general usage is to hit the btree(3) then look up in the recno(3) (btree(3) is harmless, I think, but useless: we don't do lexicographic lookup as I'd originally thought). However, look-up is slow in recno(3). I can definately see a case for using btree(3) or even hash(3) in place of recno(3). Make sense? Kristaps -- To unsubscribe send an email to tech+unsubscribe@mdocml.bsd.lv