help / color / mirror / Atom feed
From: Ingo Schwarze <>
Subject: Re: half-atomically rebuild databases
Date: Wed, 4 Jan 2012 01:29:24 +0100	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

Hi Kristaps,

Kristaps Dzonsons wrote on Tue, Jan 03, 2012 at 01:39:28PM +0100:
> Ingo Schwarze wrote:

>> Creating both files under temporary names in the same directory
>> and rename both to the normal file names when both are ready.

> I don't like this: no matter how we splice it, at some point the
> files will be inconsistent.

Agreed.  There is no way to atomically replace two existing files
with two new files, and combining both into a single file as suggested
by Joerg in not an option right now.

> I also don't think it's unreasonable to
> have a "bogus" database for a few seconds when they're being rebuilt
> so long as this is documented.

Well, with my patch, it's not a few seconds, but maybe a few
milliseconds, or even less.

On the other hand, without my patch, it is more than a few seconds.
On one of my old i386 servers (still in production use),
it is about 45 seconds.  On my sparc64, 4 minutes 40 seconds.
On the VAX or hp300, i have no idea, except that it will be *much*
worse.  During that time, apropos(1) is out of service.
And then, if the job crashes near the end, you are left without
any database, for good.

> However, if this is a problem scenario,

I don't think the rename-rename race is a problem scenario.
But i do think being without any database for several minutes,
or more likely many hours on the VAX, is a problem.

> we can just use flock(2) and
> be upfront that the database is inconsistent (apropos_db.c and
> mandocdb.c would need to sync open order) instead of crossing our
> fingers and hoping that they don't dbopen(3) between rename(2)s.

I don't object to that, but don't consider it an urgent matter
either.  It can easily be added later, on top of my patch.

> Thoughts?

I'm aware that my patch doesn't *completely* solve the issue,
but it solves it for almost all practical purposes.
So, what do you mean by "I don't like this":
Do you mean to say that you like the current situation better
than the one with the patch?
If so, why?

 To unsubscribe send an email to

  reply	other threads:[~2012-01-04  0:29 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2012-01-01 15:45 Ingo Schwarze
2012-01-01 15:55 ` Joerg Sonnenberger
2012-01-01 16:29   ` Ingo Schwarze
2012-01-03 12:39     ` Kristaps Dzonsons
2012-01-04  0:29       ` Ingo Schwarze [this message]
2012-01-03 12:45     ` Joerg Sonnenberger
2012-01-03 13:07       ` Kristaps Dzonsons
2012-01-03 13:19         ` Joerg Sonnenberger
2012-01-03 13:45           ` Kristaps Dzonsons
2012-01-03 13:49             ` Joerg Sonnenberger
2012-01-03 23:59               ` Ingo Schwarze
2012-01-04  0:09             ` Ingo Schwarze

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).