Re: half-atomically rebuild databases

[Ingo Schwarze <schwarze@usta.de>]

Hi Kristaps,

Kristaps Dzonsons wrote on Tue, Jan 03, 2012 at 01:39:28PM +0100:
> Ingo Schwarze wrote:

>> Creating both files under temporary names in the same directory
>> and rename both to the normal file names when both are ready.

> I don't like this: no matter how we splice it, at some point the
> files will be inconsistent.

Agreed.  There is no way to atomically replace two existing files
with two new files, and combining both into a single file as suggested
by Joerg in not an option right now.

> I also don't think it's unreasonable to
> have a "bogus" database for a few seconds when they're being rebuilt
> so long as this is documented.

Well, with my patch, it's not a few seconds, but maybe a few
milliseconds, or even less.

On the other hand, without my patch, it is more than a few seconds.
On one of my old i386 servers (still in production use),
it is about 45 seconds.  On my sparc64, 4 minutes 40 seconds.
On the VAX or hp300, i have no idea, except that it will be *much*
worse.  During that time, apropos(1) is out of service.
And then, if the job crashes near the end, you are left without
any database, for good.

> However, if this is a problem scenario,

I don't think the rename-rename race is a problem scenario.
But i do think being without any database for several minutes,
or more likely many hours on the VAX, is a problem.

> we can just use flock(2) and
> be upfront that the database is inconsistent (apropos_db.c and
> mandocdb.c would need to sync open order) instead of crossing our
> fingers and hoping that they don't dbopen(3) between rename(2)s.

I don't object to that, but don't consider it an urgent matter
either.  It can easily be added later, on top of my patch.

> Thoughts?

I'm aware that my patch doesn't *completely* solve the issue,
but it solves it for almost all practical purposes.
So, what do you mean by "I don't like this":
Do you mean to say that you like the current situation better
than the one with the patch?
If so, why?

Yours,
  Ingo
--
 To unsubscribe send an email to tech+unsubscribe@mdocml.bsd.lv