New db format (was Re: call for testing: mandoc-1.13.4 release candidate)

[Baptiste Daroussin <bapt@FreeBSD.org>]

[-- Attachment #1: Type: text/plain, Size: 1851 bytes --]

On Sat, Jul 09, 2016 at 08:31:58AM -0600, Anthony J. Bentley wrote:
> Baptiste Daroussin FreeBSD writes:
> > > P.S.
> > > It seems likely that this will be the last release needing SQLite
> > > for database support.  I have started work on removing the dependency
> > > on SQLite without loss of functionality and without needing any
> > > replacement, relying on POSIX interfaces only.  So it is likely
> > > that we will have a 1.14.1 release this autumn that no longer needs
> > > SQLite.
> > 
> > Tell me more, I'm really interestedin that, is there a place were we can
> > look at this work in progress code?
> 
> Ingo sent an initial patch to the OpenBSD tech@ list a few days ago:
> 
> https://marc.info/?l=openbsd-tech&m=146734380814620&w=2
> 
Thank you,

Beside minor portability issues (that I will provide fixes for once this patch
is official) I can say it works fine on FreeBSD :)

Tested on FreeBSD 12-CURRENT:

The 3 main "bad" feedbacks I got about mandoc was:
- time to create the db was long with sqlite with the new code it is longer:
  does from 4s to 11s in general for the base manpages but I do not consider
  that as a big issue given it is not run that often and the code is quite new
  and there are probably room of improvements

- size of the db (for embedded). With the new code:
  3.8M    /usr/share/man/mandoc.db
  1.6M    /usr/share/man/mandoc.new.db

  This is a big improvement

- apropos can be very slow before showing anything, with the new code, it is
  instant!
  old 'apropos ls':
          8.67 real         3.63 user         5.03 sys
  new 'apropos ls':
          0.02 real         0.02 user         0.00 sys


I notice some memory corruption in the ouput of apropos, I haven't dig into it
yet

Great work, I'm eager to see that code in!

Best regards,
Bapt

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]