Hello,

Enclosed [finally] is a patch against HEAD that uses sqlite3 for 
mandocdb and apropos (getting the others to use it is quite easy).  It's 
a work in progress, but is the simplest possible implementation before 
extensive polishing.  I'd like to get this in to start refining.

First of all, you'll need uthash (in ports) and sqlite3 to build.  I 
chose uthash for hashtables just because I'm familiar with it.  I'd like 
to use espie@'s ohash stuff instead to reduce dependencies, but since 
uthash is just a header file, I'm not that concerned.

There are a few significant differences between this and the last 
mandocdb family of tools:

  1.  apropos_db.c -> mansearch.c (simplified search interface)
  2.  UTF-8 encoding in database
  3.  fts(3) in mandocdb
  4.  for the time being, only OR'd expressions using IS/GLOB
  5.  lots of hashing (uthash) in mandocdb
  6.  only apropos/mandocdb/manpage are enabled (for now)

Point (1) is self-explanatory.  You'll also notice that mansearch.c is 
quite small: this is due to offloading the search into sqlite3 and point 
(4).

Point (2) is nice: it reduces the amount of encode juggling and allows 
people to use the sqlite3 database itself without worrying about escape 
recoding.

Point (3) considerably simplified mandocdb.c.  I think it can be even 
more simplified (too much path matching), but we need to think first. 
(It's a little longer than the initial mandocdb.c due to the SQL schema 
and UTF-8 coding.)

Point (4) is because I want to start simple.  AND operations and 
subexpressions require more SQL knowledge than I have.  Same goes with 
using GLOB/LIKE/matches and so on.  Start easy.

Point (5) makes mandocdb.c a little complicated.  There are three 
hashes: a) the keyword hash, b) the filename hash, and c) the ino/dev 
hash.  Hash (a) avoids constant allocation and reallocation of memory 
when parsing words.  This is great, but unfortunately, the table can be 
HUGE and it's possible to run out of memory.  I need to re-examine this 
strategy a bit more.  Hash (b) lets me match catpages to manpages, and 
hash (c) avoids duplicate files.

I just noticed that since I started most of these files over, I didn't 
retain the copyrights from the original---I'll put them back in later.

Thoughts?

Kristaps