example queries for nnir

example queries for nnir

[Harry Putnam]

A small request for nnir users with some experience, to post a few
sample queries  using the `glimpse' or `wais' engines.

Very unfamiliar with wais here, so any examples would be very usefull.

Glimpse queries can be difficult since glimpse doesn't allow full
regexp in the actual query string.

This can make something like searching for a subject where you don't
know the full string difficult since ^Subject:.*KEYWORDS won't work.

In fact even when you do know the subject line in toto the glimpse
syntax in nnir usage can be elusive.

Maybe, if some nice generic broad usage examples showup someone (like
me) will sort/edit them and submit a patch to Kai for possible
inclusion in the nnir comments.

Also some clear examples of how the nnir variables might be set would
be a good inclusion in that document's comments.

An example that can be confusing is(from nnir.el):

;; . . . . . . . . . . . . . . . . . . . . . . . . . . . . The
;; second variable to set is `nnir-search-engine'.  Choose one of the
;; engines listed in `nnir-engines'.

This is not really a list of engines at all but lisp code:
nnir-engines's value is 
((glimpse nnir-run-glimpse
	  ((group . "Group spec: ")))
 (wais nnir-run-waissearch nil)
 (excite nnir-run-excite-search nil)
 (imap nnir-run-imap nil))


When the needed variable setting for glimpse is:
 (setq nnir-search-engine `glimpse)

Not immediately apparent what is needed unless you know lisp.

Showing some example settings would be a little less confusing.

A final query.. Does anyone know of a search tool that uses a database
but also allows full regexp use in the queries?  This seems to be
mutually exclusive, or something.

Re: example queries for nnir

[Kai Großjohann]

Harry Putnam <reader@newsguy.com> writes:

> A small request for nnir users with some experience, to post a few
> sample queries  using the `glimpse' or `wais' engines.
> 
> Very unfamiliar with wais here, so any examples would be very usefull.

Using the example format file given in nnir.el, there are four fields:
to, from, subject, and the global field.

The to field contains the To and Cc mail headers.  The from field
contains the From mail header.  The subject field contains the Subject
mail header.

The global field contains all of the above headers, plus the body of
each message.

Thus, the query `foo' searches for foo occuring anywhere in the body
or in the above mentioned header fields.  `from=miller' searches for
`miller' in the From header.  `to=smith' searches for `smith' in the
To and Cc headers.  I think you can see what `subject=bla' does...

If you want to search for several words, you can say `to=(smith and
miller)' to search for messages which were sent to both people at the
same time.  `to=(smith or miller)' finds messages sent to smith, as
well as messages sent to miller, as well as messages sent to both
smith and miller.

Due to ranking, messages sent to both smith and miller have a higher
score than messages sent to only smith or only miller.  The score is
that four-digit number in the beginning of the subject header.

Since `or' is the default, you can say `to=(smith miller)' instead of
`to=(smith or miller)'.

You can also combine searching in several fields, such as

    from=miller and subject=hungry and blarfl

Here, a matching mail must be from miller, and `hungry' must be in the
subject header, and `blarfl' must be somewhere in the body (or in the
From, To, Cc, Subject header).

> Glimpse queries can be difficult since glimpse doesn't allow full
> regexp in the actual query string.
> 
> This can make something like searching for a subject where you don't
> know the full string difficult since ^Subject:.*KEYWORDS won't work.

It doesn't?  The documentation seems to say it should.  Does
`^Subject:#KEYWORD' do anything useful?

Oh, now I see that the documentation doesn't mention `^', so that's
why it doesn't work.  But does `Subject:#KEYWORD' work?

`#' seems to be an abbrev for `.*'.

> An example that can be confusing is(from nnir.el):
> 
> ;; . . . . . . . . . . . . . . . . . . . . . . . . . . . . The
> ;; second variable to set is `nnir-search-engine'.  Choose one of the
> ;; engines listed in `nnir-engines'.

I have now included an explicit pointer to the variable
documentation.  I have also tried to beef up the documentation for
nnir-engines.  Did it help?  New version on ftp server,
ftp://ls6-ftp.cs.uni-dortmund.de/pub/src/emacs/

> Showing some example settings would be a little less confusing.

Right.  Hm.  I'll do that.

> A final query.. Does anyone know of a search tool that uses a database
> but also allows full regexp use in the queries?  This seems to be
> mutually exclusive, or something.

Normal search engines build a word-based index, and when they see a
query for the word foo, they look in the index for all documents
containing that word.  This approach does not really play ball with
full-blown regular expressions...

For regular expressions, you would have to scan each document in
succession, and with multi-gigabyte databases, this is just much too
slow.

But it does appear that Glimpse tries to strike a balance between
added efficiency due to indexes and the power of regular expressions.

kai
-- 
I like BOTH kinds of music.

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

^^^^^8< snipped good clues and examples about freeWAIS

> It doesn't?  The documentation seems to say it should.  Does
> `^Subject:#KEYWORD' do anything useful?
> 
> Oh, now I see that the documentation doesn't mention `^', so that's
> why it doesn't work.  But does `Subject:#KEYWORD' work?
> 
> `#' seems to be an abbrev for `.*'.

You've neatly summed up the reason I say it is difficult.

Trying: C-u G G <RET>
Query: 'Subject:#queries'
or
Query: Subject:#queries

Either way produces:
Invalid read syntax: "#"

Query: Subject:.*queries
Invalid read syntax: "."

Even:
Query: ^Subject: example queries for nnir
or 
Query: Subject: example queries for nnir
with and without single quotes
Wrong type argument: listp, example

Trying to use the ";" operator (even though it would leave a gaping
whole for false hits): 
Query: Subject:;queries 
End of file during parsing

All the above leave room for false hits without the "^".  

I think this points up the weakness I mentioned.  And a good reason
why it would be nice if RE were usable in the queries of glimpse.

It appears that freeWAIS get around this problem neatly by use of
`fields' as you've described. 

Re: example queries for nnir

[Kai Großjohann]

Harry Putnam <reader@newsguy.com> writes:

> You've neatly summed up the reason I say it is difficult.
> 
> Trying: C-u G G <RET>
> Query: 'Subject:#queries'
> or
> Query: Subject:#queries
> 
> Either way produces:
> Invalid read syntax: "#"

Oh, no.  This must be a bug in nnir.el :-((

Sadly, I don't have time right now to try it out.  But I must find
that bug.  I was trying to be too clever in sneaking information past
Gnus -- the whole query is put into the group name and I really
shouldn't have tried to put such information into a string, where it
doesn't belong at all.

kai
-- 
I like BOTH kinds of music.

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Oh, no.  This must be a bug in nnir.el :-((

I guess it must be since trying these from the command line works:

glimpse-m -G -F ding2 'Subject:.*queries'
glimpse-m -G -F ding2 'Subject:#queries'

Which is news to me, and a usefull thing to know.
> 
> Sadly, I don't have time right now to try it out.  But I must find
> that bug.  I was trying to be too clever in sneaking information past
> Gnus -- the whole query is put into the group name and I really
> shouldn't have tried to put such information into a string, where it
> doesn't belong at all.

Not sure what all this means but it sounds very stealthy, gnus must be
a wise application to have spotted the ruse...

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

[...]

> > Very unfamiliar with wais here, so any examples would be very usefull.
> 
> Using the example format file given in nnir.el, there are four fields:
> to, from, subject, and the global field.
> 
> The to field contains the To and Cc mail headers.  The from field
> contains the From mail header.  The subject field contains the Subject
> mail header.
> 
> The global field contains all of the above headers, plus the body of
> each message.

So, there is no way to search just the body by itself?

[...] snipped good examples


> > ;; . . . . . . . . . . . . . . . . . . . . . . . . . . . . The
> > ;; second variable to set is `nnir-search-engine'.  Choose one of the
> > ;; engines listed in `nnir-engines'.
> 
> I have now included an explicit pointer to the variable
> documentation.  I have also tried to beef up the documentation for
> nnir-engines.  Did it help?  New version on ftp server,
> ftp://ls6-ftp.cs.uni-dortmund.de/pub/src/emacs/

Checking that version, I see what is changed but it still doesn't make
it clear how using the engines in that alist might look when included
in nnir variables.

Going form this:
  nnir-engines's value is 
  ((glimpse nnir-run-glimpse
            ((group . "Group spec: ")))
   (wais nnir-run-waissearch nil)
   (excite nnir-run-excite-search nil)
   (imap nnir-run-imap nil))

to this:
(setq nnir-search-engine 'glimpse)

Is not apparent from nnir.el  comments, unless you know a little lisp.

Including some simple examples of actual settings might be more
usefull.  Indicating which are default setting would help too. Nnir.el
does this for the some of the variables pertaining to glimpse and wais
already.

Sorry, I started to send you a patch to include some of what I think
would be usefull but quickly discovered I didn't know enough about
nnir varilables to be accurate.

What I had in mind would look something like this:

### examples for inclusion in ~/.gnus

## variable nnir-mail-backend example settings:

## If gnus-secondary-select-methods is set to only one of either nnml
## or nnmh backend then nnir will default to that backend (Kai, is
## this true?)

## If more than one mail backend is used then nnir-mail-backend can be 
## set like this (must be nnml or nnmh):
## (setq nnir-mail-backend  '(BACKEND "NAME"))  <==(probably not accurate)
## (setq nnir-mail-backend '(nnml "my-server"))

## variable nnir-search-engines example settings:
## (setq nnir-search-engine 'BACKEND)
## (setq nnir-search-engine 'glimpse)
## (setq nnir-search-engine 'wais)

Similar examples following the sections on glimpse and wais, in there
`setq' format for inclusion in gnus would be nice too:
Something like:

(setq nnir-glimpse-remove-prefix "/PATH/DIRECTORY/")

or

(setq nnir-glimpse-home "/PATH/DIRECTORY/"/

It would also be handy if all nnir vars were settable with M-x
set-variable so users wouldn't have to go to customize or know the
elisp code to eval with M-: 

That would make it easier to experiment with various settings for a
given session.  M-x customize  is fairly cumbersome.

Re: example queries for nnir

[Kai Großjohann]

Harry Putnam <reader@newsguy.com> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> 
> > The to field contains the To and Cc mail headers.  The from field
> > contains the From mail header.  The subject field contains the Subject
> > mail header.
> > 
> > The global field contains all of the above headers, plus the body of
> > each message.
> 
> So, there is no way to search just the body by itself?

This depends on the format file.  In a `region...end' stanza, the BOTH
keyword means that the words are index both in the given field and in
the global field.  If you put LOCAL there, the words are only put into
the given field, and not in the global field.

Btw, I think `stemming SOUNDEX' is not a working combination.  Use
just `SOUNDEX' instead.

> [...] snipped good examples
> 
> 
> > > ;; . . . . . . . . . . . . . . . . . . . . . . . . . . . . The
> > > ;; second variable to set is `nnir-search-engine'.  Choose one of the
> > > ;; engines listed in `nnir-engines'.
> > 
> > I have now included an explicit pointer to the variable
> > documentation.  I have also tried to beef up the documentation for
> > nnir-engines.  Did it help?  New version on ftp server,
> > ftp://ls6-ftp.cs.uni-dortmund.de/pub/src/emacs/
> 
> Checking that version, I see what is changed but it still doesn't make
> it clear how using the engines in that alist might look when included
> in nnir variables.

Are you saying that (setq nnir-search-engine 'wais) and (setq
nnir-search-engine 'imap) is not sufficient and that I should include
the full list of allowed symbols?

> Going form this:
>   nnir-engines's value is 
>   ((glimpse nnir-run-glimpse
>             ((group . "Group spec: ")))
>    (wais nnir-run-waissearch nil)
>    (excite nnir-run-excite-search nil)
>    (imap nnir-run-imap nil))
> 
> to this:
> (setq nnir-search-engine 'glimpse)
> 
> Is not apparent from nnir.el  comments, unless you know a little lisp.

Hm.  Okay.  Hm.  What can I do?  List the allowed symbols for the
default value in the doc string?  Tell the user what to type to learn
which symbols are allowed?

> Including some simple examples of actual settings might be more
> usefull.

I have tried to do this in the doc string for nnir-engines.

> Indicating which are default setting would help too. Nnir.el does
> this for the some of the variables pertaining to glimpse and wais
> already.

If you don't change a variable, type C-h v to get the default value.
If you change a variable, M-x find-variable RET might help.

> ## variable nnir-mail-backend example settings:
> 
> ## If gnus-secondary-select-methods is set to only one of either nnml
> ## or nnmh backend then nnir will default to that backend (Kai, is
> ## this true?)

No, the default value for nnir-mail-backend is (nnml ""), regardless
of the gnus-secondary-select-methods setting.

Should I change the default value to be (car
gnus-secondary-select-methods)? 

> ## If more than one mail backend is used then nnir-mail-backend can be 
> ## set like this (must be nnml or nnmh):
> ## (setq nnir-mail-backend  '(BACKEND "NAME"))  <==(probably not accurate)
> ## (setq nnir-mail-backend '(nnml "my-server"))

Also see the documentation for nnir-mail-backend which says that it
might be best to get the value from gnus-secondary-select-methods.

You must include _all_ server parameters.

> ## variable nnir-search-engines example settings:
> ## (setq nnir-search-engine 'BACKEND)
> ## (setq nnir-search-engine 'glimpse)
> ## (setq nnir-search-engine 'wais)

I have included `wais' and `imap' examples in the documentation for
nnir-engines. 

> Similar examples following the sections on glimpse and wais, in there
> `setq' format for inclusion in gnus would be nice too:
> Something like:
> 
> (setq nnir-glimpse-remove-prefix "/PATH/DIRECTORY/")
> 
> or
> 
> (setq nnir-glimpse-home "/PATH/DIRECTORY/"/

I have now tried to do this.  What do you think about the new version?

> It would also be handy if all nnir vars were settable with M-x
> set-variable so users wouldn't have to go to customize or know the
> elisp code to eval with M-: 

Oops.

> That would make it easier to experiment with various settings for a
> given session.  M-x customize  is fairly cumbersome.

Right.

kai
-- 
I like BOTH kinds of music.

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:


[...]

> > 
> > So, there is no way to search just the body by itself?
> 
> This depends on the format file.  In a `region...end' stanza, the BOTH
> keyword means that the words are index both in the given field and in
> the global field.  If you put LOCAL there, the words are only put into
> the given field, and not in the global field.

Unless there is the possibility of a `body' field, how can the above
allow a body only search?

> 
> Btw, I think `stemming SOUNDEX' is not a working combination.  Use
> just `SOUNDEX' instead.

OK .. thanks

> > [...] snipped good examples
> > 
> > 
> > > > ;; . . . . . . . . . . . . . . . . . . . . . . . . . . . . The
> > > > ;; second variable to set is `nnir-search-engine'.  Choose one of the
> > > > ;; engines listed in `nnir-engines'.
> > > 
> > > I have now included an explicit pointer to the variable
> > > documentation.  I have also tried to beef up the documentation for
> > > nnir-engines.  Did it help?  New version on ftp server,
> > > ftp://ls6-ftp.cs.uni-dortmund.de/pub/src/emacs/
> > 
> > Checking that version, I see what is changed but it still doesn't make
> > it clear how using the engines in that alist might look when included
> > in nnir variables.
> 
> Are you saying that (setq nnir-search-engine 'wais) and (setq
> nnir-search-engine 'imap) is not sufficient and that I should include
> the full list of allowed symbols?

Very sorry Kai, but I thought we were discussing the opening comments
in nnir.el.  Which should probably include the examples you refer to
above. Those that  appear in that actual code as documnet strings.  I had not
noticed those examples which probably means many new users would not either.

So no, I'm not saying all possible examples need to be shown only that
the examples should appear with the discussion in opening comments as well.

> 
> > Going form this:
> >   nnir-engines's value is 
> >   ((glimpse nnir-run-glimpse
> >             ((group . "Group spec: ")))
> >    (wais nnir-run-waissearch nil)
> >    (excite nnir-run-excite-search nil)
> >    (imap nnir-run-imap nil))
> > 
> > to this:
> > (setq nnir-search-engine 'glimpse)
> > 
> > Is not apparent from nnir.el  comments, unless you know a little lisp.
> 
> Hm.  Okay.  Hm.  What can I do?  List the allowed symbols for the
> default value in the doc string?  Tell the user what to type to learn
> which symbols are allowed?

Probably best to include the examples in the opening comments where
the variable are first mentioned, *and* instruct how to see the doc
string documentation.  Sorry if this seems like over kill but it seems
to me this should all be included where the discussion of nnir
varilables takes place.

> > Including some simple examples of actual settings might be more
> > usefull.
> 
> I have tried to do this in the doc string for nnir-engines.

Yes I see that now.. I had missed it in my previous post.

> > Indicating which are default setting would help too. Nnir.el does
> > this for the some of the variables pertaining to glimpse and wais
> > already.
> 
> If you don't change a variable, type C-h v to get the default value.
> If you change a variable, M-x find-variable RET might help.

Yes, for experienced emacs users this is common knowledge but you
seemed to feel it was necessary to mention some defaults in the
opening nnir.el comments like:

   ;; . . . . . . . . . . . . . . . . . . . . . . . . The variable
   ;; `nnir-wais-remove-prefix' should contain a regex matching exactly
   ;; this prefix.  It defaults to `$HOME/Mail/' (note the trailing
   ;; slash).

and:

  ;; . . . . . . . . . . . . . . . . . .The `nnir-glimpse-home'
  ;; variable should be set to the value of the `-H' option which allows
  ;; one to search this Glimpse index.  I have indexed my whole home
  ;; directory with Glimpse, so I assume a default of `$HOME'.

I think this is a good practice and was only suggesting that any of
the variables discussed in comments should also have any defaults
pointed out.

> > ## variable nnir-mail-backend example settings:
> > 
> > ## If gnus-secondary-select-methods is set to only one of either nnml
> > ## or nnmh backend then nnir will default to that backend (Kai, is
> > ## this true?)
> 
> No, the default value for nnir-mail-backend is (nnml ""), regardless
> of the gnus-secondary-select-methods setting.
> 
> Should I change the default value to be (car
> gnus-secondary-select-methods)? 

Err no.... probably not since that would probably give confusing
results to people with nnfolder or whatever.  Probably sufficient to
mention that `nnml ""' is the default.  Also it is clearly pointed out
in nnir.el comments that nnml and nnmh are the only backends supported.

> 
> > ## If more than one mail backend is used then nnir-mail-backend can be 
> > ## set like this (must be nnml or nnmh):
> > ## (setq nnir-mail-backend  '(BACKEND "NAME"))  <==(probably not accurate)
> > ## (setq nnir-mail-backend '(nnml "my-server"))
> 
> Also see the documentation for nnir-mail-backend which says that it
> might be best to get the value from gnus-secondary-select-methods.
> 
> You must include _all_ server parameters.
> 
> > ## variable nnir-search-engines example settings:
> > ## (setq nnir-search-engine 'BACKEND)
> > ## (setq nnir-search-engine 'glimpse)
> > ## (setq nnir-search-engine 'wais)
> 
> I have included `wais' and `imap' examples in the documentation for
> nnir-engines. 
> 
> > Similar examples following the sections on glimpse and wais, in there
> > `setq' format for inclusion in gnus would be nice too:
> > Something like:
> > 
> > (setq nnir-glimpse-remove-prefix "/PATH/DIRECTORY/")
> > 
> > or
> > 
> > (setq nnir-glimpse-home "/PATH/DIRECTORY/"/
> 
> I have now tried to do this.  What do you think about the new version?

I don't see that in the opening comments.  The variable documentation
is superb now.  I may be all wet thinking these examplse should also
appear there but it seems to follow if it makes sense to have the
discussion in the opening comments.  Not a big issue I guess.

Re: example queries for nnir

[Kai Großjohann]

Harry Putnam <reader@newsguy.com> writes:

> Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:
> 
> > This depends on the format file.  In a `region...end' stanza, the BOTH
> > keyword means that the words are index both in the given field and in
> > the global field.  If you put LOCAL there, the words are only put into
> > the given field, and not in the global field.
> 
> Unless there is the possibility of a `body' field, how can the above
> allow a body only search?

If you specify the empty line, ie /^$/ as the beginning regexp, and
for the end regexp you either use something which never matches (for
nnml) or the record separator (/^From /, for nnfolder or mbox files),
then you have defined a region which comprises the body of a message.

Once you have such a region, you can either put all the words in it
into the global field, or into some other field.  In both cases, if
you want to be able to restrict the search to the body, you should NOT
specify another region which puts the words into the global field, or
into the other field.

My suggestion meant that you put all the words from the body into the
global field, and no other words into the global field.  Hence
searching the global field is the same as searching the body.

> > Are you saying that (setq nnir-search-engine 'wais) and (setq
> > nnir-search-engine 'imap) is not sufficient and that I should include
> > the full list of allowed symbols?
> 
> Very sorry Kai, but I thought we were discussing the opening
> comments in nnir.el.  Which should probably include the examples you
> refer to above. Those that appear in that actual code as documnet
> strings.  I had not noticed those examples which probably means many
> new users would not either.

Hm.  Okay.  I have tried to explicitly include a pointer to the
docstrings in the opening comments, but apparently this was not
explicit enough.  Do you think a wording along the lines of `type C-h
v nnir-search-engine RET for more information' would be clear enough?

If I were to document the same thing in the opening comments and in
the doc strings, chances are that one of them will be obsolete after
the next nnir.el changes.  Therefore, I prefer not to document the
same thing twice.

Do you think it would be better to document it twice?  What about the
newly changed wording in the opening comments?

> I think this is a good practice and was only suggesting that any of
> the variables discussed in comments should also have any defaults
> pointed out.

Okay.  FWIW, I have tried to make the variable documentation more
explicit, too.

> I don't see that in the opening comments.  The variable documentation
> is superb now.  I may be all wet thinking these examplse should also
> appear there but it seems to follow if it makes sense to have the
> discussion in the opening comments.  Not a big issue I guess.

I have already said why I think it's better not to document the same
thing twice...

kai
-- 
I like BOTH kinds of music.

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:


[...]

> 
> My suggestion meant that you put all the words from the body into the
> global field, and no other words into the global field.  Hence
> searching the global field is the same as searching the body.


Hoookay now I get it... click!

> 
> > > Are you saying that (setq nnir-search-engine 'wais) and (setq
> > > nnir-search-engine 'imap) is not sufficient and that I should include
> > > the full list of allowed symbols?
> > 
> > Very sorry Kai, but I thought we were discussing the opening
> > comments in nnir.el.  Which should probably include the examples you
> > refer to above. Those that appear in that actual code as documnet
> > strings.  I had not noticed those examples which probably means many
> > new users would not either.
> 
> Hm.  Okay.  I have tried to explicitly include a pointer to the
> docstrings in the opening comments, but apparently this was not
> explicit enough.  Do you think a wording along the lines of `type C-h
> v nnir-search-engine RET for more information' would be clear enough?

Looking at version 1.57. It looks very clear... I even understood it..
he he

> Do you think it would be better to document it twice?  What about the
> newly changed wording in the opening comments?

After looking at the new stuff, and understanding better the reasoning
involved, I guess I was making a lot of fuss about nothing.  It seems
quite clear how to proceed now. 

> 
> > I think this is a good practice and was only suggesting that any of
> > the variables discussed in comments should also have any defaults
> > pointed out.
> 
> Okay.  FWIW, I have tried to make the variable documentation more
> explicit, too.

I see the new documentation.  Seems much more complete and usefull.
Thanks for being so responsive to this, even though I lack much skill
in this stuff.

Is that really true that wais needs:
`nnir-wais-remove-prefix'?  It doesn't default to /home/$USER/Mail/?

I've only been using:
; (setq nnir-wais-database "/home/reader/.wais/mail1")
; (setq nnir-search-engine `wais)
in ~/.gnus when trying to work with wais.
Maybe part of my troubles.

This new documentation you've done is really well done, I hope it
turns out to be worth the work you put in.

Re: example queries for nnir

[Kai Großjohann]

Harry Putnam <reader@newsguy.com> writes:

> Is that really true that wais needs:
> `nnir-wais-remove-prefix'?  It doesn't default to /home/$USER/Mail/?

It defaults to "$HOME/Mail/".  Which is the same thing in many cases,
but at work my home dir is /home-local/grossjoh...

Should it default to (file-name-as-directory (expand-file-name "~/Mail/"))?

kai
-- 
I like BOTH kinds of music.

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Harry Putnam <reader@newsguy.com> writes:
> 
> > Is that really true that wais needs:
> > `nnir-wais-remove-prefix'?  It doesn't default to /home/$USER/Mail/?
> 
> It defaults to "$HOME/Mail/".  Which is the same thing in many cases,
> but at work my home dir is /home-local/grossjoh...

That seems the most appropriate
> 
> Should it default to (file-name-as-directory (expand-file-name "~/Mail/"))?

No, I think not.. What I was commenting on was that the doc string
says:
"Note that you have to set additional variables for most backends.  For
example, the `wais' backend needs the variables `nnir-wais-program',
`nnir-wais-database' and `nnir-wais-remove-prefix'.

When in fact most people will not have to set the `remove prefix' but
will have to set the other two.

It wasn't a complaint, or cause for change, only made me wonder if I
should have been setting that explicitly, which of course would hurt
nothing.

Re: example queries for nnir

[Norbert Koch]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Should it default to 
> (file-name-as-directory (expand-file-name "~/Mail/"))?

At least it's at least as good as any other possibility :-)

norbert
-- 
Hello.  I know the divorce rate among unmarried Catholic
 Alaskan females!!

Re: example queries for nnir

[Harry Putnam]

Kai.Grossjohann@CS.Uni-Dortmund.DE (Kai Großjohann) writes:

> Once you have such a region, you can either put all the words in it
> into the global field, or into some other field.  In both cases, if
> you want to be able to restrict the search to the body, you should NOT
> specify another region which puts the words into the global field, or
> into the other field.
> 
> My suggestion meant that you put all the words from the body into the
> global field, and no other words into the global field.  Hence
> searching the global field is the same as searching the body.

Is this really possible?  I've experimented at some length with GLOBAL
and LOCAL keywords.  

Using the nnir  example *.fmt file, leaving the regexps as is and only
changing the index specs.  In short replace every occurance of BOTH
with LOCAL  and for simplicity every field spec is set to TEXT rather
than SOUNDEX.  So that they read `TEXT LOCAL'.

Only the `body' (/^$/) RE is left GLOBAL.

Posting the file in case I've got something else wrong in there.   But
this *.fmt file causes waisindex to dump core.  There are only about
1000 messages in the data base. 

 # Each mail is in a file, much like the MH format.
                                         
 # Document separator should never match -- each file is a document.
 record-sep: /^@this regex should never match@$/
                                         
 # Searchable fields specification.      
                                         
 region: /^[sS]ubject:/ /^[sS]ubject: */ 
         subject "Subject header" stemming TEXT LOCAL
 end: /^[^ \t]/                          
                                         
 region: /^([tT][oO]|[cC][cC]):/ /^([tT][oO]|[cC][cC]): */
         to "To and Cc headers" TEXT LOCAL
 end: /^[^ \t]/                          
                                         
 region: /^[fF][rR][oO][mM]:/ /^[fF][rR][oO][mM]: */
         from "From header" TEXT LOCAL
 end: /^[^ \t]/                          
                                         
 region: /^$/                            
         stemming TEXT GLOBAL            
 end: /^@this regex should never match@$/


With this command line:


   The last few lines of output: 

   2433: 7305: Jul 18 21:07:23 2000: 100: Total word count for dictionary
   is: 0

   2433: 7306: Jul 18 21:07:23 2000: -1: error finding
   total_word_count in dictionary ./mail
   
   2433: 7307: Jul 18 21:07:23 2000: -1: Could not read the dictionary
   block 139980800, length 1000

   Segmentation fault (core dumped)