Re: searching multiple groups from the Group buffer

[Harry Putnam <reader@newsguy.com>]

Colin W. asks:
>>	Is there a way I can search for a regexp in multiple groups?  For
>>	xample, I archive my mail in nnfolder archives by month, like
>>	xmail-2001-02, mail-2001-01, etc.


Kai Answers:
> For searching multiple groups, you can create an nnvirtual group that
> contains them all, then search normally.  Or you create an nnkiboze
> group which is similar to nnvirtual, but has the searching built-in.
 
> And then, there's always nnir.el which requires a search engine.
> Several people have volunteered to make it work with find/grep, but so
> far no go.  However, Glimpse is sufficiently simple to set up so it
> might do the trick for you.

Not sure about nnkiboze but the nnvirtual technique is the only one
mentioned here that really uses regexp.  From what I know, nnkiboze
isn't really working well either.

None of the currently supported search engines for nnir.el support
real regexp, far as I know. ( Is that correct Kai?)  Glimpse comes
closest, but doesn't allow full regexp either.

FreeWAIS is really a pain to setup and get working on linux.  I
finally got it working on my Freebsd box but it was such a pain (as
Kai will remember.. since I pestered him to death with it) I wouldn't
wish it on anybody.  It is very fast though. It is based on regexp in
the *.fmt file but the command lines are not.  Does not seem to be
near as robust as the glimpse software.

Colin, 

I've written a small shell/awk script that I use a lot, but kind of
stoped improving it once I got it working.  It is *NOT* integrated
into gnus or nnir as yet since I'm working on a more elaborate version
(sporadically).

To view

http://www.ptw.com/~reader/exp/search2.html

To download

http://www.ptw.com/~reader/exp/download/search2.tar.gz

It requires 3 regexp but has a default mode where you supply only one to
find in the body of messages.  Two regexp are searched for in headers,
one in body.  In the default mode  `From: ' and `Subject: ' are supplied by
the program and user supplies the body regexp on the command line with
`-b' flag.  Along with the directories to search with a `-d' flag.

NOTE: WARNING The script includes what documentation there is in the
form of `here documents' and comments.  Thes script may be somewhat
unconventional since I'm not really very experienced as a shell/awk
programmer.

The script knows when its in headers and when not and returns the hits
accordingly.  There must be 3 hits for anything to be returned.

A command line would look like:

 $ search2.sh -b "marking.*M-&" -d ~/Mail/prinb

  ( This will find your message in my  incoming pre-view group and
    return the regexp line plus `From' and `Subject' headers)

 (output would look like)

	/home/reader/Mail/prinb/30271
	30271|Subject: searching multiple groups from the Group buffer
	30271|From: Colin Walters <walters@cis.ohio-state.edu>
	30271|50|I tried marking those groups, and doing 'M-& RET', which did visit
	-- 

First section above shows file name where the hit occured and line
numbers of the actual hits, for quick viewing with command line tools
like less or vi.  Had there been more instances of the regexp in one
file they would be listed too.

If more files contain the hits then they are listed in the same way
with a separating `-- '.
	
	Regular expressions used:
	Header1 = ^Subject: 
	Header2 = ^From: 
	Body    = marking.*M-&
	Searched:    60 files 4803 lines
	under directory /home/reader/Mail/prinb
	  Finish = Feb 17 16:16:47 2001
	  Start  = Feb 17 16:16:47 2001


A final `stats' output (above) is displayed that may be of interest or
helpfull in adjusting the regexps.

I sometimes use a filter which writes copies of the files with hits to
a spool style file that gnus can access with nndoc (or mutt with `-f'
flag)  If the search seems to merit it.

   http://www.ptw.com/~reader/exp/download/nnml-nntp2mbox.sh

A very crude way to allow gnus tools to come into the foray.

I use the filter (nnml-nntp2mbox.sh) on the command line like:

   nnml-nntp2mbox.sh `search2.sh -b  "marking.*M-&" -d ~/Mail/prinb\
   |egrep '^/[^/]+/'` >FILE

Adjust the egrep regexp as needed.  This should generate a spool
(mbox) file that gnus can open with nndoc or mutt with `mutt -f FILE'.

NOTE -- about shabby filter 
No guarantee that the filter will handle all nntp messages.  Some that
are produced by list to news gateways have abnormal first lines.

I would like to get some feed back from an experienced person if you
are willing to try this home made script out.  It should work OK on a
linux box with `gawk' installed but not sure about other OSs with
different  `awks' (nawk mawk tawk etc etc)


NOTE:  A slick awk (manipulating ARGV )technique supplied by Arnold
       Robbins (of awk fame) allows just about any number of
       files to be searched without resorting to xargs.

That does not apply to the `filter' which passes the files on the
command line via "$@"  which can run into the `too many args' problem
but only if there are really a lot of hits.

Please try this out and give feed back if you have time.  At some
point after I get a newer better search.sh written I want to integrate
this into nnir as one of the engines.