A small patch for nnir.el

A small patch for nnir.el

[Mark Triggs]

Hi all,

I've written my own little Lucene-based mail indexer to index my Gnus
mail, and have hooked this into nnir.el.  It's been working well for a
while, but I've always been a little disappointed that 90% of the time
my search results take to come back is Gnus hitting the disk to grab
mail headers--especially when I've got those headers available in my
search indexes anyway.

To get around this, I modified nnir.el to add a hook that gets called at
the point of fetching NOV header data.  If this hook is non-nil, it is
called for each search result hit and is expected to return the
necessary headers in NOV format.  This allowed me to use the headers
returned by my indexing engine, and makes rendering the results summary
very fast, even for thousands of results.

I'm writing here because I'm a bit unsure who I should offer my (tiny)
nnir.el patch to.  The ding list seems to have fallen to the spammers,
and Kai Großjohann said that, although my patch looked sensible, he's no
longer the primary maintainer of nnir.el.

Can anyone point me in the right direction?

Thanks!

Mark

-- 
Mark Triggs
<mst@dishevelled.net>

Re: A small patch for nnir.el

[Ted Zlatanov]

On Thu, 18 Mar 2010 14:04:26 +1100 Mark Triggs <mst@dishevelled.net> wrote: 

MT> I'm writing here because I'm a bit unsure who I should offer my (tiny)
MT> nnir.el patch to.  The ding list seems to have fallen to the spammers,
MT> and Kai Großjohann said that, although my patch looked sensible, he's no
MT> longer the primary maintainer of nnir.el.

Send it to the ding list or post here.  I'll take a look.

You need to have signed papers if your patch is non-trivial.

Ted

Re: A small patch for nnir.el

[Mark Triggs]

[-- Attachment #1: Type: text/plain, Size: 955 bytes --]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 18 Mar 2010 14:04:26 +1100 Mark Triggs <mst@dishevelled.net> wrote: 
>
> MT> I'm writing here because I'm a bit unsure who I should offer my (tiny)
> MT> nnir.el patch to.  The ding list seems to have fallen to the spammers,
> MT> and Kai Großjohann said that, although my patch looked sensible, he's no
> MT> longer the primary maintainer of nnir.el.
>
> Send it to the ding list or post here.  I'll take a look.
>
> You need to have signed papers if your patch is non-trivial.


Thanks Ted.  I've attached my patch, which must be at least close to
trivial--it looks bigger than it really is because wrapping the old code
in an "unless" has changed the indentation.

I've signed FSF copyright papers before, but never for Gnus (I think my
last one was against ERC).  I'm happy to sign and submit whatever wants
signing and submitting, though.

Many thanks,

Mark

-- 
Mark Triggs
<mst@dishevelled.net>


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: nnir.patch --]
[-- Type: text/x-patch, Size: 2545 bytes --]

--- /home/mst/media/mark/src/cvs/emacs/lisp/gnus/nnir.el	2010-03-12 08:45:24.000000000 +1100
+++ nnir.el	2010-03-12 11:34:07.000000000 +1100
@@ -358,6 +358,14 @@
 (defvar nnir-imap-search-argument-history ()
   "The history for querying search options in nnir")
 
+(defvar nnir-get-article-nov-function nil
+  "If non-nil, a function that will be passed each search result.  This
+should return a message's headers in NOV format.
+
+If this variable is nil, or if the provided function returns nil for a search
+result, `gnus-retrieve-headers' will be called instead.")
+
+
 ;;; Developer Extension Variable:
 
 (defvar nnir-engines
@@ -779,25 +787,29 @@
 	(nnir-possibly-change-server server)
         (let ((gnus-override-method
 	       (gnus-server-to-method server)))
-	  (case (setq foo (gnus-retrieve-headers (list artno) artfullgroup nil))
-	    (nov
-	     (goto-char (point-min))
-	     (setq novitem (nnheader-parse-nov))
-	     (unless novitem
-	       (pop-to-buffer nntp-server-buffer)
-	       (error
-		"nnheader-parse-nov returned nil for article %s in group %s"
-		artno artfullgroup)))
-	    (headers
-	     (goto-char (point-min))
-	     (setq novitem (nnheader-parse-head))
-	     (unless novitem
-	       (pop-to-buffer nntp-server-buffer)
-	       (error
-		"nnheader-parse-head returned nil for article %s in group %s"
-		artno artfullgroup)))
-	    (t (error "Unknown header type %s while requesting article %s of group %s"
-		      foo artno artfullgroup))))
+          (setq novitem (and nnir-get-article-nov-function
+                             (funcall nnir-get-article-nov-function
+                                      artitem)))
+          (unless novitem
+	    (case (setq foo (gnus-retrieve-headers (list artno) artfullgroup nil))
+	      (nov
+	       (goto-char (point-min))
+	       (setq novitem (nnheader-parse-nov))
+	       (unless novitem
+		 (pop-to-buffer nntp-server-buffer)
+		 (error
+		  "nnheader-parse-nov returned nil for article %s in group %s"
+		  artno artfullgroup)))
+	      (headers
+	       (goto-char (point-min))
+	       (setq novitem (nnheader-parse-head))
+	       (unless novitem
+		 (pop-to-buffer nntp-server-buffer)
+		 (error
+		  "nnheader-parse-head returned nil for article %s in group %s"
+		  artno artfullgroup)))
+	      (t (error "Unknown header type %s while requesting article %s of group %s"
+			foo artno artfullgroup)))))
 	;; replace article number in original group with article number
         ;; in nnir group
         (mail-header-set-number novitem art)

[-- Attachment #3: Type: text/plain, Size: 161 bytes --]

_______________________________________________
info-gnus-english mailing list
info-gnus-english@gnu.org
http://lists.gnu.org/mailman/listinfo/info-gnus-english

Re: A small patch for nnir.el

[Ted Zlatanov]

On Sat, 20 Mar 2010 09:09:29 +1100 Mark Triggs <mst@dishevelled.net> wrote: 

MT> Ted Zlatanov <tzz@lifelogs.com> writes:
>> On Thu, 18 Mar 2010 14:04:26 +1100 Mark Triggs <mst@dishevelled.net> wrote: 
>> 
MT> I'm writing here because I'm a bit unsure who I should offer my (tiny)
MT> nnir.el patch to.  The ding list seems to have fallen to the spammers,
MT> and Kai Großjohann said that, although my patch looked sensible, he's no
MT> longer the primary maintainer of nnir.el.
>> 
>> Send it to the ding list or post here.  I'll take a look.
>> 
>> You need to have signed papers if your patch is non-trivial.

MT> Thanks Ted.  I've attached my patch, which must be at least close to
MT> trivial--it looks bigger than it really is because wrapping the old code
MT> in an "unless" has changed the indentation.

MT> I've signed FSF copyright papers before, but never for Gnus (I think my
MT> last one was against ERC).  I'm happy to sign and submit whatever wants
MT> signing and submitting, though.

Comitted.  I renamed the function with "-override-" so it's clear it
overrides the normal flow and used an if with comments instead of the
unless.  The changes are still tiny.  Can you test and write back if
it works for you or not?

Thanks
Ted

Re: A small patch for nnir.el

[Mark Triggs]

Ted Zlatanov <tzz@lifelogs.com> writes:

> Comitted.  I renamed the function with "-override-" so it's clear it
> overrides the normal flow and used an if with comments instead of the
> unless.  The changes are still tiny.  Can you test and write back if
> it works for you or not?

Many thanks, Ted; that's working fine for me.  Just for reference,
here's what I'm doing in my own code:

  http://github.com/marktriggs/mailindex/raw/master/contrib/mailindex.el

Thanks again,

Mark

-- 
Mark Triggs
<mst@dishevelled.net>

Re: A small patch for nnir.el

[Ted Zlatanov]

On Sun, 21 Mar 2010 08:01:32 +1100 Mark Triggs <mst@dishevelled.net> wrote: 

MT> Ted Zlatanov <tzz@lifelogs.com> writes:
>> Comitted.  I renamed the function with "-override-" so it's clear it
>> overrides the normal flow and used an if with comments instead of the
>> unless.  The changes are still tiny.  Can you test and write back if
>> it works for you or not?

MT> Many thanks, Ted; that's working fine for me.  Just for reference,
MT> here's what I'm doing in my own code:

MT>   http://github.com/marktriggs/mailindex/raw/master/contrib/mailindex.el

I'm pushing for moving Gnus to Git (discussed on the ding mailing list
over the last few months).  When that happens collaboration through Git
repos will be much easier.

Specifically for mailindex.el, you could contribute it to Gnus if you
like and the license allows it.  I hope there is some documentation too,
I didn't examine the whole repo but that file doesn't have any.

Thanks
Ted