new nnir features

new nnir features

[Andrew Cohen]

Over a year ago I made changes to nnir that I intended to push but never
got around to it. But things have been quiet so here goes. Note that
this is a major re-write and it is quite likely that there are bugs.
I've been using this new version for all this time without incident,
however. 


User-visible changes:

1. New function 'gnus-summary-make-nnir-group' allows searching from the
   summary buffer (someone requested this on the mailing list). I
   couldn't come up with a good key-binding so its unbound by
   default.
2. nnir summary buffers now play nice with each other so you can have
   multiple search results simultaneously. 
3. Marks should now be correct in nnir summary buffers. 
4. Rudimentary support for "permanent" search groups (i.e., not
   ephemeral). These are groups that end up in the .newsrc and will show
   up in the group buffer. The search result is not updated by default
   (since it might be slow depending on the search) but can be updated
   by M-g while point is on the group line. 


These changes have led me to modify the way nnir summary buffers are
treated in the window configuration. An nnir summary buffer now sticks
around until you explicitly exit from it (e.g. if you warp to an article
the nnir summary buffer gets buried but not deleted). 

Best,
Andy

Re: new nnir features

[Katsumi Yamaoka]

[-- Attachment #1: Type: text/plain, Size: 522 bytes --]

Andrew Cohen wrote:
> Note that this is a major re-write and it is quite likely that
> there are bugs.

Could you provide the ChangeLog entry?  In addition, could you
confirm whether it was right that I've merged your changes to
the Emacs trunk?  Before and after this merging, the difference
in nnir.el between the Emacs trunk and the Gnus master remains
the diff below.  Please note that nnir.el between them pretty
differs because of the pending changes that have made in the
Gnus master.

Thanks in advance.
Regards,


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: Type: text/x-patch, Size: 544 bytes --]

--- Gnus-master/nnir.el	2013-03-25 21:54:59.677016600 +0000
+++ Emacs-trunk/nnir.el	2013-03-25 22:12:02.455811300 +0000
@@ -290,3 +290,3 @@
   (autoload 'nnimap-command "nnimap")
-  (autoload 'nnimap-change-group "nnimap")
+  (autoload 'nnimap-possibly-change-group "nnimap")
   (autoload 'nnimap-make-thread-query "nnimap")
@@ -955,3 +955,3 @@
               (condition-case ()
-                  (when (nnimap-change-group
+                  (when (nnimap-possibly-change-group
                          (gnus-group-short-name group) server)

Re: new nnir features

[Katsumi Yamaoka]

Katsumi Yamaoka wrote:
> Please note that nnir.el between them pretty
                   ^^^^^^^
Oops, it should be: nnimap.el

> differs because of the pending changes that have made in the
> Gnus master.

Re: new nnir features

[Erik Colson]

Hi,

I installed gnus latest commits (including your nnir).
Works great here on MacOSX Emacs 24.3 (from homebrew).

Thanks for the features !
--
erik

Re: new nnir features

[Eric Abrahamsen]

Andrew Cohen <cohen@bu.edu> writes:

> Over a year ago I made changes to nnir that I intended to push but never
> got around to it. But things have been quiet so here goes. Note that
> this is a major re-write and it is quite likely that there are bugs.
> I've been using this new version for all this time without incident,
> however. 

Thanks for this! Permanent search groups in particular are very welcome.
I'm seeing some odd behavior, though, which I think must be related to
this patch. I've got my servers set up this way:

(setq gnus-select-method '(nntp "news.gmane.org"))
(setq gnus-secondary-select-methods '((nnml "" (nnir-search-engine namazu))))

It always worked fine: "G G" would behave differently depending on the
group under point. I'd get namazu for nnml groups, and whatever the
default was for nntp groups.

(Come to think of it, it didn't match the docs in one respect: they say
that putting point over an nnml group would *only* search that group,
but it always searched all groups.)

Anyway, searching nntp groups now gives me this error:

Debugger entered--Lisp error: (error "Couldn't request group nnir:nnir-877gksuul4.fsf: Search produced empty results.")
  signal(error ("Couldn't request group nnir:nnir-877gksuul4.fsf: Search produced empty results."))
  error("Couldn't request group %s: %s" "nnir:nnir-877gksuul4.fsf" "Search produced empty results.")
  gnus-select-newsgroup("nnir:nnir-877gksuul4.fsf" t nil)
  gnus-summary-read-group-1("nnir:nnir-877gksuul4.fsf" t t nil nil nil)
  gnus-summary-read-group("nnir:nnir-877gksuul4.fsf" t t nil nil nil nil)
  gnus-group-read-group(t t "nnir:nnir-877gksuul4.fsf" nil)
  byte-code("\b	^[\306\f\206\n\307\307\x0e\b$*\205

Re: new nnir features

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Andrew Cohen <cohen@bu.edu> writes:
>
>> Over a year ago I made changes to nnir that I intended to push but never
>> got around to it. But things have been quiet so here goes. Note that
>> this is a major re-write and it is quite likely that there are bugs.
>> I've been using this new version for all this time without incident,
>> however. 
>
> Thanks for this! Permanent search groups in particular are very welcome.
> I'm seeing some odd behavior, though, which I think must be related to
> this patch. I've got my servers set up this way:
>
> (setq gnus-select-method '(nntp "news.gmane.org"))
> (setq gnus-secondary-select-methods '((nnml "" (nnir-search-engine namazu))))
>
> It always worked fine: "G G" would behave differently depending on the
> group under point. I'd get namazu for nnml groups, and whatever the
> default was for nntp groups.
>
> (Come to think of it, it didn't match the docs in one respect: they say
> that putting point over an nnml group would *only* search that group,
> but it always searched all groups.)
>
> Anyway, searching nntp groups now gives me this error:
>
> Debugger entered--Lisp error: (error "Couldn't request group nnir:nnir-877gksuul4.fsf: Search produced empty results.")
>   signal(error ("Couldn't request group nnir:nnir-877gksuul4.fsf: Search produced empty results."))
>   error("Couldn't request group %s: %s" "nnir:nnir-877gksuul4.fsf" "Search produced empty results.")
>   gnus-select-newsgroup("nnir:nnir-877gksuul4.fsf" t nil)
>   gnus-summary-read-group-1("nnir:nnir-877gksuul4.fsf" t t nil nil nil)
>   gnus-summary-read-group("nnir:nnir-877gksuul4.fsf" t t nil nil nil nil)
>   gnus-group-read-group(t t "nnir:nnir-877gksuul4.fsf" nil)

Huh, it wasn't kidding about the control characters warning. Here's the
edited backtrace:

Debugger entered--Lisp error: (error "Couldn't request group nnir:nnir-877gksuul4.fsf: Search produced empty results.")
  signal(error ("Couldn't request group nnir:nnir-877gksuul4.fsf: Search produced empty results."))
  error("Couldn't request group %s: %s" "nnir:nnir-877gksuul4.fsf" "Search produced empty results.")
  gnus-select-newsgroup("nnir:nnir-877gksuul4.fsf" t nil)
  gnus-summary-read-group-1("nnir:nnir-877gksuul4.fsf" t t nil nil nil)
  gnus-summary-read-group("nnir:nnir-877gksuul4.fsf" t t nil nil nil nil)
  gnus-group-read-group(t t "nnir:nnir-877gksuul4.fsf" nil)
  byte-code("<DELETED>"
[gnus-large-ephemeral-newsgroup gnus-fetch-old-ephemeral-headers
gnus-fetch-old-headers gnus-large-newsgroup number group
gnus-group-read-group t select-articles] 5)
  gnus-group-read-ephemeral-group("nnir-877gksuul4.fsf" (nnir "nnir")
nil nil nil nil ((nnir-specs (nnir-query-spec (query . "archlinux"))
(nnir-group-spec ("nntp:news.gmane.org"
("gmane.comp.finance.ledger.general"))))))
  gnus-group-make-nnir-group(nil)
  call-interactively(gnus-group-make-nnir-group nil nil)

The bogus Message-ID seems to change every time. I don't believe I have
any other nnir-specific customizations. Perhaps something needs to be
set explicitly now that was previously a default?

Thanks!
Eric

Re: new nnir features

[Andrew Cohen]

>>>>> "Eric" == Eric Abrahamsen <eric@ericabrahamsen.net> writes:

    Eric> Eric Abrahamsen <eric@ericabrahamsen.net> writes:

[...]

    >> 
    >> (Come to think of it, it didn't match the docs in one respect:
    >> they say that putting point over an nnml group would *only*
    >> search that group, but it always searched all groups.)
    >> 

Yes, that's a doc bug :) Some search engines (like swish and namazu)
don't have the concept of groups and so search everything no matter
the group information passed to the search.

    >> Anyway, searching nntp groups now gives me this error:
    >> 
    >> Debugger entered--Lisp error: (error "Couldn't request group
    >> nnir:nnir-877gksuul4.fsf: Search produced empty results.")
    >> signal(error ("Couldn't request group nnir:nnir-877gksuul4.fsf:
    >> Search produced empty results."))  error("Couldn't request group
    >> %s: %s" "nnir:nnir-877gksuul4.fsf" "Search produced empty
    >> results.")  gnus-select-newsgroup("nnir:nnir-877gksuul4.fsf" t
    >> nil) gnus-summary-read-group-1("nnir:nnir-877gksuul4.fsf" t t nil
    >> nil nil) gnus-summary-read-group("nnir:nnir-877gksuul4.fsf" t t
    >> nil nil nil nil) gnus-group-read-group(t t
    >> "nnir:nnir-877gksuul4.fsf" nil)

    Eric> Huh, it wasn't kidding about the control characters
    Eric> warning. Here's the edited backtrace:

    Eric> Debugger entered--Lisp error: (error "Couldn't request group
    Eric> nnir:nnir-877gksuul4.fsf: Search produced empty results.")
    Eric> signal(error ("Couldn't request group
    Eric> nnir:nnir-877gksuul4.fsf: Search produced empty results."))
    Eric> error("Couldn't request group %s: %s"
    Eric> "nnir:nnir-877gksuul4.fsf" "Search produced empty results.")
    Eric> gnus-select-newsgroup("nnir:nnir-877gksuul4.fsf" t nil)
    Eric> gnus-summary-read-group-1("nnir:nnir-877gksuul4.fsf" t t nil
    Eric> nil nil) gnus-summary-read-group("nnir:nnir-877gksuul4.fsf" t
    Eric> t nil nil nil nil) gnus-group-read-group(t t
    Eric> "nnir:nnir-877gksuul4.fsf" nil) byte-code("<DELETED>"
    Eric> [gnus-large-ephemeral-newsgroup
    Eric> gnus-fetch-old-ephemeral-headers gnus-fetch-old-headers
    Eric> gnus-large-newsgroup number group gnus-group-read-group t
    Eric> select-articles] 5)
    Eric> gnus-group-read-ephemeral-group("nnir-877gksuul4.fsf" (nnir
    Eric> "nnir") nil nil nil nil ((nnir-specs (nnir-query-spec (query
    Eric> . "archlinux")) (nnir-group-spec ("nntp:news.gmane.org"
    Eric> ("gmane.comp.finance.ledger.general"))))))
    Eric> gnus-group-make-nnir-group(nil)
    Eric> call-interactively(gnus-group-make-nnir-group nil nil)

    Eric> The bogus Message-ID seems to change every time. I don't
    Eric> believe I have any other nnir-specific customizations. Perhaps
    Eric> something needs to be set explicitly now that was previously a
    Eric> default?

This all looks correct (except for the lack of search hits:)). I tried
reproducing this (same search query) and it works as expected. I suspect
that its not actually related to the recent nnir changes but something
else (like maybe some .elc files not up to date). 

Can you try re-byte compiling the gnus directory and restarting emacs?
If that doesn't fix it can you try tracing the function
'nnir-request-group' and see what is happening?


Regards,
Andy

Re: new nnir features

[Andrew Cohen]

>>>>> "Andy" == Andrew Cohen <cohen@bu.edu> writes:

>>>>> "Eric" == Eric Abrahamsen <eric@ericabrahamsen.net> writes:
    Eric> Eric Abrahamsen <eric@ericabrahamsen.net> writes:

    Andy> [...]

I spoke too soon---there is something fishy going on. This call

 gnus-group-read-ephemeral-group("nnir-877gksuul4.fsf" (nnir
 "nnir") nil nil nil nil ((nnir-specs (nnir-query-spec (query
 . "archlinux")) (nnir-group-spec ("nntp:news.gmane.org"
 ("gmane.comp.finance.ledger.general"))))))

is missing the element 'nnir-artlist in the parameters. That is,
it should look like 

 gnus-group-read-ephemeral-group("nnir-877gksuul4.fsf" (nnir
 "nnir") nil nil nil nil ((nnir-specs (nnir-query-spec (query
 . "archlinux")) (nnir-group-spec ("nntp:news.gmane.org"
 ("gmane.comp.finance.ledger.general")))) (nnir-artlist nil)))

I can't see how this is possible unless something is screwed up with the
byte-compiled files (the call to this function in
'gnus-group-make-nnir-group passes this parameter explicitly).


Are you using gnus-master from git?

Re: new nnir features

[Eric Abrahamsen]

Andrew Cohen <cohen@bu.edu> writes:

>>>>>> "Andy" == Andrew Cohen <cohen@bu.edu> writes:
>
>>>>>> "Eric" == Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>     Eric> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>     Andy> [...]
>
> I spoke too soon---there is something fishy going on. This call
>
>  gnus-group-read-ephemeral-group("nnir-877gksuul4.fsf" (nnir
>  "nnir") nil nil nil nil ((nnir-specs (nnir-query-spec (query
>  . "archlinux")) (nnir-group-spec ("nntp:news.gmane.org"
>  ("gmane.comp.finance.ledger.general"))))))
>
> is missing the element 'nnir-artlist in the parameters. That is,
> it should look like 
>
>  gnus-group-read-ephemeral-group("nnir-877gksuul4.fsf" (nnir
>  "nnir") nil nil nil nil ((nnir-specs (nnir-query-spec (query
>  . "archlinux")) (nnir-group-spec ("nntp:news.gmane.org"
>  ("gmane.comp.finance.ledger.general")))) (nnir-artlist nil)))
>
> I can't see how this is possible unless something is screwed up with the
> byte-compiled files (the call to this function in
> 'gnus-group-make-nnir-group passes this parameter explicitly).
>
>
> Are you using gnus-master from git?

Yup, just updated. I'm now running non-byte-compiled gnus, and am seeing
slightly different behavior. I edebugged a couple of times, and saw that
"(nnir-artlist)" (though not "(nnir-artlist nil)") was in the
"parameters" argument to `gnus-group-read-ephemeral-group', though it
still errored out. Then I removed debugging, toggled debug on error, and
got a rather enormous (I guess because it's not byte compiled)
traceback, which I pasted here:

http://paste.lisp.org/display/136379

Any help?

Eric

Re: new nnir features

[Andrew Cohen]

>>>>> "Andrew" == Andrew Cohen <cohen@bu.edu> writes:

    Andrew> Over a year ago I made changes to nnir that I intended to
    Andrew> push but never got around to it. But things have been quiet
    Andrew> so here goes. Note that this is a major re-write and it is
    Andrew> quite likely that there are bugs.  I've been using this new
    Andrew> version for all this time without incident, however.

    Andrew> User-visible changes:


[...]


    Andrew> 4. Rudimentary support for "permanent" search groups (i.e.,
    Andrew> not ephemeral). These are groups that end up in the .newsrc
    Andrew> and will show up in the group buffer. The search result is
    Andrew> not updated by default (since it might be slow depending on
    Andrew> the search) but can be updated by M-g while point is on the
    Andrew> group line.

For anyone who wants to try this out:

1. Backup your .newsrc! (It's unlikely that anything will go wrong, but
   better safe than sorry). 

2. There are two ways to make a search group (provided you are using the
   update I pushed to git a few minutes ago:))

   The HARD way: Just type "G m" (bound to 'gnus-group-make-group) in
   the group buffer. You will be prompted for the NAME you wish to use
   for the new group and a SELECT method. For the select method just
   choose "nnir". Then you should be prompted for a QUERY followed by a
   prompt for a SERVER. Better not to try anything complicated at this
   point; for example just type a single word for the query and your
   preferred server (e.g. "nnimap:home"). You can change this later by
   editing the group parameters with "G p".  The format of these things
   is described in the comments in nnir.el. 

   The EASY way: Just do whatever nnir search you like as you usually
   would. Then from the nnir summary buffer invoke the function
   'gnus-summary-create-nnir-group (not bound to any key at the
   moment). You should be prompted for the NAME you wish to use for the
   new group and that's it. The group should be created and should be
   visible somewhere in your *Group* buffer. Again, you can always
   modify the search by editing the group parameters.


Regards,
Andy

Re: new nnir features

[Erik Colson]

Andrew Cohen <cohen@bu.edu> writes:

>    The HARD way: Just type "G m" (bound to 'gnus-group-make-group) in
>    the group buffer. You will be prompted for the NAME you wish to use
>    for the new group and a SELECT method. For the select method just
>    choose "nnir". Then you should be prompted for a QUERY followed by a
>    prompt for a SERVER. Better not to try anything complicated at this
>    point; for example just type a single word for the query and your
>    preferred server (e.g. "nnimap:home"). You can change this later by
>    editing the group parameters with "G p".  The format of these things
>    is described in the comments in nnir.el. 
>
>    The EASY way: Just do whatever nnir search you like as you usually
>    would. Then from the nnir summary buffer invoke the function
>    'gnus-summary-create-nnir-group (not bound to any key at the
>    moment). You should be prompted for the NAME you wish to use for the
>    new group and that's it. The group should be created and should be
>    visible somewhere in your *Group* buffer. Again, you can always
>    modify the search by editing the group parameters.

Hi Andrew,

Thanks for the explanation! I'm used to run a query over several groups
which happen _not_ to share the same server. Would that be a problem to
save such a query ?

Thank you
--
erik

gnus-summary-mail-other-window now giving wrong-type-argument!

[jidanni]

Debugger entered--Lisp error: (wrong-type-argument stringp nil)
  string-match("^nnir:" nil)
  gnus-summary-mail-other-window(nil)
  call-interactively(gnus-summary-mail-other-window nil nil)
  command-execute(gnus-summary-mail-other-window)

emacs-snapshot:
 *** 2:20130403-1 0
        500 http://emacs.naquadah.org/ unstable/ Packages

Re: gnus-summary-mail-other-window now giving wrong-type-argument!

[Andrew Cohen]

>>>>> "jidanni" == jidanni  <jidanni@jidanni.org> writes:

    jidanni> Debugger entered--Lisp error: (wrong-type-argument stringp
    jidanni> nil) string-match("^nnir:" nil)
    jidanni> gnus-summary-mail-other-window(nil)
    jidanni> call-interactively(gnus-summary-mail-other-window nil nil)
    jidanni> command-execute(gnus-summary-mail-other-window)


This should have been fixed in Gnus-master a couple of days ago. Maybe
it hasn't yet made its way to Emacs-master?

Re: new nnir features

[Steinar Bang]

First: I really like the new features in nnir. :-)

Here's a feature request:
 When you do `A T' in an nnir search result summary buffer, to go from a
 search summary buffer to the actual group, it would be nice if `q'
 returned you to the search summary buffer.

Re: new nnir features

[Andrew Cohen]

>>>>> "Steinar" == Steinar Bang <sb@dod.no> writes:

    Steinar> First: I really like the new features in nnir. :-) Here's a
    Steinar> feature request: When you do `A T' in an nnir search result
    Steinar> summary buffer, to go from a search summary buffer to the
    Steinar> actual group, it would be nice if `q' returned you to the
    Steinar> search summary buffer.

Yes, it would, wouldn't it :) I have been unsuccessful at coercing gnus
to do this.  I'll give it another try when I get a chance. 

Re: new nnir features

[Sergio Durigan Junior]

On Thursday, March 28 2013, Eric Abrahamsen wrote:

> Yup, just updated. I'm now running non-byte-compiled gnus, and am seeing
> slightly different behavior. I edebugged a couple of times, and saw that
> "(nnir-artlist)" (though not "(nnir-artlist nil)") was in the
> "parameters" argument to `gnus-group-read-ephemeral-group', though it
> still errored out. Then I removed debugging, toggled debug on error, and
> got a rather enormous (I guess because it's not byte compiled)
> traceback, which I pasted here:
>
> http://paste.lisp.org/display/136379

I haven't been able to view the link above, but I'm wondering if this
problem is the same as I'm seeing here:

  Debugger entered--Lisp error: (wrong-number-of-arguments #[(group &optional server fast) "\306\b!\210	
  \232\203..\v\f\232\203..\b\203..\b.\232\203....\207\307	!..\212..q\210\310..G!\203?.\311\211..\311\211...\312\313\314\"\202V.	.\b\203G.\b.\v.\315\316\317..!\320\317..!	%)\207" [server group nnir-current-query gnus-group-marked nnir-current-group-marked nnir-current-server nnir-possibly-change-server nnir-run-query zerop nil nnheader-report nnir "Search produced empty results." nnheader-insert "211 %d %d %d %s
  " nnir-artlist-length 1 nnir-artlist nntp-server-buffer] 7 ("/usr/share/emacs/site-lisp/gnus-bonus/nnir.elc" . 12724)] 4)
  nnir-request-group("((query . \"test\"))" "-ephemeral" nil ("nnir:((query . \"test\"))" 3 nil nil (nnir "-ephemeral" (nnir-address "")) ((quit-config #<buffer *Group*> . group))))
  gnus-activate-group("nnir:((query . \"test\"))" scan)
  gnus-group-read-ephemeral-group("nnir:((query . \"test\"))" (nnir "") t (#<buffer *Group*> . group) nil)
  gnus-group-make-nnir-group(nil "test")
  call-interactively(gnus-group-make-nnir-group nil nil)

I configure nnir to use the imap backend.  I'm not using Gnus master here.

-- 
Sergio

Re: new nnir features

[Eric Abrahamsen]

Sergio Durigan Junior <sergiodj@riseup.net> writes:

> On Thursday, March 28 2013, Eric Abrahamsen wrote:
>
>> Yup, just updated. I'm now running non-byte-compiled gnus, and am seeing
>> slightly different behavior. I edebugged a couple of times, and saw that
>> "(nnir-artlist)" (though not "(nnir-artlist nil)") was in the
>> "parameters" argument to `gnus-group-read-ephemeral-group', though it
>> still errored out. Then I removed debugging, toggled debug on error, and
>> got a rather enormous (I guess because it's not byte compiled)
>> traceback, which I pasted here:
>>
>> http://paste.lisp.org/display/136379
>
> I haven't been able to view the link above, but I'm wondering if this
> problem is the same as I'm seeing here:

Probably not, unfortunately, mine turned out to just be a typo in the
source code. Sorry...