gnus-search: do not limit search to specific groups?

gnus-search: do not limit search to specific groups?

[Eric S Fraga]

Hello,

TL;DR: I just want the results from doing a notmuch search without
specifying groups at all, e.g. what I would get if I simply did

   notmuch search --output=files xxx

on the command line.  Is this possible?

With nnir and notmuch, I could do a search and get the results
regardless of which group they were in.  This functionality is critical
for my use of search but gnus-search (nnselect) does not seem to have an
equivalent capability?

I split emails to a significant number of groups but want all of them
searched usually.  A solution would be to search at the topic level, I
guess.  However, I tried setting gnus-search-engine in the topic but I
get the error: "No search engine defined for nnml:" when I try a search.

Thank you,
eric


-- 
Eric S Fraga via Emacs 28.0.50 & org 9.4 on Debian bullseye/sid

Re: gnus-search: do not limit search to specific groups?

[Adam Sjøgren]

Eric writes:

> TL;DR: I just want the results from doing a notmuch search without
> specifying groups at all

[...]

> A solution would be to search at the topic level, I guess.

If you search with point on a topic, I think the current behaviour is to
search all the groups in that topic.

E.g. if I press G G on my [ email ] topic, I see this in *Messages*:

  Doing notmuch query on (nnml:husk nnml:tmp nnml:bogus nnml:cbb
  nnml:from-me ...[long list of every group under the email topic
  elided]... nnml:fullrate nnml:normal)


  Best regards,

    Adam

-- 
 "Bara russin, ingen kaka"                                  Adam Sjøgren
                                                       asjo@koldfront.dk

Re: gnus-search: do not limit search to specific groups?

[Eric S Fraga]

Hi Adam.

> If you search with point on a topic, I think the current behaviour is to
> search all the groups in that topic.

Which is what I thought but it did not work for me.  Having seen that it
works for you, it turns out that the search fails completely if there
are any "strange" groups [1] within the topic.  Deleting such has fixed
this for me.

Suggestion: it might be good if gnus-search were a bit more resilient to
errors.

thank you,
eric

Footnotes:
[1] I had a group "nnml:mail.pandian", i.e. without a server name after
     the nnml.  Gnus does not complain when I visit that group, ending
     up in the same place as visiting another group
     "nnml+outlook:mail.pandian".

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.4 on Debian bullseye/sid

Re: gnus-search: do not limit search to specific groups?

[Adam Sjøgren]

Eric writes:

> Which is what I thought but it did not work for me.  Having seen that it
> works for you, it turns out that the search fails completely if there
> are any "strange" groups [1] within the topic.

Ouch.

> Suggestion: it might be good if gnus-search were a bit more resilient to
> errors.

Agreed!

> [1] I had a group "nnml:mail.pandian", i.e. without a server name after
>      the nnml.  Gnus does not complain when I visit that group, ending
>      up in the same place as visiting another group
>      "nnml+outlook:mail.pandian".

All my nnml-groups are without server name. But maybe it's only mixing
that confuses Gnus?!


  Best regards,

    Adam

-- 
 "Bara russin, ingen kaka"                                  Adam Sjøgren
                                                       asjo@koldfront.dk

Re: gnus-search: do not limit search to specific groups?

[Eric S Fraga]

On Wednesday, 25 Nov 2020 at 15:24, Adam Sjøgren wrote:
> All my nnml-groups are without server name. 

Interesting.  I had definitely been under the impression that a server
name would not be necessary for nnml groups (what does it really mean in
that context anyway?) so glad to get confirmation.

> But maybe it's only mixing that confuses Gnus?!

Well, the error output from gnus-search is:

,----
| nnselect-run: gnus-search-run-query on
| ((search-query-spec (query . test gnus search with nnml) (raw))
|  (search-group-spec
|   (nnml: nnml:mail.pandian) 
|   (nnml:outlook nnml+outlook:mail.aaa
|                 nnml+outlook:mail.bbb
|                 nnml+outlook:mail.pandian)))
| 
| gave error (error No search engine defined for nnml:)
`----

(I've reformatted for ease of reading and elided some groups etc.)

So it looks like I have to define the search engine for nnml: (without
server).  Maybe that would fix this mixed server case.

Although the problem is currently solved for me by having deleted the
server-less group, it would be nice for gnus-search to have returned the
results for the second group-spec above, maybe with a warning that not
all groups were searched?

Let's see if Eric A. picks up this thread and comments.

Thanks again,
eric

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.4 on Debian bullseye/sid

Re: gnus-search: do not limit search to specific groups?

[Adam Sjøgren]

Eric writes:

> it would be nice for gnus-search to have returned the results for the
> second group-spec above, maybe with a warning that not all groups were
> searched?

That sounds like a good way to improve the resilience of the search
functionality!


  Best regards,

    Adam

-- 
 "You make a fair point. Knowing when not to say            Adam Sjøgren
  anything is not something the lazyweb had            asjo@koldfront.dk
  historically been good at."

Re: gnus-search: do not limit search to specific groups?

[Eric Abrahamsen]

Eric S Fraga <e.fraga@ucl.ac.uk> writes:

> On Wednesday, 25 Nov 2020 at 15:24, Adam Sjøgren wrote:
>> All my nnml-groups are without server name. 
>
> Interesting.  I had definitely been under the impression that a server
> name would not be necessary for nnml groups (what does it really mean in
> that context anyway?) so glad to get confirmation.
>
>> But maybe it's only mixing that confuses Gnus?!
>
> Well, the error output from gnus-search is:
>
> ,----
> | nnselect-run: gnus-search-run-query on
> | ((search-query-spec (query . test gnus search with nnml) (raw))
> |  (search-group-spec
> |   (nnml: nnml:mail.pandian) 
> |   (nnml:outlook nnml+outlook:mail.aaa
> |                 nnml+outlook:mail.bbb
> |                 nnml+outlook:mail.pandian)))
> | 
> | gave error (error No search engine defined for nnml:)
> `----
>
> (I've reformatted for ease of reading and elided some groups etc.)
>
> So it looks like I have to define the search engine for nnml: (without
> server).  Maybe that would fix this mixed server case.

I think the behavior here will depend on how you've defined your search
engine. If you've done it in gnus-secondary-select-methods, like:

'(nnml "outlook"
  (gnus-search-engine notmuch))

Then that will work for "nnml+outlook", but not bare "nnml". If you've
defined it for all nnml servers, like:

(add-to-list 'gnus-search-default-engines '(nnml . notmuch))

It will apply to all nnml backends. But then I guess you'd have to
define your remove-prefix globally, as well.

In general, the purpose of the string label (like "outlook") is to
differentiate one installation from another: you might have another set
of nnml files somewhere else, with a separate search index, and that's
how Gnus will tell them apart.

Another way of searching "all" your messages is to go to the *Server*
buffer and search a whole server with "G". In your *Server* buffer, did
you used to see separate servers for "nnml" and "nnml+outlook"?

I'm surprised that nnir let you search all messages without restricting
to a group or set of groups. As far as I know the code for selecting the
scope of the search hasn't changed.

> Although the problem is currently solved for me by having deleted the
> server-less group, it would be nice for gnus-search to have returned the
> results for the second group-spec above, maybe with a warning that not
> all groups were searched?

I'm talking to Andy now about how best to do this: I have a local patch
that allows individual search failures without bodging the whole
process, but the demoted error warnings currently don't make it through
to where the user can see them. This is also connected to the fact that
if ephemeral group creation leads to an error, the group isn't cleaned
up correctly. Hopefully we can get all these things fixed at once, soon.

Thanks,
Eric