Re: gnus-search-engine set to gnus-search-notmuch and refer threads

[Eric Abrahamsen <eric@ericabrahamsen.net>]

jao <jao@gnu.org> writes:

> On Thu, Dec 30 2021, Eric Abrahamsen wrote:
>
>
> [...]
>
>> Okay, here goes the next try. A few things to note:
>>
>> - I realized notmuch already has a "thread:{<sub-query>}" syntax that
>>   does the double search I was doing in elisp, so now we just use that instead.
>
> makes sense.
>
>> - In all my testing I couldn't see that having "duplicate=1" on thread
>>   searches causes any problems, so I've taken it off. Can you please
>>   doublecheck this? If it's still mucking it up for you, I'll put it
>>   back in. I wish I really understood what the problem is (I think it
>>   has to do with notmuch potentially storing the same message in
>>   multiple locations, using symlinks).
>
> hmm, are you sure you've removed it? i can see, after applying your
> diff, at least for files searches:
>
>     (cl-defmethod gnus-search-indexed-search-command ((engine gnus-search-notmuch)
>     						  (qstring string)
>     						  query &optional _groups)
>       ;; Theoretically we could use the GROUPS parameter to pass a
>       ;; --folder switch to notmuch, but I'm not confident of getting the
>       ;; format right.
>       (let ((limit (alist-get 'limit query)))
>         (with-slots (switches config-file) engine
>           (append
>            (list (format "--config=%s" config-file)
>                  "search"
>                  "--output=files"
>                  "--duplicate=1")
>            (when limit (list (format "--limit=%d" limit)))
>            switches
>            (list qstring)))))
>
> at any rate, i had already tried searches without it in my patched
> version and haven't seen any adverse effects.  my understanding is that
> notmuch is clever enough to detect duplicate messages with different
> filenames .
>
>
>> - The search result filtration now won't filter on group names if the
>>   search is a thread search. This should resolve the issue you were
>>   seeing where "A T" would only search within the group you had searched
>>   in to begin with. I guess I think that an explicit thread search by
>>   the user should result in a full scan of the server. We can see if
>>   that surprises/annoys anyone, though.
>
> the behaviour for me is the same as with my previous patch: A T stays in
> the nnselect group.  a thing to notice is that, in general, there is no
> single "the group you had searched in to begin width" (pretty often, i
> do searches accross all my nnml groups, of which i have plenty)... a
> full scan of the server is, i think, precisely what a notmuch user would
> expect :) (but i don't know if this is really supposed to work for
> gnus-search: in general, collecting all messages of a thread will return
> messages from a list of different gnus groups: should we be able to show
> all of them in an ephemeral group then?).
>
> be it as it may, even with the full original thread belonging to a
> single nnml group, A T is leaving me in nnselect with only the messages
> that were already there (i.e., it's not equivalent to A W followed by A
> T... but then again, maybe it's not supposed to be?)

I can't believe how long this is taking me...

I was confusing myself because there are two separate problems: notmuch
thread searching was simply broken, and referring a thread from within
an nnselect group only finds messages already in that select group.

I've pushed a patch that should simply get thread searching into a
working state (notmuch's "thread{}" syntax turned out to be more
complicated and less useful than I thought, so I dropped it).

Next, I think it's a reasonable design decision to say that referring a
thread from within an nnselect group should search that group's
constituent groups, not the group itself. What I'm seeing is that
nnselect actually does that (if we're using search for thread-referral,
it nixes out the group argument and searches the whole original
server(s)), but then an error is raised later on, when we try to fetch
the headers (line 656). I can see that all the thread article numbers
are returned correctly, but then when we get to `gnus-fetch-headers' I
get an args-out-of-range error, it looks like because we're looking for
the index of an article number in the original (smaller) list of
newsgroup headers. This was referring a thread from a summary buffer
that contained only a single message.

Andy, does that sound familiar?

One other bug I'm not sure what to do with: notmuch only accepts
message-ids as search terms *without* the angle brackets. If the user is
using unparsed (raw) queries, presumably they know not to include angle
brackets. If they're using parsed queries, the parsing process strips
any brackets out. But if they're using raw queries *and* refer threads,
nnselect passes in the thread search query with the angle brackets
(reasonable, since that's how `mail-header-id' and friends return them).

But that causes failure in the subsequent notmuch search. I don't want
to special-case this, but on the other hand if all other search engines
are also able to handle the no-brackets case, maybe we could just always
strip the brackets?

Eric