Re: gnus-search-engine set to gnus-search-notmuch and refer threads

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> jao <jao@gnu.org> writes:
>
>> On Thu, Dec 30 2021, Eric Abrahamsen wrote:
>>
>>
>> [...]
>>
>>> Okay, here goes the next try. A few things to note:
>>>
>>> - I realized notmuch already has a "thread:{<sub-query>}" syntax that
>>>   does the double search I was doing in elisp, so now we just use that instead.
>>
>> makes sense.
>>
>>> - In all my testing I couldn't see that having "duplicate=1" on thread
>>>   searches causes any problems, so I've taken it off. Can you please
>>>   doublecheck this? If it's still mucking it up for you, I'll put it
>>>   back in. I wish I really understood what the problem is (I think it
>>>   has to do with notmuch potentially storing the same message in
>>>   multiple locations, using symlinks).
>>
>> hmm, are you sure you've removed it? i can see, after applying your
>> diff, at least for files searches:
>>
>>     (cl-defmethod gnus-search-indexed-search-command ((engine gnus-search-notmuch)
>>     						  (qstring string)
>>     						  query &optional _groups)
>>       ;; Theoretically we could use the GROUPS parameter to pass a
>>       ;; --folder switch to notmuch, but I'm not confident of getting the
>>       ;; format right.
>>       (let ((limit (alist-get 'limit query)))
>>         (with-slots (switches config-file) engine
>>           (append
>>            (list (format "--config=%s" config-file)
>>                  "search"
>>                  "--output=files"
>>                  "--duplicate=1")
>>            (when limit (list (format "--limit=%d" limit)))
>>            switches
>>            (list qstring)))))
>>
>> at any rate, i had already tried searches without it in my patched
>> version and haven't seen any adverse effects.  my understanding is that
>> notmuch is clever enough to detect duplicate messages with different
>> filenames .
>>
>>
>>> - The search result filtration now won't filter on group names if the
>>>   search is a thread search. This should resolve the issue you were
>>>   seeing where "A T" would only search within the group you had searched
>>>   in to begin with. I guess I think that an explicit thread search by
>>>   the user should result in a full scan of the server. We can see if
>>>   that surprises/annoys anyone, though.
>>
>> the behaviour for me is the same as with my previous patch: A T stays in
>> the nnselect group.  a thing to notice is that, in general, there is no
>> single "the group you had searched in to begin width" (pretty often, i
>> do searches accross all my nnml groups, of which i have plenty)... a
>> full scan of the server is, i think, precisely what a notmuch user would
>> expect :) (but i don't know if this is really supposed to work for
>> gnus-search: in general, collecting all messages of a thread will return
>> messages from a list of different gnus groups: should we be able to show
>> all of them in an ephemeral group then?).
>>
>> be it as it may, even with the full original thread belonging to a
>> single nnml group, A T is leaving me in nnselect with only the messages
>> that were already there (i.e., it's not equivalent to A W followed by A
>> T... but then again, maybe it's not supposed to be?)
>
> I can't believe how long this is taking me...
>
> I was confusing myself because there are two separate problems: notmuch
> thread searching was simply broken, and referring a thread from within
> an nnselect group only finds messages already in that select group.
>
> I've pushed a patch that should simply get thread searching into a
> working state (notmuch's "thread{}" syntax turned out to be more
> complicated and less useful than I thought, so I dropped it).
>
> Next, I think it's a reasonable design decision to say that referring a
> thread from within an nnselect group should search that group's
> constituent groups, not the group itself. What I'm seeing is that
> nnselect actually does that (if we're using search for thread-referral,
> it nixes out the group argument and searches the whole original
> server(s)), but then an error is raised later on, when we try to fetch
> the headers (line 656). I can see that all the thread article numbers
> are returned correctly, but then when we get to `gnus-fetch-headers' I
> get an args-out-of-range error, it looks like because we're looking for
> the index of an article number in the original (smaller) list of
> newsgroup headers. This was referring a thread from a summary buffer
> that contained only a single message.
>
> Andy, does that sound familiar?
>
> One other bug I'm not sure what to do with: notmuch only accepts
> message-ids as search terms *without* the angle brackets. If the user is
> using unparsed (raw) queries, presumably they know not to include angle
> brackets. If they're using parsed queries, the parsing process strips
> any brackets out. But if they're using raw queries *and* refer threads,
> nnselect passes in the thread search query with the angle brackets
> (reasonable, since that's how `mail-header-id' and friends return them).
>
> But that causes failure in the subsequent notmuch search. I don't want
> to special-case this, but on the other hand if all other search engines
> are also able to handle the no-brackets case, maybe we could just always
> strip the brackets?

Okay, after some hairy off-list debugging, I believe everything should
be sorted and working okay. Jose, next time you update Emacs, would you
give it a whirl?

Eric