Gnus development mailing list
 help / color / mirror / Atom feed
* search for mark over Groups topic not working
@ 2021-05-17  2:31 Jose A. Ortega Ruiz
  2021-06-03 18:39 ` Eric Abrahamsen
  0 siblings, 1 reply; 12+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-05-17  2:31 UTC (permalink / raw)
  To: ding


Hi,

I'm using the latest gnus-search in emacs master, with groups in nnimap
over several mailboxes and also notmuch folders, distributed among
several topics (all of them under [Gnus], no deeper nesting).  Searching
for marks on those topics (e.g. mark:!), works fine.  But when i go to
the [Gnus] parent topic and repeat the search, it always comes back
empty.  I see traces of all subgroups being searched, but the nnselect
group comes back empty.  Other searches (say, over the body of the
messages) work fine (i.e., if the search finds mails on topic [Foo]
under [Gnus], it also finds those same messages when the search is
performed on [Gnus]).

Is anybody experiencing this problem?

Thanks a lot,
jao
-- 
"I didn't do it, and I'll never do it again."
 -Derrik Weeks



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-05-17  2:31 search for mark over Groups topic not working Jose A. Ortega Ruiz
@ 2021-06-03 18:39 ` Eric Abrahamsen
  2021-06-03 19:39   ` Jose A. Ortega Ruiz
  0 siblings, 1 reply; 12+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 18:39 UTC (permalink / raw)
  To: ding

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> Hi,
>
> I'm using the latest gnus-search in emacs master, with groups in nnimap
> over several mailboxes and also notmuch folders, distributed among
> several topics (all of them under [Gnus], no deeper nesting).  Searching
> for marks on those topics (e.g. mark:!), works fine.  But when i go to
> the [Gnus] parent topic and repeat the search, it always comes back
> empty.  I see traces of all subgroups being searched, but the nnselect
> group comes back empty.  Other searches (say, over the body of the
> messages) work fine (i.e., if the search finds mails on topic [Foo]
> under [Gnus], it also finds those same messages when the search is
> performed on [Gnus]).
>
> Is anybody experiencing this problem?

Sorry, I missed this when you first posted it.

That's pretty weird! So to make sure I've got this right: the search
only fails when you search the [Gnus] topic, and only when searching for
marks?

When a search returns no results, it probably means that there was an
error in the search routine somewhere. If you're running a very recent
Emacs, you can M-x toggle-debug-on-error to see the full backtrace. If
your Emacs isn't super recent (like the past few weeks), the error
should still show up in *Messages*, search for "nnselect-run:".

Eric



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-03 18:39 ` Eric Abrahamsen
@ 2021-06-03 19:39   ` Jose A. Ortega Ruiz
  2021-06-03 21:27     ` Eric Abrahamsen
  0 siblings, 1 reply; 12+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-06-03 19:39 UTC (permalink / raw)
  To: ding

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>
>> Hi,
>>
>> I'm using the latest gnus-search in emacs master, with groups in nnimap
>> over several mailboxes and also notmuch folders, distributed among
>> several topics (all of them under [Gnus], no deeper nesting).  Searching
>> for marks on those topics (e.g. mark:!), works fine.  But when i go to
>> the [Gnus] parent topic and repeat the search, it always comes back
>> empty.  I see traces of all subgroups being searched, but the nnselect
>> group comes back empty.  Other searches (say, over the body of the
>> messages) work fine (i.e., if the search finds mails on topic [Foo]
>> under [Gnus], it also finds those same messages when the search is
>> performed on [Gnus]).
>>
>> Is anybody experiencing this problem?
>
> Sorry, I missed this when you first posted it.
>
> That's pretty weird! So to make sure I've got this right: the search
> only fails when you search the [Gnus] topic,

right

> and only when searching for marks?

i cannot say categorically that it's /only/ for marks, but it definitely
happens with marks.  i have an emacs compiled yesterday.

the logs in message show no error for the groups containing marked
articles.  only for the nndraft:drafts group:

  Search engine for nndraft: improperly configured: No search engine configured for nndraft:

(as far as i can tell, nndraft is an "automatic" server, i haven't added
it explicitly in my config, so i'm no sure what's the proper way of
configuring its search engine.)

and i also get one of those messages for the nnselect server (which i
guess it's a little bug, in that those groups cannot have a search
engine, can they?)

  Search engine for nnselect: improperly configured: No search engine configured for nnselect:

my expectation is that those errors shouldn't affect the collection of
results, right? all the other groups are either nnimap or nntp (with
notmuch as the search engine).

cheers,
jao



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-03 19:39   ` Jose A. Ortega Ruiz
@ 2021-06-03 21:27     ` Eric Abrahamsen
  2021-06-03 21:36       ` Eric Abrahamsen
  0 siblings, 1 reply; 12+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 21:27 UTC (permalink / raw)
  To: ding

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> On Thu, Jun 03 2021, Eric Abrahamsen wrote:
>
>> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>>
>>> Hi,
>>>
>>> I'm using the latest gnus-search in emacs master, with groups in nnimap
>>> over several mailboxes and also notmuch folders, distributed among
>>> several topics (all of them under [Gnus], no deeper nesting).  Searching
>>> for marks on those topics (e.g. mark:!), works fine.  But when i go to
>>> the [Gnus] parent topic and repeat the search, it always comes back
>>> empty.  I see traces of all subgroups being searched, but the nnselect
>>> group comes back empty.  Other searches (say, over the body of the
>>> messages) work fine (i.e., if the search finds mails on topic [Foo]
>>> under [Gnus], it also finds those same messages when the search is
>>> performed on [Gnus]).
>>>
>>> Is anybody experiencing this problem?
>>
>> Sorry, I missed this when you first posted it.
>>
>> That's pretty weird! So to make sure I've got this right: the search
>> only fails when you search the [Gnus] topic,
>
> right
>
>> and only when searching for marks?
>
> i cannot say categorically that it's /only/ for marks, but it definitely
> happens with marks.  i have an emacs compiled yesterday.
>
> the logs in message show no error for the groups containing marked
> articles.  only for the nndraft:drafts group:
>
>   Search engine for nndraft: improperly configured: No search engine configured for nndraft:
>
> (as far as i can tell, nndraft is an "automatic" server, i haven't added
> it explicitly in my config, so i'm no sure what's the proper way of
> configuring its search engine.)
>
> and i also get one of those messages for the nnselect server (which i
> guess it's a little bug, in that those groups cannot have a search
> engine, can they?)
>
>   Search engine for nnselect: improperly configured: No search engine configured for nnselect:
>
> my expectation is that those errors shouldn't affect the collection of
> results, right? all the other groups are either nnimap or nntp (with
> notmuch as the search engine).

You're right, and I just realized that the code I put in place to handle
this exact situation (if you're searching across multiple backends,
search failure on one backend shouldn't prevent other backends from
returning their results) is brain-dead: it's a condition-case, and even
if we handle the error that doesn't mean that the code will continue
executing. O for Common Lisp's restarts! Give me a little bit and I'll
rework the code to do what it's supposed to do.

Thanks for the report,
Eric



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-03 21:27     ` Eric Abrahamsen
@ 2021-06-03 21:36       ` Eric Abrahamsen
  2021-06-03 23:41         ` Jose A. Ortega Ruiz
                           ` (2 more replies)
  0 siblings, 3 replies; 12+ messages in thread
From: Eric Abrahamsen @ 2021-06-03 21:36 UTC (permalink / raw)
  To: ding

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>
>> On Thu, Jun 03 2021, Eric Abrahamsen wrote:
>>
>>> "Jose A. Ortega Ruiz" <jao@gnu.org> writes:
>>>
>>>> Hi,
>>>>
>>>> I'm using the latest gnus-search in emacs master, with groups in nnimap
>>>> over several mailboxes and also notmuch folders, distributed among
>>>> several topics (all of them under [Gnus], no deeper nesting).  Searching
>>>> for marks on those topics (e.g. mark:!), works fine.  But when i go to
>>>> the [Gnus] parent topic and repeat the search, it always comes back
>>>> empty.  I see traces of all subgroups being searched, but the nnselect
>>>> group comes back empty.  Other searches (say, over the body of the
>>>> messages) work fine (i.e., if the search finds mails on topic [Foo]
>>>> under [Gnus], it also finds those same messages when the search is
>>>> performed on [Gnus]).
>>>>
>>>> Is anybody experiencing this problem?
>>>
>>> Sorry, I missed this when you first posted it.
>>>
>>> That's pretty weird! So to make sure I've got this right: the search
>>> only fails when you search the [Gnus] topic,
>>
>> right
>>
>>> and only when searching for marks?
>>
>> i cannot say categorically that it's /only/ for marks, but it definitely
>> happens with marks.  i have an emacs compiled yesterday.
>>
>> the logs in message show no error for the groups containing marked
>> articles.  only for the nndraft:drafts group:
>>
>>   Search engine for nndraft: improperly configured: No search engine configured for nndraft:
>>
>> (as far as i can tell, nndraft is an "automatic" server, i haven't added
>> it explicitly in my config, so i'm no sure what's the proper way of
>> configuring its search engine.)
>>
>> and i also get one of those messages for the nnselect server (which i
>> guess it's a little bug, in that those groups cannot have a search
>> engine, can they?)
>>
>>   Search engine for nnselect: improperly configured: No search engine configured for nnselect:
>>
>> my expectation is that those errors shouldn't affect the collection of
>> results, right? all the other groups are either nnimap or nntp (with
>> notmuch as the search engine).
>
> You're right, and I just realized that the code I put in place to handle
> this exact situation (if you're searching across multiple backends,
> search failure on one backend shouldn't prevent other backends from
> returning their results) is brain-dead: it's a condition-case, and even
> if we handle the error that doesn't mean that the code will continue
> executing.

Oh wait, I was wrong, that's exactly what it should do. Would you edebug
`gnus-search-run-query', step through it, and see what's going on?


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-03 21:36       ` Eric Abrahamsen
@ 2021-06-03 23:41         ` Jose A. Ortega Ruiz
  2021-06-03 23:50         ` Jose A. Ortega Ruiz
  2021-06-04  0:14         ` Jose A. Ortega Ruiz
  2 siblings, 0 replies; 12+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-06-03 23:41 UTC (permalink / raw)
  To: ding

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

[...]

> Oh wait, I was wrong, that's exactly what it should do. Would you edebug
> `gnus-search-run-query', step through it, and see what's going on?

i'm very bad at using debuggers, and keep forgetting how to use edebug,
but will try! i take it that it's working fine for you?



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-03 21:36       ` Eric Abrahamsen
  2021-06-03 23:41         ` Jose A. Ortega Ruiz
@ 2021-06-03 23:50         ` Jose A. Ortega Ruiz
  2021-06-04  0:14         ` Jose A. Ortega Ruiz
  2 siblings, 0 replies; 12+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-06-03 23:50 UTC (permalink / raw)
  To: ding

On Thu, Jun 03 2021, Eric Abrahamsen wrote:

> Oh wait, I was wrong, that's exactly what it should do. Would you edebug
> `gnus-search-run-query', step through it, and see what's going on?

i don't think the problem has anything to do with the failing groups.
when the search is performed at the [Gnus] level, one enters the
corresponding spec in gnus-search-run-query's mapc, but the individual
group inside the server (nnimap in my case) don't find any results (i've
found out with my favourite debugging too, message).

so what's going on seems to be that a query that says, for instance

   Searching nnimap:feeds/emacs...done

finds results when it's performed in a subtopic of gnus but not when
it's performed in Gnus.  let me explore a little more...



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-03 21:36       ` Eric Abrahamsen
  2021-06-03 23:41         ` Jose A. Ortega Ruiz
  2021-06-03 23:50         ` Jose A. Ortega Ruiz
@ 2021-06-04  0:14         ` Jose A. Ortega Ruiz
  2021-06-04 16:58           ` Eric Abrahamsen
  2021-06-04 19:27           ` Eric Abrahamsen
  2 siblings, 2 replies; 12+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-06-04  0:14 UTC (permalink / raw)
  To: ding


so i don't know why it happens, but the problem seems to be in the
gnus-search-run-search method for imap servers.  specifically, this
snippet translating the query spec to a query string:

      (setq q-string
	    (gnus-search-make-query-string engine query))


- when i run the query mark:! with point at a subtopic, this code in that
  method sets q-string to "FLAGGED", and things work.
- when i run the query mark:! with point at Gnus, the code above sets
  q-string to "KEYWORD flag" (and, naturally, no results are found).

the reason for that is that, in the first case, query is

  ((parsed-query (mark . flag)) (query . mark:!) (raw))

while, in the second case, it is
   
  ((parsed-query (keyword . flag)) (query . mark:!) (raw))

so it seems that the reader of the query, or its translator to a sexp
down the line, is sensitive to where in the Groups buffer point is.



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-04  0:14         ` Jose A. Ortega Ruiz
@ 2021-06-04 16:58           ` Eric Abrahamsen
  2021-06-04 19:27           ` Eric Abrahamsen
  1 sibling, 0 replies; 12+ messages in thread
From: Eric Abrahamsen @ 2021-06-04 16:58 UTC (permalink / raw)
  To: ding

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> so i don't know why it happens, but the problem seems to be in the
> gnus-search-run-search method for imap servers.  specifically, this
> snippet translating the query spec to a query string:
>
>       (setq q-string
> 	    (gnus-search-make-query-string engine query))
>
>
> - when i run the query mark:! with point at a subtopic, this code in that
>   method sets q-string to "FLAGGED", and things work.
> - when i run the query mark:! with point at Gnus, the code above sets
>   q-string to "KEYWORD flag" (and, naturally, no results are found).
>
> the reason for that is that, in the first case, query is
>
>   ((parsed-query (mark . flag)) (query . mark:!) (raw))
>
> while, in the second case, it is
>    
>   ((parsed-query (keyword . flag)) (query . mark:!) (raw))
>
> so it seems that the reader of the query, or its translator to a sexp
> down the line, is sensitive to where in the Groups buffer point is.

Whoa, that really shouldn't be the case! Thanks for doing all this
digging, I will have a look this weekend and figure out why it would be
doing that.



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-04  0:14         ` Jose A. Ortega Ruiz
  2021-06-04 16:58           ` Eric Abrahamsen
@ 2021-06-04 19:27           ` Eric Abrahamsen
  2021-06-30 18:53             ` Jose A. Ortega Ruiz
  1 sibling, 1 reply; 12+ messages in thread
From: Eric Abrahamsen @ 2021-06-04 19:27 UTC (permalink / raw)
  To: ding

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> so i don't know why it happens, but the problem seems to be in the
> gnus-search-run-search method for imap servers.  specifically, this
> snippet translating the query spec to a query string:
>
>       (setq q-string
> 	    (gnus-search-make-query-string engine query))
>
>
> - when i run the query mark:! with point at a subtopic, this code in that
>   method sets q-string to "FLAGGED", and things work.
> - when i run the query mark:! with point at Gnus, the code above sets
>   q-string to "KEYWORD flag" (and, naturally, no results are found).
>
> the reason for that is that, in the first case, query is
>
>   ((parsed-query (mark . flag)) (query . mark:!) (raw))
>
> while, in the second case, it is
>    
>   ((parsed-query (keyword . flag)) (query . mark:!) (raw))
>
> so it seems that the reader of the query, or its translator to a sexp
> down the line, is sensitive to where in the Groups buffer point is.

This is melting my brain. I can't reproduce this, and don't see how it's
even possible. It seems to be the query parsing process that's going
wrong, but that happens before the code even looks at which groups are
being searched, and parsing simply happens to a string: it doesn't vary
depending on the servers being searched, much less on where point is
when you start the search.

Neither does there seem to be any code that could transform "mark" ->
'keyword!

I might need to ask you to up your debugging skills... :)



^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-04 19:27           ` Eric Abrahamsen
@ 2021-06-30 18:53             ` Jose A. Ortega Ruiz
  2021-06-30 19:21               ` Eric Abrahamsen
  0 siblings, 1 reply; 12+ messages in thread
From: Jose A. Ortega Ruiz @ 2021-06-30 18:53 UTC (permalink / raw)
  To: Eric Abrahamsen; +Cc: ding


hi eric,

a quick update on this... after cleaning up all my indexes, ditching
newsrc.eld and reconfiguring my groups, and also moving to xapian as
indexer for dovecot (not that any of this has necessarily had any
influence, mind you), i haven't been able to reproduce the problem, and
all my searches (including both dovecot/imap and notmuch engines) have
been working well lately.  i suspect newsrc.eld (that file tends to
accrue and cherish stale state, in my experience), but i've got no solid
reason to offer :)

cheers,
jao
-- 
‘Very good,’ I said coldly. ‘In that case, tinkerty tonk.’
And I meant it to sting. -- Bertie Wooster


^ permalink raw reply	[flat|nested] 12+ messages in thread

* Re: search for mark over Groups topic not working
  2021-06-30 18:53             ` Jose A. Ortega Ruiz
@ 2021-06-30 19:21               ` Eric Abrahamsen
  0 siblings, 0 replies; 12+ messages in thread
From: Eric Abrahamsen @ 2021-06-30 19:21 UTC (permalink / raw)
  To: Jose A. Ortega Ruiz; +Cc: ding

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> hi eric,
>
> a quick update on this... after cleaning up all my indexes, ditching
> newsrc.eld and reconfiguring my groups, and also moving to xapian as
> indexer for dovecot (not that any of this has necessarily had any
> influence, mind you), i haven't been able to reproduce the problem, and
> all my searches (including both dovecot/imap and notmuch engines) have
> been working well lately.  i suspect newsrc.eld (that file tends to
> accrue and cherish stale state, in my experience), but i've got no solid
> reason to offer :)

Thanks for the update, and that's great to hear things are stable now. I
certainly wish we had a better understanding of why things destabilize,
but that might come later...


^ permalink raw reply	[flat|nested] 12+ messages in thread

end of thread, other threads:[~2021-06-30 19:21 UTC | newest]

Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-05-17  2:31 search for mark over Groups topic not working Jose A. Ortega Ruiz
2021-06-03 18:39 ` Eric Abrahamsen
2021-06-03 19:39   ` Jose A. Ortega Ruiz
2021-06-03 21:27     ` Eric Abrahamsen
2021-06-03 21:36       ` Eric Abrahamsen
2021-06-03 23:41         ` Jose A. Ortega Ruiz
2021-06-03 23:50         ` Jose A. Ortega Ruiz
2021-06-04  0:14         ` Jose A. Ortega Ruiz
2021-06-04 16:58           ` Eric Abrahamsen
2021-06-04 19:27           ` Eric Abrahamsen
2021-06-30 18:53             ` Jose A. Ortega Ruiz
2021-06-30 19:21               ` Eric Abrahamsen

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).