is it possible to search by list-id?

is it possible to search by list-id?

[Jose A. Ortega Ruiz]

hi,

in the new gnus search, we're told in the docs that 'Supported keys
include all the usual mail headers: “from”, “subject”, “cc”, etc', but,
if i'm not mistaken, that "etc" does not include List-id, does it?

thanks,
jao
-- 
Never let your sense of morals get in the way of doing what's
right. -Isaac Asimov, scientist and writer (1920-1992)

Re: is it possible to search by list-id?

[Eric S Fraga]

On Thursday,  8 Jul 2021 at 02:28, Jose A. Ortega Ruiz wrote:
> in the new gnus search, we're told in the docs that 'Supported keys
> include all the usual mail headers: “from”, “subject”, “cc”, etc', but,
> if i'm not mistaken, that "etc" does not include List-id, does it?

Doesn't seem to.  Would be nice to have, however.

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.4.6 on Debian bullseye/sid

Re: is it possible to search by list-id?

[Jose A. Ortega Ruiz]

On Thu, Jul 08 2021, Eric S Fraga wrote:

> On Thursday,  8 Jul 2021 at 02:28, Jose A. Ortega Ruiz wrote:
>> in the new gnus search, we're told in the docs that 'Supported keys
>> include all the usual mail headers: “from”, “subject”, “cc”, etc', but,
>> if i'm not mistaken, that "etc" does not include List-id, does it?
>
> Doesn't seem to.  Would be nice to have, however.

Right.  Moreover, a query including List-ID is not even send to the
underlying engine (which could perhaps make something of it), because it
throws an exeception... for instance, this is what i see for the query
"List-ID: foo" for some nnimap groups (which i have elided):

nnselect-run: gnus-search-run-query on ((search-query-spec (query . "List-ID: foo") (raw)) (search-group-spec ...)) gave error (wrong-type-argument stringp nil)

which looks as if the query parser gets confused.

Cheers,
jao
-- 
"There are three kinds of people: those who can count, and those who can't"

Re: is it possible to search by list-id?

[Jose A. Ortega Ruiz]

On Thu, Jul 08 2021, Jose A. Ortega Ruiz wrote:

> On Thu, Jul 08 2021, Eric S Fraga wrote:
>
>> On Thursday,  8 Jul 2021 at 02:28, Jose A. Ortega Ruiz wrote:
>>> in the new gnus search, we're told in the docs that 'Supported keys
>>> include all the usual mail headers: “from”, “subject”, “cc”, etc', but,
>>> if i'm not mistaken, that "etc" does not include List-id, does it?
>>
>> Doesn't seem to.  Would be nice to have, however.
>
> Right.  Moreover, a query including List-ID is not even send to the
> underlying engine (which could perhaps make something of it), because it
> throws an exeception... for instance, this is what i see for the query
> "List-ID: foo" for some nnimap groups (which i have elided):
>
> nnselect-run: gnus-search-run-query on ((search-query-spec (query . "List-ID: foo") (raw)) (search-group-spec ...)) gave error (wrong-type-argument stringp nil)
>
> which looks as if the query parser gets confused.

as a matter of fact, it's a bug very easy to reproduce: just evaluate,
for instance:

      (gnus-search-parse-query "foo:bar")

Re: is it possible to search by list-id?

[Jose A. Ortega Ruiz]

it seems to be possible to make this word (at least for searches backed
by dovecot's fts) simply with:

  (add-to-list 'gnus-search-expandable-keys "list-id")

that is makng the error parsing the query go away, and transforming to
the spec ((list-id . "foo")), which in turn dovecot's IMAP search engine
seems to like.

not sure if this is intended behaviour, but, as they say,
se non è vero, è ben trovato! :)

cheers,
jao
-- 
A student came to the master and asked, for the master was one of them
who knew such things: "Does Emacs have the Buddha nature?" The master
contemplated this for some time, and answered: "I don't see why not,
it has about everything else."

Re: is it possible to search by list-id?

[Eric Abrahamsen]

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> it seems to be possible to make this word (at least for searches backed
> by dovecot's fts) simply with:
>
>   (add-to-list 'gnus-search-expandable-keys "list-id")
>
> that is makng the error parsing the query go away, and transforming to
> the spec ((list-id . "foo")), which in turn dovecot's IMAP search engine
> seems to like.
>
> not sure if this is intended behaviour, but, as they say,
> se non è vero, è ben trovato! :)

No, this was a misunderstanding about how `completion-all-completions'
works, the queries are definitely meant to pass unknown keywords
straight through to the underlying engine. I've patched this now, and
added new tests to prevent regression. Thanks for pointing it out!

Eric

Re: is it possible to search by list-id?

[Adam Sjøgren]

Eric writes:

> I've patched this now, and added new tests to prevent regression.
                             ↑
This is great! Are tests easy to write for Gnus?


  All the thumbs up,

    Adam

-- 
 "Supposing my writing is strange, that is because          Adam Sjøgren
  I'm not a good writer except Japanese.  Malice is    asjo@koldfront.dk
  not in me so don't shoot me. :)"

Re: is it possible to search by list-id?

[Eric Abrahamsen]

Adam Sjøgren <asjo@koldfront.dk> writes:

> Eric writes:
>
>> I've patched this now, and added new tests to prevent regression.
>                              ↑
> This is great! Are tests easy to write for Gnus?

They most certainly aren't! Almost all the crucial code that you'd want
to test requires a full working Gnus installation, with a .newsrc.eld
and local data. There's very little code isolation between the various
bits, and any subsystem you try to test ends up calling all the other
subsystems.

gnus-search is meant to be more modular, but even here you can really
only test query parsing, and the transformation of parsed queries into
search-engine-specific strings. If you want to test the parsing of
search results, you again need a working installation, and external
programs.

I wrote a separate package called gnus-mock to address this: it sets up
a temporary working Gnus installation, and starts a new Emacs process
that you can use to play with Gnus code and wreck the data, and then
delete it all and start afresh. That's what I use for most Gnus hacking.

I also started writing an interactive test suite for this package, so
you could start it up and run the tests, and it would simulate user
behavior, but my courage failed me before I got very far. Maybe someday
when I'm in bed for a week recovering from a broken leg or pneumonia
I'll finish it.

Eric

Re: is it possible to search by list-id?

[Jose A. Ortega Ruiz]

On Sat, Jul 10 2021, Eric Abrahamsen wrote:

>> by dovecot's fts) simply with:
>>
>>   (add-to-list 'gnus-search-expandable-keys "list-id")
>>
>> that is makng the error parsing the query go away, and transforming to
>> the spec ((list-id . "foo")), which in turn dovecot's IMAP search engine
>> seems to like.
>>
>> not sure if this is intended behaviour, but, as they say,
>> se non è vero, è ben trovato! :)
>
> No, this was a misunderstanding about how `completion-all-completions'
> works, the queries are definitely meant to pass unknown keywords
> straight through to the underlying engine. I've patched this now, and
> added new tests to prevent regression. Thanks for pointing it out!

Excellent! Thanks a lot.  It's still nice to add extra keys one uses
often to gnus-search-expandable-keys, i think that i'm keeping that in
my config.

Cheers,
jao
-- 
All parts should go together without forcing. You must remember that
the parts you are reassembling were disassembled by you. Therefore, if
you can’t get them together again, there must be a reason. By all
means, do not use a hammer. —IBM Manual, 1925

Re: is it possible to search by list-id?

[Adam Sjøgren]

Eric writes:

>> This is great! Are tests easy to write for Gnus?

> They most certainly aren't! Almost all the crucial code that you'd want
> to test requires a full working Gnus installation, with a .newsrc.eld
> and local data. There's very little code isolation between the various
> bits, and any subsystem you try to test ends up calling all the other
> subsystems.

I suspected as much - all the more admirable that you added some, then!

It's my experience that retrofitting tests into a project is much more
complicated than having some from the beginning - but that's always easy
to say in hindsight...

> I wrote a separate package called gnus-mock to address this

👍

> I also started writing an interactive test suite for this package, so
> you could start it up and run the tests, and it would simulate user
> behavior, but my courage failed me before I got very far. Maybe someday
> when I'm in bed for a week recovering from a broken leg or pneumonia
> I'll finish it.

;-) I hope you'll find the time and energy some day, without having to
become ill or sustain physical injury first!


  Best regards,

    Adam

-- 
 "You gotta realize, you can kill somebody with             Adam Sjøgren
  kindness too."                                       asjo@koldfront.dk

Re: is it possible to search by list-id?

[Eric Abrahamsen]

"Jose A. Ortega Ruiz" <jao@gnu.org> writes:

> On Sat, Jul 10 2021, Eric Abrahamsen wrote:
>
>>> by dovecot's fts) simply with:
>>>
>>>   (add-to-list 'gnus-search-expandable-keys "list-id")
>>>
>>> that is makng the error parsing the query go away, and transforming to
>>> the spec ((list-id . "foo")), which in turn dovecot's IMAP search engine
>>> seems to like.
>>>
>>> not sure if this is intended behaviour, but, as they say,
>>> se non è vero, è ben trovato! :)
>>
>> No, this was a misunderstanding about how `completion-all-completions'
>> works, the queries are definitely meant to pass unknown keywords
>> straight through to the underlying engine. I've patched this now, and
>> added new tests to prevent regression. Thanks for pointing it out!
>
> Excellent! Thanks a lot.  It's still nice to add extra keys one uses
> often to gnus-search-expandable-keys, i think that i'm keeping that in
> my config.

Yes, you should definitely make personal use of this option for common
keywords.

Re: is it possible to search by list-id?

[Eric Abrahamsen]

Adam Sjøgren <asjo@koldfront.dk> writes:

> Eric writes:
>
>>> This is great! Are tests easy to write for Gnus?
>
>> They most certainly aren't! Almost all the crucial code that you'd want
>> to test requires a full working Gnus installation, with a .newsrc.eld
>> and local data. There's very little code isolation between the various
>> bits, and any subsystem you try to test ends up calling all the other
>> subsystems.
>
> I suspected as much - all the more admirable that you added some, then!
>
> It's my experience that retrofitting tests into a project is much more
> complicated than having some from the beginning - but that's always easy
> to say in hindsight...

You're right -- it might be possible to isolate Gnus' elisp data
structures from any on-disk/online behavior, but it would require pretty
much a complete rewrite of everything, eeep.

>> I wrote a separate package called gnus-mock to address this
>
> 👍
>
>> I also started writing an interactive test suite for this package, so
>> you could start it up and run the tests, and it would simulate user
>> behavior, but my courage failed me before I got very far. Maybe someday
>> when I'm in bed for a week recovering from a broken leg or pneumonia
>> I'll finish it.
>
> ;-) I hope you'll find the time and energy some day, without having to
> become ill or sustain physical injury first!

I hope so too!