Re: nnvirtual

[Eric Abrahamsen <eric@ericabrahamsen.net>]

Andrew Cohen <cohen@bu.edu> writes:

>>>>>> "Eric" == Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>     Eric> Andrew Cohen <cohen@bu.edu> writes:
>     >>>>>>> "Eric" == Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>     >> 
>
> [...]
>
>     Eric> I'd say we have to assume that, if the functionality is there,
>     Eric> someone is using it, and combining groups is definitely
>     Eric> something people do -- see Dovecot's virtual groups, for
>     Eric> instance.
>
> Yes, I am not proposing eliminating it, I'm proposing making it more
> useful (but I don't know what that means yet:()
>
>
>     Eric> As for the large newsgroup prompting, I think it would be
>     Eric> desirable simply because it makes select groups behave like
>     Eric> other Gnus groups, and means the limit isn't hardcoded. It
>     Eric> short, it reduces users' mental friction when using select
>     Eric> groups.
>
> Yes, I agree, and my intention is to add it (since its trivial) but
> first I want to know if I can make the whole thing more useful.

Yeah, I'm not sure either. The two general principles I'm after are:
give users the ability to set up commonly-used patterns, and at the same
time give them as much flexibility as possible. I mentioned earlier the
idea of search "presets", the code for which is pretty much done (and
very simple). Each preset is a list of groups to search, and a base
search query. The user selects a preset, adds additional query terms,
an ephemeral search group is created. I made this because it seemed like
the right balance between configuration and flexibility.

But if, for example, it were possible to start a search from within a
summary buffer, saying "run this search within this group", then presets
might not be necessary: users could create permanent select groups
incorporating several other groups, then run ad-hoc searches within
them.

All of which is only to say that I don't really know, either. But Gnus
users are a strange bunch, I'd say support as much as possible.

>     Eric> Random other things:
>
>     Eric> How does sorting work in select groups? Right now gnus-search
>     Eric> does a pass in `gnus-search-run-query', sorting on rsv, but
>     Eric> since nearly all engines return a score of 100, it's fairly
>     Eric> meaningless. 
>
> I agree. I have gone back and forth about whether to eliminate it
> entirely. Eliminating it would allow some compression in various
> internal variables but this probably doesn't make much of a difference.
> But nnselect is currently ignoring the RSV---it is still available so
> that sorting functions can use it, however. nnselect is agnostic about
> the whole thing.

I'd still like to keep it, and the infrastructure to support it. It
might become more useful later. Namazu hints that it supports scoring (I
really need to get a working installation), imap returns relevancy with
FUZZY, and future engines for solr or xapian-based stuff would provide
scoring.

> All nnselect groups should be sorted exactly as any other gnus group,
> and should be as flexible as the usual sorting system. If that isn't
> working right its a bug.

> So currently to use the RSV for sorting purposes requires constructing
> predicates that look at the RSV. This is trivial to do (and I'll
> probably include them as part of nnselect or in gnus-sum with the other
> sorting predicates at some point). But we can (and probably should)
> ignore any sorting issues coming out of gnus-search.

Responding to both messages here: if there were a new predicate for
sorting on score, I think we'd be fine. Part of me still would like to
let users specify sorting strategies on the fly, as part of the search
query. Sorting could also be passed to the backends, and combined with
limit:n could result in faster queries. At this point that's probably
more work than it's worth, though.

>     Eric> I've provided a count:t meta key in the search language, but
>     Eric> it isn't used right now. The idea is that, if the user
>     Eric> includes this key, the search returns a total number of hits
>     Eric> per engine, rather than actually displaying the results (hits
>     Eric> per group would be nice, but that information could be
>     Eric> difficult extract from some engines). It will be fairly easy
>     Eric> to hot-wire `gnus-group-make-search-group' to return this
>     Eric> value, and permanent groups could simply strip it out.
>
> Hmm, the old nnir reported this (it went by too fast to be useful, but
> it was always available after the fact in the message buffer).

Right, but the idea is that sometimes the count itself is all you want
to see.

>     Eric> If you like, it could also be a cheap way of updating message
>     Eric> counts in permanent search groups. I note now that, at
>     Eric> start-up for instance, the permanent groups do not show counts
>     Eric> until they're entered. 
>
> This is a temporary artifact of my not yet activating the groups at
> startup. The counts are all present and ready to be reported (as soon as
> the group is created), I just haven't gotten around to keeping the
> proper list of nnselect groups to allow activation and re-scanning as a
> matter of course.
>
>     Eric> Sending a count request ought to be significantly cheaper than
>     Eric> actually retrieving results, and might be useful.
>
> The counts are currently updated without retrieving any headers, and is
> therefore quite fast. For example I have a variety of groups with
> thousands of messages that are part of the nnselection and the time it
> takes to update the group is a fraction of a second.

Well, it's probably premature optimization. But it would be pretty
simple to implement...