Re: gnus nnmaildir and notmuch

[Andrew Cohen <cohen@bu.edu>]

>>>>> "Eric" == Eric Abrahamsen <eric@ericabrahamsen.net> writes:


[...]

    Eric> Andy, I wonder if any of your local changes affect this
    Eric> behavior?

Sorry, I sent a reply to the other Erik but didn't cc the list.


The behavior is intentional. As you learned, ephemeral groups explicitly
do not propagate marks. And nnir searches are ephemeral.

But as you also discovered nnir has the necessary code to propagate
marks. This is because it allows for non-ephemeral search groups as
well. I sent a message to this list about this feature several years
ago.

The new feature (permanent search groups with full marks propagation)
can be explored as follows (I didn't add the code to make it
particularly user friendly when I introduced it and then got too busy to
do much more):

Do an nnir search as you normally would (e.g. "G G" in the Group
buffer). While in the nnir Summary buffer invoke the command
'gnus-summary-create-nnir-group. This will prompt for a name and create
a group using the search parameters that yielded the nnir summary
buffer. This Group should then show up in the Group buffer. You can
modify the search by editing the Group parameters (e.g. "G p" on the
group in the Group buffer). This group behaves (almost) like any other
Gnus group, and marks should propagate as you would expect.

For example I have a group that finds all of my "flagged" messages.
For reference the group parameters for this search include:

 (nnir-specs
  (nnir-query-spec
   (query . "FLAGGED")
   (criteria . ""))
  (nnir-group-spec
   ("nnimap:home")))

This searches for all the flagged message on my nnimap:home server and
creates a a group out of the results. (It currently has over a thousand
messages:()

I also have a bunch of groups for certain correspondents; and groups for
messages in certain date ranges. And a variety of others.

In fact these "virtual" search groups are probably the primary way I
have been using gnus for the past few years. Although I haven't done it
yet, I keep thinking I should stop moving my email around and keep it
all in a single inbox and then just use these permanent search groups
instead.

I made the decision at the time that regular searches should properly be
ephemeral and not propagate marks. (And while modifying the quit-config
parameter on an ephemeral group mostly works there are some gotchas that
make it not by itself a good solution). But "permanent" search  groups
should behave more, uhm,  permanently. 

There are a few wrinkles to be aware of:

1. Since searching can be slow I chose to not redo the search on every
   scan and use a cached search result. You can always force a hard
   rescan. (This could be managed by using levels to avoid scanning on
   groups that are slow to scan instead.)

2. These permanent groups don't do expiry. Expiry is tricky for groups
   whose messages are aggregated from different sources: some of the
   sources may not support expiry; should the search group have its own
   expiry parameters or rely on the expiry rules from each article's
   home group? Since expiry rules often rely on time information, whose
   time? The search group or the home group? Of course choices could be
   made but I think any choice would lead to some surprises so I chose
   to not use it. (I visit the home groups of most articles in my
   virtual groups not infrequently and these home groups have their own
   expiry rules that then get applied).

3. Gnus sometimes gets confused about how many articles are in the
   virtual group and how many are unread. I never found it annoying
   enough to track down but there are still bugs in this regard.

4. The interface isn't very user friendly. (Actually it works pretty
   well---if you are conversant in raw imap search format. To make a new
   group I just do a random search, create the group, and then edit the
   group parameter to put in my properly crafted imap search query. The
   formats of the nnir-query-spec and nnir-group-spec are documented in
   nnir.el)

5. Unfortunately "virtual" has already been used in Gnus to mean
   something else so I can't call these groups (created from messages
   spread out across real groups) virtual. I have been (in my head)
   calling them "select" groups.

Finally I should say that when I rewrote nnir from scratch I
deliberately separated out the "searching" part from the message and
group handling part to allow this. Everything is more or less factored
out and my plan was to create an "nnselect" backend that includes all of
the non-searching parts and just uses a group parameter to specify which
articles should be included in the group. This parameter could, for
example, be an nnir search query; or it could be any function that
returns a list of articles in an appropriate format (this format is also
described in nnir.el). Everything is in place to do this (its probably
an afternoon's work to rename and move the pieces around) but I just
haven't found time to do so.

Best,
Andy