Fixed bug with custom marks in registry

Fixed bug with custom marks in registry

[Eric Abrahamsen]

Obviously not too many people were using this feature, or else we would
have heard about it sooner, but there was a bug in the Gnus registry
where entries with custom marks attached to them could still be pruned
(deleted), even though they're not supposed to be.

That's fixed now, and the fix will go into Emacs 26.2 and 27, so if
you'd run into this before and decided not to use custom marks in the
registry, watch for updates.

Eric

Re: Fixed bug with custom marks in registry

[Emanuel Berg]

Eric Abrahamsen wrote:

> That's fixed now, and the fix will go into
> Emacs 26.2 and 27, so if you'd run into this
> before and decided not to use custom marks in
> the registry, watch for updates.

Great job!

-- 
underground experts united
http://user.it.uu.se/~embe8573

Re: Fixed bug with custom marks in registry

[Ted Zlatanov]

On Thu, 11 Oct 2018 11:26:00 -0700 Eric Abrahamsen <eric@ericabrahamsen.net> wrote: 

EA> Obviously not too many people were using this feature, or else we would
EA> have heard about it sooner, but there was a bug in the Gnus registry
EA> where entries with custom marks attached to them could still be pruned
EA> (deleted), even though they're not supposed to be.

EA> That's fixed now, and the fix will go into Emacs 26.2 and 27, so if
EA> you'd run into this before and decided not to use custom marks in the
EA> registry, watch for updates.

Thanks Eric!

Ted

Re: Fixed bug with custom marks in registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Obviously not too many people were using this feature, or else we would
> have heard about it sooner, but there was a bug in the Gnus registry
> where entries with custom marks attached to them could still be pruned
> (deleted), even though they're not supposed to be.
>
> That's fixed now, and the fix will go into Emacs 26.2 and 27, so if
> you'd run into this before and decided not to use custom marks in the
> registry, watch for updates.

I've a question about the setup.  I want Gnus to always auto-show all
registry-marked articles in the summary (maybe we should have an option
for that?).  From the docs I derived that something like this would do:

#+begin_src emacs-lisp
(add-function
 :around gnus-alter-articles-to-read-function
 (defun my-gnus-auto-show-registry-marked (f group-name article-list)
   (cl-union
    (delq nil
          (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
                  (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
                           using (hash-values v)
                           when (cdr (assoc 'mark v))
                           collect key)))
    (funcall f group-name article-list))))
#+end_src

That works but has the side effect that when I add or remove a registry
mark - e.g. simply M M i or M M I, after that the article line in the
summary is duplicated - one line has the mark, the other doesn't.  Apart
from that, the lines are identical.  When I restart Gnus, I get one
article line again as expected.

Any hints how to do it better? (FWIW, I do not remember that this
happened to me in the past, but I'm not sure)


Thanks,

Michael.

Re: Fixed bug with custom marks in registry

[Michael Heerdegen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> That works but has the side effect that when I add or remove a registry
> mark - e.g. simply M M i or M M I, after that the article line in the
> summary is duplicated - one line has the mark, the other doesn't.

This doesn't happen to all articles.  I guess it happens only to
articles that are not ticked or that are shown additionally.

Michael.

Re: Fixed bug with custom marks in registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Obviously not too many people were using this feature, or else we would
>> have heard about it sooner, but there was a bug in the Gnus registry
>> where entries with custom marks attached to them could still be pruned
>> (deleted), even though they're not supposed to be.
>>
>> That's fixed now, and the fix will go into Emacs 26.2 and 27, so if
>> you'd run into this before and decided not to use custom marks in the
>> registry, watch for updates.
>
> I've a question about the setup.  I want Gnus to always auto-show all
> registry-marked articles in the summary (maybe we should have an option
> for that?).  From the docs I derived that something like this would do:
>
> #+begin_src emacs-lisp
> (add-function
>  :around gnus-alter-articles-to-read-function
>  (defun my-gnus-auto-show-registry-marked (f group-name article-list)
>    (cl-union
>     (delq nil
>           (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
>                   (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
>                            using (hash-values v)
>                            when (cdr (assoc 'mark v))
>                            collect key)))
>     (funcall f group-name article-list))))
> #+end_src

TBH I hadn't even looked into this part of the code before you brought
it up last time, so I'm not really ahead of you in this game. I think in
principle it would be nice to have a higher-level, more user-friendly
way of adjusting this behavior. I could certainly imagine using an
option that let me say "always display any message that has registry
keys marked "precious"".

> That works but has the side effect that when I add or remove a registry
> mark - e.g. simply M M i or M M I, after that the article line in the
> summary is duplicated - one line has the mark, the other doesn't.  Apart
> from that, the lines are identical.  When I restart Gnus, I get one
> article line again as expected.

I tried this with a basic setup, and didn't see a problem -- the message
lines were not duplicated as I added/removed marks. Do you have multiple
functions added to `gnus-alter-articles-to-read-function' that might be
interacting with each other?

I actually tried this with gnus-mock, which I continue to think is
helpful. If you want to try that and see if it helps track down the
problem, first wait until gnus-mock updates to 0.2.1 (*ahem*), then
install it, set `gnus-mock-emacs-program' to whatever emacs you want to
use, set `gnus-mock-gnus-file' to a random file and then put this in the
file:

#+begin_src elisp
(setq gnus-summary-line-format "%U%R%z%uM%I%(%[%4L: %-23,23f%]%) %s\n")

(defalias 'gnus-user-format-function-M 'gnus-registry-article-marks-to-chars)

(gnus-registry-initialize)

(add-function
 :around gnus-alter-articles-to-read-function
 (defun my-gnus-auto-show-registry-marked (f group-name article-list)
   (cl-union
    (delq nil
          (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
                  (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
                           using (hash-values v)
                           when (cdr (assoc 'mark v))
                           collect key)))
    (funcall f group-name article-list))))
#+end_src

That should get you as close to "gnus -Q" as you can get, and might
provide a starting place for tracking down the weird behavior.

Eric

Re: Fixed bug with custom marks in registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> TBH I hadn't even looked into this part of the code before you brought
> it up last time, so I'm not really ahead of you in this game. I think in
> principle it would be nice to have a higher-level, more user-friendly
> way of adjusting this behavior. I could certainly imagine using an
> option that let me say "always display any message that has registry
> keys marked "precious"".

Yeah, something like that would be nice.

> I tried this with a basic setup, and didn't see a problem -- the message
> lines were not duplicated as I added/removed marks. Do you have multiple
> functions added to `gnus-alter-articles-to-read-function' that might be
> interacting with each other?

No, only this one.

> I actually tried this with gnus-mock [...]
> [...]
> That should get you as close to "gnus -Q" as you can get, and might
> provide a starting place for tracking down the weird behavior.

That worked nice.  I couldn't reproduce there.  Furthermore, I can't now
reproduce with my own setup as well.  Dunno why - maybe all Gnus wanted
was a restart.

Thanks for your help.


Michael.

Re: Fixed bug with custom marks in registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> TBH I hadn't even looked into this part of the code before you brought
>> it up last time, so I'm not really ahead of you in this game. I think in
>> principle it would be nice to have a higher-level, more user-friendly
>> way of adjusting this behavior. I could certainly imagine using an
>> option that let me say "always display any message that has registry
>> keys marked "precious"".
>
> Yeah, something like that would be nice.

I'll put that on the list of things to consider!

>> I tried this with a basic setup, and didn't see a problem -- the message
>> lines were not duplicated as I added/removed marks. Do you have multiple
>> functions added to `gnus-alter-articles-to-read-function' that might be
>> interacting with each other?
>
> No, only this one.
>
>> I actually tried this with gnus-mock [...]
>> [...]
>> That should get you as close to "gnus -Q" as you can get, and might
>> provide a starting place for tracking down the weird behavior.
>
> That worked nice.  I couldn't reproduce there.  Furthermore, I can't now
> reproduce with my own setup as well.  Dunno why - maybe all Gnus wanted
> was a restart.
>
> Thanks for your help.

I expended very little effort :)

Re: Fixed bug with custom marks in registry

[Eric Abrahamsen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Michael Heerdegen <michael_heerdegen@web.de> writes:
>
>> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>>
>>> TBH I hadn't even looked into this part of the code before you brought
>>> it up last time, so I'm not really ahead of you in this game. I think in
>>> principle it would be nice to have a higher-level, more user-friendly
>>> way of adjusting this behavior. I could certainly imagine using an
>>> option that let me say "always display any message that has registry
>>> keys marked "precious"".
>>
>> Yeah, something like that would be nice.
>
> I'll put that on the list of things to consider!

Right now the thing that makes the most sense to me is:

1. Make a new group parameter that matches the "display" parameter that
   you can set for individual groups. Ie, we'd have a
   `gnus-parameter-display-alist' variable.
2. Allow that parameter to take more than just a number to display: it
   should be a list, and allow pre-set symbols (like 'registry-marked,
   'registry-precious), and maybe functions? I haven't thought it all
   the way through.
3. Consult that parameter when constructing Summary buffers.


How's that sound?

Eric

Re: Fixed bug with custom marks in registry

[Michael Heerdegen]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

> Right now the thing that makes the most sense to me is:
>
> 1. Make a new group parameter that matches the "display" parameter that
>    you can set for individual groups. Ie, we'd have a
>    `gnus-parameter-display-alist' variable.
> 2. Allow that parameter to take more than just a number to display: it
>    should be a list, and allow pre-set symbols (like 'registry-marked,
>    'registry-precious), and maybe functions? I haven't thought it all
>    the way through.
> 3. Consult that parameter when constructing Summary buffers.
>
> How's that sound?

I guess it makes sense.  I don't use Gnus parameters much.

The approach that came to my mind would be to just factor out common
parts and provide them as predefined functions.  For example, we could
provide

#+begin_src emacs-lisp
(defun gnus-alter-articles-to-read-create-function (f)
  (lambda (group-name article-list)
    (cl-union
     (delq nil
           (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
                   (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
                            using (hash-values v)
                            when (funcall f v)
                            collect key))))))
#+end_src

and the user could do

#+begin_src emacs-lisp
(add-function
 :around gnus-alter-articles-to-read-function
 (defun my-gnus-to-read-add-marked (orig-f group-name article-list)
   (cl-union (funcall (gnus-alter-articles-to-read-create-function
                       (lambda (v) (cdr (assoc 'mark v))))
                      group-name article-list)
             (funcall orig-f group-name article-list))))
#+end_src

The advantage would be that this doesn't require knowledge about
database internals, and would provide a maximum of configurability.

Would that approach be compatible with your proposal?


Michael.

Re: Fixed bug with custom marks in registry

[Eric Abrahamsen]

Michael Heerdegen <michael_heerdegen@web.de> writes:

> Eric Abrahamsen <eric@ericabrahamsen.net> writes:
>
>> Right now the thing that makes the most sense to me is:
>>
>> 1. Make a new group parameter that matches the "display" parameter that
>>    you can set for individual groups. Ie, we'd have a
>>    `gnus-parameter-display-alist' variable.
>> 2. Allow that parameter to take more than just a number to display: it
>>    should be a list, and allow pre-set symbols (like 'registry-marked,
>>    'registry-precious), and maybe functions? I haven't thought it all
>>    the way through.
>> 3. Consult that parameter when constructing Summary buffers.
>>
>> How's that sound?
>
> I guess it makes sense.  I don't use Gnus parameters much.

They're pretty handy!

> The approach that came to my mind would be to just factor out common
> parts and provide them as predefined functions.  For example, we could
> provide
>
> #+begin_src emacs-lisp
> (defun gnus-alter-articles-to-read-create-function (f)
>   (lambda (group-name article-list)
>     (cl-union
>      (delq nil
>            (mapcar (lambda (id) (cdr (gnus-request-head id group-name)))
>                    (cl-loop for key being the hash-keys of (oref gnus-registry-db data)
>                             using (hash-values v)
>                             when (funcall f v)
>                             collect key))))))
> #+end_src
>
>
> and the user could do
>
> #+begin_src emacs-lisp
> (add-function
>  :around gnus-alter-articles-to-read-function
>  (defun my-gnus-to-read-add-marked (orig-f group-name article-list)
>    (cl-union (funcall (gnus-alter-articles-to-read-create-function
>                        (lambda (v) (cdr (assoc 'mark v))))
>                       group-name article-list)
>              (funcall orig-f group-name article-list))))
> #+end_src
>
> The advantage would be that this doesn't require knowledge about
> database internals, and would provide a maximum of configurability.

But this seems to require just enough knowledge of the database
internals (and Gnus internals) to be offputting to regular users, while
not saving expert users all that much effort.

> Would that approach be compatible with your proposal?

I guess what I want is an approach that works earlier in the process and
adds/filters efficiently, altering the list of articles to read *before*
requesting heads (or at least not requesting redundant heads then doing
`cl-union' afterwards), and also providing a list of no-brainer default
options.

Then `gnus-alter-articles-to-read-function' could be applied on top of
this, but it would be understood to be for expert users to do custom
tweaking with -- and probably not very performantly.

E