gnus registry takes 21 seconds to save and I can't prune it

gnus registry takes 21 seconds to save and I can't prune it

[Rémi Letot]

Hello,

I have noticed quite a few slowdowns in my emacs for some times, but it
never became annoying enough for me to investigate.

Until now :-)

Starting emacs, gnus, or quitting them has become annoyingly long. It is
not really problematic because I leave emacs/gnus on for weeks at a
time, but still. Besides, pressing s in the group buffer is far too long
to be comfortable.

So, I diagnosed it to gnus-registry-save taking 21 seconds. 

I tried to see what happens in there, and noticed that I could set
gnus-registry-max-entries to a sensible number, but then I get an error
whenever I try to prune the registry (meaning as soon as I try to do
anything in gnus).

The error is "cl-some: Wrong type argument: listp, quote".

That happens in registry-collect-prune-candidates, on the "(memq (car
entry-key) precious)))" line, but there stops my lisp and edebug foo.

In short: can anyone help ? :-)
-- 
Rémi

Re: gnus registry takes 21 seconds to save and I can't prune it

[Eric S Fraga]

On Wednesday, 18 Mar 2020 at 13:57, Rémi Letot wrote:
> Starting emacs, gnus, or quitting them has become annoyingly long. It is
> not really problematic because I leave emacs/gnus on for weeks at a
> time, but still. Besides, pressing s in the group buffer is far too long
> to be comfortable.
>
> So, I diagnosed it to gnus-registry-save taking 21 seconds. 

I've noticed this recently as well.  Looking at the git logs, nothing
seems to have changed recently that would account for this.  I put it
down to my registry getting large enough now for the processing to be
significant (I have around 12k entries with max allowed of 20k).  I've
not tried pruning.

-- 
Eric S Fraga via Emacs 28.0.50 & org 9.3.6 on Debian bullseye/sid

Re: gnus registry takes 21 seconds to save and I can't prune it

[Eric Abrahamsen]

hobbes@poukram.net (Rémi Letot) writes:

> Hello,
>
> I have noticed quite a few slowdowns in my emacs for some times, but it
> never became annoying enough for me to investigate.
>
> Until now :-)
>
> Starting emacs, gnus, or quitting them has become annoyingly long. It is
> not really problematic because I leave emacs/gnus on for weeks at a
> time, but still. Besides, pressing s in the group buffer is far too long
> to be comfortable.
>
> So, I diagnosed it to gnus-registry-save taking 21 seconds. 
>
> I tried to see what happens in there, and noticed that I could set
> gnus-registry-max-entries to a sensible number, but then I get an error
> whenever I try to prune the registry (meaning as soon as I try to do
> anything in gnus).
>
> The error is "cl-some: Wrong type argument: listp, quote".

Slow saving is a separate issue, and one that annoys a lot of people,
but this error sounds a lot like something that was going on a year or
more ago. If it's the same thing, there ought to be a solution:

1. Shut down Gnus
2. Back up your registry file
3. Start Gnus again
4. Eval the code below
5. Run M-x gnus-registry-fixit
6. Save Gnus

With any luck that will resolve the error.

(defun gnus-registry-strip-quotes (lst)
  (let (acc)
    (when (consp lst)
      (while (eq (car lst) 'quote)
	(setq lst (cadr lst)))
      (while (consp lst)
	(if (eq (car lst) 'quote)
	    (setq lst (cadr lst))
	  (push (gnus-registry-strip-quotes (car lst)) acc)
	  (setq lst (cdr lst)))))
    (nconc (nreverse (delete-dups acc))
	   lst)))

(defun gnus-registry-fixit ()
  "Fix Gnus registry after eieio-persistent patches.
For use with Emacs master branch, after installing 0afb43eeb, or
the 26 branch, after daa9e853bd."
  (interactive)
  (unless (gnus-alive-p)
    (gnus))
  (when gnus-registry-enabled
    (with-slots (tracker data) gnus-registry-db
      (maphash
       (lambda (track-sym hsh)
	 (maphash
	  (lambda (k v)
	    (setf (gethash k hsh)
		  (gnus-registry-strip-quotes v)))
	  hsh))
       tracker)
      (maphash
       (lambda (k v)
	 (setf (gethash k data) (gnus-registry-strip-quotes v)))
       data))))   

Re: gnus registry takes 21 seconds to save and I can't prune it

[Bob Newell]

I remember encountering this a year ago (maybe more or less, I didn't
look it up), and coming up with a kludge solution, which Eric replaced
with his far nicer code. It definitely works and makes things usable
again. I run gnus-registry-fixit automatically on Emacs shutdown.

--
Bob Newell
Via Linux/Emacs/Gnus/BBDB.

Re: gnus registry takes 21 seconds to save and I can't prune it

[Eric Abrahamsen]

Bob Newell <bobnewell@bobnewell.net> writes:

> I remember encountering this a year ago (maybe more or less, I didn't
> look it up), and coming up with a kludge solution, which Eric replaced
> with his far nicer code. It definitely works and makes things usable
> again. I run gnus-registry-fixit automatically on Emacs shutdown.

You shouldn't need to keep running it! Once you're past 0afb43eeb on
Emcas 27, or after daa9e853b on Emacs 26, you should only have to run it
once -- the problem won't return.

(And if you find saving the registry is annoyingly slow, this will make
it worse!)

Re: gnus registry takes 21 seconds to save and I can't prune it

[hobbes]

Eric Abrahamsen <eric@ericabrahamsen.net> writes:

>
> Slow saving is a separate issue, and one that annoys a lot of people,
> but this error sounds a lot like something that was going on a year or
> more ago. If it's the same thing, there ought to be a solution:
>
> 1. Shut down Gnus
> 2. Back up your registry file
> 3. Start Gnus again
> 4. Eval the code below
> 5. Run M-x gnus-registry-fixit
> 6. Save Gnus
>
> With any luck that will resolve the error.

it did, and my gnus-registry is now pruned at 30M. Still slow, but a lot
more manageable, thanks :-)

-- 
Rémi Letot