spam move is very slow

spam move is very slow

[joakim]

Hello,

I've recently started trying out the spam module, albeit just a very
small subset.

I mark spam with $ while reading a group, and then press q.
The spam is very slowly moved to the spamlearn group I've configured.

Sometimes spam-move just hangs, and then I press c-g and it continues
for some reason. If I dont press c-g the spam move process often never terminates.

The spam eventually winds up in the correct spamlearn group, but there
it is marked as E, for expired. I'm not sure why I'd want this, but the
documentation says it is normal.

All of this is with imap from dovecot. I use rspamd as spam filter and
use the spamlearn group to learn rspamd about spam.

How can I debug this slowness? I dont think its dovecot being slow,
because dovecot isnt usually very slow.
-- 
Joakim Verona
joakim@verona.se
+46705459454

Re: spam move is very slow

[Steinar Bang]

>>>>> joakim@verona.se:

> How can I debug this slowness? I dont think its dovecot being slow,
> because dovecot isnt usually very slow.

Could the dovecot slowness be because Gnus move on dovecot is COPY
followed by a delete (which is accomplished by setting a delete flag and
expunging the folder)?

The change happened in March 2016?
     http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/gnus/nnimap.el?id=d5427e71da28856c5e43396a37ffd0c906947499
     http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/gnus/nnimap.el?id=8ed27bdf88cab134c26f3609c4d11c574f74705c

I don't know the reason for the change, and if the reason still is a
real issue with current versions of dovecot?

Re: spam move is very slow

[joakim]

Steinar Bang <sb@dod.no> writes:

>>>>>> joakim@verona.se:
>
>> How can I debug this slowness? I dont think its dovecot being slow,
>> because dovecot isnt usually very slow.
>
> Could the dovecot slowness be because Gnus move on dovecot is COPY
> followed by a delete (which is accomplished by setting a delete flag and
> expunging the folder)?
>
> The change happened in March 2016?
>      http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/gnus/nnimap.el?id=d5427e71da28856c5e43396a37ffd0c906947499
>      http://git.savannah.gnu.org/cgit/emacs.git/commit/lisp/gnus/nnimap.el?id=8ed27bdf88cab134c26f3609c4d11c574f74705c
>
> I don't know the reason for the change, and if the reason still is a
> real issue with current versions of dovecot?

Thanks,

I also tested with just marking the spams with the process mark, #, and
then moving them with B m. This is much faster than using the spam
function to move the spam. Does this still support the above idea?
>
>
>
-- 
Joakim Verona
joakim@verona.se
+46705459454

Re: spam move is very slow

[Ted Zlatanov]

On Thu, 28 Sep 2017 22:14:14 +0200 joakim@verona.se wrote: 
j> I mark spam with $ while reading a group, and then press q.
j> The spam is very slowly moved to the spamlearn group I've configured.

j> Sometimes spam-move just hangs, and then I press c-g and it continues
j> for some reason. If I dont press c-g the spam move process often never terminates.

That's very strange. The actual method used to move the articles is
`spam-copy-or-move-routine' and it just relies on the Gnus internals
(mark articles, then move them). So if the normal method to move the
articles is fast, but spam.el is slow, something is not right.

j> The spam eventually winds up in the correct spamlearn group, but there
j> it is marked as E, for expired. I'm not sure why I'd want this, but the
j> documentation says it is normal.

Typically people don't want to keep spam. You can then set the policy
for what to do with expired articles, but if you think there's a need
for marking them as something else, what would it be?

j> How can I debug this slowness? I dont think its dovecot being slow,
j> because dovecot isnt usually very slow.

Process-mark articles and copy/move them to your spamlearn group.

Ted

Re: spam move is very slow

[Steinar Bang]

>>>>> joakim@verona.se:

> I also tested with just marking the spams with the process mark, #, and
> then moving them with B m. This is much faster than using the spam
> function to move the spam. Does this still support the above idea?

No, that would seem to undermine the theory...

Re: spam move is very slow

[joakim]

Ted Zlatanov <tzz@lifelogs.com> writes:

> On Thu, 28 Sep 2017 22:14:14 +0200 joakim@verona.se wrote: 
> j> I mark spam with $ while reading a group, and then press q.
> j> The spam is very slowly moved to the spamlearn group I've configured.
>
> j> Sometimes spam-move just hangs, and then I press c-g and it continues
> j> for some reason. If I dont press c-g the spam move process often never terminates.
>
> That's very strange. The actual method used to move the articles is
> `spam-copy-or-move-routine' and it just relies on the Gnus internals
> (mark articles, then move them). So if the normal method to move the
> articles is fast, but spam.el is slow, something is not right.
>
> j> The spam eventually winds up in the correct spamlearn group, but there
> j> it is marked as E, for expired. I'm not sure why I'd want this, but the
> j> documentation says it is normal.
>
> Typically people don't want to keep spam. You can then set the policy
> for what to do with expired articles, but if you think there's a need
> for marking them as something else, what would it be?

I just want to move spam to a spamlearn group so I can run something
like rspamc spamlearn on the spams later. Also I would like to be able
to run other spam detectors on the spams to see how they
react. Therefore I'd like to keep my spams around. Just now I forgot
about expiry and entered my spamlearn group, and a lot of spam was
epxired and now gone. How should I've avoided this?


> j> How can I debug this slowness? I dont think its dovecot being slow,
> j> because dovecot isnt usually very slow.
>
> Process-mark articles and copy/move them to your spamlearn group.

I did this, and this is not slow.

After moving a number of messages this way, I retried using the $
method, and now its not slow anymore. So maybe my mail box was too large
or something? I started out with about 10k mails in the inbox, and now I
have about 5k mails. So I moved around 5k spams to spamlearn.



>
> Ted
>
>
>
-- 
Joakim Verona
joakim@verona.se
+46705459454

Re: spam move is very slow

[Ted Zlatanov]

On Mon, 02 Oct 2017 08:31:36 +0200 joakim@verona.se wrote: 

j> Ted Zlatanov <tzz@lifelogs.com> writes:

>> Typically people don't want to keep spam. You can then set the policy
>> for what to do with expired articles, but if you think there's a need
>> for marking them as something else, what would it be?

j> I just want to move spam to a spamlearn group so I can run something
j> like rspamc spamlearn on the spams later. Also I would like to be able
j> to run other spam detectors on the spams to see how they
j> react. Therefore I'd like to keep my spams around. Just now I forgot
j> about expiry and entered my spamlearn group, and a lot of spam was
j> epxired and now gone. How should I've avoided this?

You can turn off expiry for a group so articles marked E don't get
deleted. If you don't think that's a good solution, I could add a new
global option.

j> After moving a number of messages this way, I retried using the $
j> method, and now its not slow anymore. So maybe my mail box was too large
j> or something? I started out with about 10k mails in the inbox, and now I
j> have about 5k mails. So I moved around 5k spams to spamlearn.

I'm glad it's working!

Ted