spam-report.el pinging too fast?

spam-report.el pinging too fast?

[Lars Magne Ingebrigtsen]

I just saw the load on spam.gmane.org go to 120, so I grepped through
the access.log, and saw lots of stuff from spam-report.el there.  :-)

Now, the problem is that spam.php is slow, because it tries to give
sensible feedback on duplicate reports and stuff.  spam-report.el
doesn't wait for any response, though, which means that a user that
has a lot of spam to report will drive up the load.

So there's two solutions to this: Either spam-report.el could wait
for a response (and serialize the reports), or I could write a new
"programmatic" spam.php that queues the reports and does the real
handling of them later.

What do you think?
 
-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: spam-report.el pinging too fast?

[Lars Magne Ingebrigtsen]

Lars Magne Ingebrigtsen <larsi@gnus.org> writes:

> So there's two solutions to this: Either spam-report.el could wait
> for a response (and serialize the reports), or I could write a new
> "programmatic" spam.php that queues the reports and does the real
> handling of them later.

I've now done the latter.  :-)

The new URL is

http://gmane.org/ping-spam.php?group=some.group&article=1234&command=spam

(or unspam).

spam.gmane.org checks User-Agent, and if it contains
"spam-report.el", it'll just queue the report.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen

Re: spam-report.el pinging too fast?

[Reiner Steib]

On Tue, May 25 2004, Lars Magne Ingebrigtsen wrote:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
>
>> So there's two solutions to this: Either spam-report.el could wait
>> for a response (and serialize the reports), or I could write a new
>> "programmatic" spam.php that queues the reports and does the real
>> handling of them later.
>
> I've now done the latter.  :-)
>
> The new URL is
>
> http://gmane.org/ping-spam.php?group=some.group&article=1234&command=spam
>
> (or unspam).

It would be slightly nicer (easier parsing X-Report-Spam and
construction the ping-URL) to have ping-URLs similar to the normal
ones, e.g. by pre- or appending some keywords:

X-Report-Spam: http://spam.gmane.org/GROUP:NUMBER

           --> http://spam.gmane.org/GROUP:NUMBER&ping
or
           --> http://spam.gmane.org/ping?group=GROUP:NUMBER

While at it...  Some time ago we discussed[1] whether it would make
sense to report more than one article in a single HTTP request.
spam-report currently sends out one request for each article, see
`spam-report-gmane' and `spam-report-url-ping-plain'.  We could
collect the reports for each group (actually we already have a list of
articles in the first place) and do a single request, say
"http://spam.gmane.org/GROUP:NUMBER,NUMBER,NUMBER,NUMBER,NUMBER".
Would that be useful from Gmane's point of view?

Bye, Reiner.

[1] <URL:http://thread.gmane.org/v9r7xesw98.fsf@marauder.physik.uni-ulm.de>
    <news:v9d68tuqn3.fsf@marauder.physik.uni-ulm.de>
    <news:4nisigasec.fsf@collins.bwh.harvard.edu>
-- 
       ,,,
      (o o)
---ooO-(_)-Ooo--- PGP key available via WWW   http://rsteib.home.pages.de/

Re: spam-report.el pinging too fast?

[Ted Zlatanov]

On Tue, 25 May 2004, larsi@gnus.org wrote:

> Lars Magne Ingebrigtsen <larsi@gnus.org> writes:
> 
>> So there's two solutions to this: Either spam-report.el could wait
>> for a response (and serialize the reports), or I could write a new
>> "programmatic" spam.php that queues the reports and does the real
>> handling of them later.
> 
> I've now done the latter.  :-)
> 
> The new URL is
> 
> http://gmane.org/ping-spam.php?group=some.group&article=1234&command=spam
> 
> (or unspam).
> 
> spam.gmane.org checks User-Agent, and if it contains
> "spam-report.el", it'll just queue the report.

So this code in spam-report.el will have to change:

      (if spam-report-gmane-use-article-number
          (spam-report-url-ping 
           "spam.gmane.org"
           (format "/%s:%d"
                   (gnus-group-real-name gnus-newsgroup-name)
                   article))

or does it still work?  When I connected directly it seemed to be OK,
but I wasn't sure if you're rewriting properly or it's going to
/dev/null.

Hmmm, maybe I should also have a ham Gmane reporter!  I didn't know
you could "unspam."  Does it work with the
http://unspam.gmane.org/GROUP:ARTICLE format as you would expect?  I
tested and it looked OK, but same question as above...

Thanks
Ted