(gnus-data-list nil) does not give valid data sometimes

(gnus-data-list nil) does not give valid data sometimes

[Ted Zlatanov]

The function that fails is:

(defun spam-fetch-field-fast (article field)
  "Fetch a field quickly, using the internal gnus-data-list function"
  (when (numberp article)
    (gnus-get-newsgroup-headers)
    (let* ((header (assoc article (gnus-data-list nil)))
	   (data-header (if header (gnus-data-header header) nil)))
      (if (arrayp data-header)
	(cond
	 ((equal field 'from)
	  (mail-header-from data-header))
	 ((equal field 'message-id)
	  (mail-header-message-id data-header))
	 ((equal field 'subject)
	  (mail-header-subject data-header))
	 ((equal field 'references)
	  (mail-header-references data-header))
	 ((equal field 'date)
	  (mail-header-date data-header))
	 ((equal field 'xref)
	  (mail-header-xref data-header))
	 ((equal field 'extra)
	  (mail-header-extra data-header))
	 (t
	  nil))
	(gnus-error 5 "Article %d has a nil data header" article)))))

When entering a group the first time, lots of errors are generated
above because data-header is nil.  After C-g, the next time the group
is entered everything is fine.  BUT the next time (3rd time) the
group is entered, the errors happen again - and so on, this can be
repeated forever.  I don't use the agent - I'm not sure what's wrong.

This, by the way, is called by spam-insert-fake-headers which is
called by spam-find-spam; spam-find-spam is hooked on
gnus-summary-prepare-hook.

Thanks
Ted

Re: (gnus-data-list nil) does not give valid data sometimes

[Ted Zlatanov]

On Thu, 08 Jan 2004, tzz@lifelogs.com wrote:

> When entering a group the first time, lots of errors are generated
> above because data-header is nil.  After C-g, the next time the
> group is entered everything is fine.  BUT the next time (3rd time)
> the group is entered, the errors happen again - and so on, this can
> be repeated forever.  I don't use the agent - I'm not sure what's
> wrong.
> 
> This, by the way, is called by spam-insert-fake-headers which is
> called by spam-find-spam; spam-find-spam is hooked on
> gnus-summary-prepare-hook.

I only see this problem on nntp.perl.org.  It does not happen on any
of my other news servers.  It may be a NNTP server issue or a Gnus bug.

Ted

bug/feature?: gnus-data-list is not ready in gnus-summary-prepare-hook (was: (gnus-data-list nil) does not give valid data sometimes)

[Ted Zlatanov]

On Thu, 08 Jan 2004, tzz@lifelogs.com wrote:

On Thu, 08 Jan 2004, tzz@lifelogs.com wrote:
> 
>> When entering a group the first time, lots of errors are generated
>> above because data-header is nil.  After C-g, the next time the
>> group is entered everything is fine.  BUT the next time (3rd time)
>> the group is entered, the errors happen again - and so on, this can
>> be repeated forever.  I don't use the agent - I'm not sure what's
>> wrong.
>> 
>> This, by the way, is called by spam-insert-fake-headers which is
>> called by spam-find-spam; spam-find-spam is hooked on
>> gnus-summary-prepare-hook.
> 
> I only see this problem on nntp.perl.org.  It does not happen on any
> of my other news servers.  It may be a NNTP server issue or a Gnus
> bug.

Hmm, now I got the error in other news servers.  I think the
gnus-data-list function is not ready to give me valid data in the
gnus-summary-prepare hook.  Can anyone tell me how I can circumvent
this?  

The goal is to gather overview data for the articles in the group, so
I can detect if they are spam.  I used to retrieve the whole article
in a temporary buffer and run spam detection there, but that is
overkill for most spam-check-* functions.  So I'm trying (in
spam-find-spam) to get the overview headers for the unseen/unread
articles in the group.

Thanks
Ted

Re: bug/feature?: gnus-data-list is not ready in

[Ted Zlatanov]

On Fri, 09 Jan 2004, tzz@lifelogs.com wrote:

> Hmm, now I got the error in other news servers.  I think the
> gnus-data-list function is not ready to give me valid data in the
> gnus-summary-prepare hook.  Can anyone tell me how I can circumvent
> this?  

Bah, it was so simple I never thought of it.  gnus-newsgroup-data is
buffer-local to the gnus-summary-buffer.  I was doing my insertion in
a (with-temp-buffer) form, so debugging was showing me everything was
fine but when I ran things they didn't work.  Incidentally,
(with-current-buffer) inside a (with-temp-buffer) form does not do
the right thing with regards to buffer-local variables.  I'm fighting
the urge to spell "buffer" as "bugger" :)

Sorry for the trouble.  Fix is in CVS - the latest spam.el should be
much, much faster at autodetecting articles when using non-statistical
backends such as spam-use-BBDB.  It will still request the complete
article if a backend that uses statistics or extra headers needs it,
such as spam-use-bogofilter, but when possible a "fake" article will
be built instead from the overview data, and the backend function
will be run on that.  The speedup is tremendous - spam-autodetect is
finally, I think, fast enough for regular usage.

Ted