[patch] `gnus-agent-fetch-headers' problems

[patch] `gnus-agent-fetch-headers' problems

[Mike McEwan]

  I think I may have finally found out how my `agent' .overview files
sometimes have their NOV lines out of sequence :-).

  The problem is when, in what I may term my `headers only' groups, I
mark articles for download via `gnus-downloadable-mark'. The articles
are downloaded just fine, but come expiry time, this means that higher 
numbered headers/articles can be expired *before* these lower numbered
articles I've later taken an interest in: Having been more recently
downloaded their entries appear later in the history file.

The net effect of this is that I end up with gaps in the
`gnus-agent-group-alist' for the group concerned. When then
calculating what articles to download in `gnus-agent-fetch-headers',
the resultant list of articles includes *again* the already expired
articles in these `gaps'. This list, with old headers to fetch, is not
necessarily a continuous sequence, but `nntp-retrieve-headers-with-xover' 
thinks that it *is* and fetches the headers via `XOVER first-last' where
the gap between articles is less than 5 (default). Headers can then be
retrieved that were *not* asked for in the list of headers to fetch.

  Anyone falling asleep here :-). Continuing...

  Having retrieved headers into ` *Gnus agent overview*' that are not
in the list of articles requested then results in in the test:

 (if (eq article (read (current-buffer)))

 in `gnus-agent-copy-nov-line' never being true when we hit one of the 
nov lines that we didn't ask for. `gnus-agent-braid-nov' continues:

 (setq articles (cdr articles))

  until we hit an article that *should* have its NOV line appended at
the end of the buffer. At this point the entire contents of ` *Gnus
agent overview*' are whacked on to the end of the .overview file,
regardless of the lowly numbered wanted/unwanted articles at the
beginning of this buffer.

  My .overview file is now out of sequence and can have duplicate NOV
lines where unwanted articles have been retrieved a second time. 

 Good to get that out of my system, it's been bugging me for months
:-).  The patch below does it for me (so far). Lars, I don't
understand the method that is used in `gnus-agent-fetch-headers' to
work out what we want to download. It asks for everything that is
unread, but not in `gnus-agent-group-alist'. Surely we're only
interested in those headers *after* the last one in this alist?
Headers that is, articles are obviously different.

-- 
Mike.

(I think the following should apply OK. I've had to do a bit of manual 
editing to hack out some other stuff I had in here.)  

(setq articles (cdr articles))--- ChangeLog.orig	Wed Aug 26 05:20:31 1998
+++ ChangeLog	Thu Aug 27 08:06:35 1998
@@ -1,3 +1,16 @@
+1998-08-27  Mike McEwan  <mike@lotusland.demon.co.uk>
+
+	* gnus-agent.el (gnus-agent-fetch-group-1): Leave the calculation
+	of `articles' to `gnus-agent-fetch-headers'.
+	(gnus-agent-fetch-headers): We only want headers that are after
+	the last entry in `gnus-group-alist'.
+
 Sat Aug 22 10:28:25 1998  Lars Magne Ingebrigtsen  <larsi@menja.ifi.uio.no>
 
 	* gnus.el: Gnus v5.6.39 is released.
--- gnus-agent.el.orig	Sun Aug 23 03:21:05 1998
+++ gnus-agent.el	Thu Aug 27 08:01:40 1998
@@ -750,18 +750,10 @@
 	(insert "\n"))
       (pop gnus-agent-group-alist))))
 
-(defun gnus-agent-fetch-headers (group articles &optional force)
-  (gnus-agent-load-alist group)
-  ;; Find out what headers we need to retrieve.
-  (when articles
-    (while (and articles
-		(assq (car articles) gnus-agent-article-alist))
-      (pop articles))
-    (let ((arts articles))
-      (while (cdr arts)
-	(if (assq (cadr arts) gnus-agent-article-alist)
-	    (setcdr arts (cddr arts))
-	  (setq arts (cdr arts)))))
+(defun gnus-agent-fetch-headers (group &optional force)
+  (let ((articles (gnus-uncompress-range 
+		   (cons (1+ (caar (last (gnus-agent-load-alist group))))
+			 (cdr (gnus-active group))))))
     ;; Fetch them.
     (when articles
       (gnus-message 7 "Fetching headers for %s..." group)
@@ -782,7 +774,7 @@
 	  (gnus-agent-enter-history "last-header-fetched-for-session"
 				    (list (cons group (nth (- (length  articles) 1) articles)))
 				    (gnus-time-to-day (current-time)))
-	t)))))
+	  articles)))))
 
 (defsubst gnus-agent-copy-nov-line (article)
   (let (b e)
@@ -842,7 +834,7 @@
 	   (gnus-agent-article-name ".agentview" group)))))
 
 (defun gnus-agent-save-alist (group &optional articles state dir)
-  "Load the article-state alist for GROUP."
+  "Save the article-state alist for GROUP."
   (nnheader-temp-write (if dir
 			   (concat dir ".agentview")
 			 (gnus-agent-article-name ".agentview" group))
@@ -896,8 +888,7 @@
 	category predicate info marks score-param)
     ;; Fetch headers.
     (when (and (or (gnus-active group) (gnus-activate-group group))
-	       (setq articles (gnus-list-of-unread-articles group))
-	       (gnus-agent-fetch-headers group articles))
+	       (setq articles (gnus-agent-fetch-headers group)))
       ;; Parse them and see which articles we want to fetch.
       (setq gnus-newsgroup-dependencies
 	    (make-vector (length articles) 0))

Re: [patch] `gnus-agent-fetch-headers' problems

[Lars Magne Ingebrigtsen]

Mike McEwan <mike@lotusland.demon.co.uk> writes:

> Surely we're only interested in those headers *after* the last one
> in this alist?  Headers that is, articles are obviously different.

Yup.

Thanks for the patch; I've applied it to Gnus v5.6.41.

-- 
(domestic pets only, the antidote for overdose, milk.)
  larsi@gnus.org * Lars Magne Ingebrigtsen