From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.io/gmane.emacs.gnus.general/16383 Path: main.gmane.org!not-for-mail From: Mike McEwan Newsgroups: gmane.emacs.gnus.general Subject: [patch] `gnus-agent-fetch-heades' (again) Date: 29 Aug 1998 14:07:52 +0100 Sender: owner-ding@hpc.uh.edu Message-ID: NNTP-Posting-Host: coloc-standby.netfonds.no Mime-Version: 1.0 (generated by tm-edit 7.108) Content-Type: text/plain; charset=US-ASCII X-Trace: main.gmane.org 1035155265 27457 80.91.224.250 (20 Oct 2002 23:07:45 GMT) X-Complaints-To: usenet@main.gmane.org NNTP-Posting-Date: Sun, 20 Oct 2002 23:07:45 +0000 (UTC) Return-Path: Original-Received: from gizmo.hpc.uh.edu (gizmo.hpc.uh.edu [129.7.102.31]) by sclp3.sclp.com (8.8.5/8.8.5) with ESMTP id JAA25656 for ; Sat, 29 Aug 1998 09:23:36 -0400 (EDT) Original-Received: from sina.hpc.uh.edu (sina.hpc.uh.edu [129.7.3.5]) by gizmo.hpc.uh.edu (8.7.6/8.7.3) with ESMTP id HAF12769; Sat, 29 Aug 1998 07:53:17 -0500 Original-Received: by sina.hpc.uh.edu (TLB v0.09a (1.20 tibbs 1996/10/09 22:03:07)); Sat, 29 Aug 1998 08:21:59 -0500 (CDT) Original-Received: from sclp3.sclp.com (root@sclp3.sclp.com [209.195.19.139]) by sina.hpc.uh.edu (8.7.3/8.7.3) with ESMTP id IAA11162 for ; Sat, 29 Aug 1998 08:21:47 -0500 (CDT) Original-Received: from post.mail.demon.net (post-12.mail.demon.net [194.217.242.41]) by sclp3.sclp.com (8.8.5/8.8.5) with SMTP id JAA25635 for ; Sat, 29 Aug 1998 09:21:39 -0400 (EDT) Original-Received: from (lotusland.demon.co.uk) [158.152.62.156] by post.mail.demon.net with smtp (Exim 1.82 #2) id 0zCkwf-0005pd-00; Sat, 29 Aug 1998 13:21:33 +0000 Original-Received: from mike by lotusland.demon.co.uk with local (Exim 2.02 #1) id 0zCkwe-0003Co-00 for ding@gnus.org; Sat, 29 Aug 1998 14:21:32 +0100 Original-To: ding@gnus.org X-Mailer: Gnus v5.6.41/XEmacs 20.4 - "Emerald" Original-Lines: 310 Precedence: list X-Majordomo: 1.94.jlt7 Xref: main.gmane.org gmane.emacs.gnus.general:16383 X-Report-Spam: http://spam.gmane.org/gmane.emacs.gnus.general:16383 From my previous mailing on the subject: "Lars, I don't understand the method that is used in `gnus-agent-fetch-headers' to work out what we want to download. It asks for everything that is unread, but not in `gnus-agent-group-alist'. Surely we're only interested in those headers *after* the last one in this alist?" Well I'm coming to understand a little more now :-). Following my last patch (Lars, at least you added a check to see that we actually have a `gnus-agent-article-alist') I attempted to subscribe to a new newsgroup, but not having a .agentview initially `~fetch-headers' does not select any articles :-(. Having thought a little more about the possible situations, just fetching all headers after the last one in .agentview does not cut it. Apart from the situation above, *all* headers in .agentview could have been expired. It could be that all headers bar a few recently downloaded via `gnus-downloadable-mark' had been expired. In this case headers after the last one in .agentview may have already been downloaded. The following *two* patches tackle the problem in slightly different ways. The first, my preferred option, sets the last header fetched to in a group's parameters. Headers can be safely downloaded from this point on, the next time we connect to the news-server. The only slight problem with this is that, being in a group's parameters, it is visible to the user. Do you trust the user user not to meddle with this strange new parameter :-P. I do. Indeed this where I'm currently storing this information for the `new articles' functions I'm tweaking with. This is where, if enabled, the timestamp of a group's last entry is kept, and this provides a similar function. I don't suppose it'd be easy to tuck this away in the main body of a group's info though? The second option is nearly a return to what I couldn't understand above. Headers are selected for download if they are after the last header in .agentview *and* the aren't in the list of read articles. This could still be a bit dodgy I reckon. You have to make sure that *all* expired articles are marked read - see below. I'm still not happy with this. When calculating unread articles, gnus could still re-download some headers because of an unlucky spread of cancelled articles (not being read, they are unread no?). As mentioned in my previous mailing, the mechanism in: `nntp-retrieve-headers-with-xover' assumes we're always downloading a contiguous series of headers. Sorry to be so big below. That's what happens when you change the indentation. -- Mike. --- ChangeLog~ Thu Aug 27 10:03:55 1998 +++ ChangeLog Sat Aug 29 13:00:51 1998 @@ -1,3 +1,10 @@ +1998-08-29 Mike McEwan + + * gnus-agent.el (gnus-agent-fetch-headers): Cater for when there's + no .agentview, all articles have been expired, or everything bar a + few downloaded arts have been expired. + (gnus-agent-expire): Mark *all* expired articles as read. + Thu Aug 27 11:03:59 1998 Lars Magne Ingebrigtsen * gnus.el: Gnus v5.6.41 is released. ;;; ;;; Patch 1 ;;; --- gnus-agent.el.orig Thu Aug 27 10:03:56 1998 +++ gnus-agent.el Sat Aug 29 12:51:39 1998 @@ -751,32 +751,41 @@ (pop gnus-agent-group-alist)))) (defun gnus-agent-fetch-headers (group &optional force) - (when (gnus-agent-load-alist group) - (let ((articles (gnus-uncompress-range - (cons (1+ (caar (last (gnus-agent-load-alist group)))) - (cdr (gnus-active group)))))) - ;; Fetch them. - (when articles - (gnus-message 7 "Fetching headers for %s..." group) - (save-excursion - (set-buffer nntp-server-buffer) - (unless (eq 'nov (gnus-retrieve-headers articles group)) - (nnvirtual-convert-headers)) - ;; Save these headers for later processing. - (copy-to-buffer gnus-agent-overview-buffer (point-min) (point-max)) - (let (file) - (when (file-exists-p - (setq file (gnus-agent-article-name ".overview" group))) - (gnus-agent-braid-nov group articles file)) - (gnus-make-directory (nnheader-translate-file-chars - (file-name-directory file))) - (write-region (point-min) (point-max) file nil 'silent) - (gnus-agent-save-alist group articles nil) - (gnus-agent-enter-history - "last-header-fetched-for-session" - (list (cons group (nth (- (length articles) 1) articles))) - (gnus-time-to-day (current-time))) - articles)))))) + (let* ((last (gnus-group-get-parameter group 'last)) + (max (cdr (gnus-active group))) + (articles (if last + (if (> max last) + (gnus-uncompress-range + (cons (1+ last) (cdr (gnus-active group)))) + nil) + (gnus-list-of-unread-articles group)))) + ;; Fetch them. + (when articles + (gnus-message 7 "Fetching headers for %s..." group) + (save-excursion + (set-buffer nntp-server-buffer) + (unless (eq 'nov (gnus-retrieve-headers articles group)) + (nnvirtual-convert-headers)) + ;; Save these headers for later processing. + (copy-to-buffer gnus-agent-overview-buffer (point-min) (point-max)) + (let (file) + (when (file-exists-p + (setq file (gnus-agent-article-name ".overview" group))) + (gnus-agent-braid-nov group articles file)) + (gnus-make-directory (nnheader-translate-file-chars + (file-name-directory file))) + (write-region (point-min) (point-max) file nil 'silent) + (gnus-agent-load-alist group) + (gnus-agent-save-alist group articles nil) + ;; Mark where we've fetched to. + (gnus-group-set-parameter group 'last (cdr (gnus-active group))) + (gnus-agent-enter-history + ;; The last header/article may have been cancelled, but + ;; put an entry in the history for expiry processing. + "headers-fetched-to-here-this-session" + (list (cons group (cdr (gnus-active group)))) + (gnus-time-to-day (current-time))) + articles))))) (defsubst gnus-agent-copy-nov-line (article) (let (b e) @@ -1346,7 +1355,8 @@ (sort gnus-agent-article-alist 'car-less-than-car)) (let* ((alist gnus-agent-article-alist) (prev (cons nil alist)) - (first prev)) + (first prev) + expired) (while (and alist (<= (caar alist) article)) (if (or (not (cdar alist)) @@ -1355,22 +1365,33 @@ (number-to-string (caar alist)) group)))) - (setcdr prev (setq alist (cdr alist))) + (progn + (push (caar alist) expired) + (setcdr prev (setq alist (cdr alist)))) (setq prev alist alist (cdr alist)))) (setq gnus-agent-article-alist (cdr first)) - ;;; Mark all articles up to the first article - ;;; in `gnus-article-alist' as read. + (gnus-agent-save-alist group) + ;; Mark all articles up to the first article + ;; in `gnus-article-alist' as read. (when (caar gnus-agent-article-alist) (setcar (nthcdr 2 info) (gnus-range-add (nth 2 info) (cons 1 (- (caar gnus-agent-article-alist) 1))))) + ;; Maybe everything has been expired from `gnus-article-alist' + ;; and so the above marking as read could not be conducted, + ;; or there are expired article within the range of the alist. + (when (or (not (caar gnus-agent-article-alist)) + (> (car expired) (caar gnus-agent-article-alist))) + (setcar (nthcdr 2 info) + (gnus-add-to-range + (nth 2 info) + (nreverse expired)))) (gnus-dribble-enter (concat "(gnus-group-set-info '" (gnus-prin1-to-string info) - ")")) - (gnus-agent-save-alist group))) + ")")))) expiry-hashtb) (set-buffer history) (setq histories (nreverse (sort histories '<))) ;;; ;;; Patch 2 ;;; --- gnus-agent.el.orig Thu Aug 27 10:03:56 1998 +++ gnus-agent.el Sat Aug 29 12:37:40 1998 @@ -751,32 +751,35 @@ (pop gnus-agent-group-alist)))) (defun gnus-agent-fetch-headers (group &optional force) - (when (gnus-agent-load-alist group) - (let ((articles (gnus-uncompress-range - (cons (1+ (caar (last (gnus-agent-load-alist group)))) - (cdr (gnus-active group)))))) - ;; Fetch them. - (when articles - (gnus-message 7 "Fetching headers for %s..." group) - (save-excursion - (set-buffer nntp-server-buffer) - (unless (eq 'nov (gnus-retrieve-headers articles group)) - (nnvirtual-convert-headers)) - ;; Save these headers for later processing. - (copy-to-buffer gnus-agent-overview-buffer (point-min) (point-max)) - (let (file) - (when (file-exists-p - (setq file (gnus-agent-article-name ".overview" group))) - (gnus-agent-braid-nov group articles file)) - (gnus-make-directory (nnheader-translate-file-chars - (file-name-directory file))) - (write-region (point-min) (point-max) file nil 'silent) - (gnus-agent-save-alist group articles nil) - (gnus-agent-enter-history - "last-header-fetched-for-session" - (list (cons group (nth (- (length articles) 1) articles))) - (gnus-time-to-day (current-time))) - articles)))))) + (let ((articles (if (gnus-agent-load-alist group) + (gnus-sorted-intersection + (gnus-list-of-unread-articles group) + (gnus-uncompress-range + (cons (1+ (caar (last gnus-agent-article-alist))) + (cdr (gnus-active group))))) + (gnus-list-of-unread-articles group)))) + ;; Fetch them. + (when articles + (gnus-message 7 "Fetching headers for %s..." group) + (save-excursion + (set-buffer nntp-server-buffer) + (unless (eq 'nov (gnus-retrieve-headers articles group)) + (nnvirtual-convert-headers)) + ;; Save these headers for later processing. + (copy-to-buffer gnus-agent-overview-buffer (point-min) (point-max)) + (let (file) + (when (file-exists-p + (setq file (gnus-agent-article-name ".overview" group))) + (gnus-agent-braid-nov group articles file)) + (gnus-make-directory (nnheader-translate-file-chars + (file-name-directory file))) + (write-region (point-min) (point-max) file nil 'silent) + (gnus-agent-save-alist group articles nil) + (gnus-agent-enter-history + "last-header-fetched-for-session" + (list (cons group (nth (- (length articles) 1) articles))) + (gnus-time-to-day (current-time))) + articles))))) (defsubst gnus-agent-copy-nov-line (article) (let (b e) @@ -1346,7 +1349,8 @@ (sort gnus-agent-article-alist 'car-less-than-car)) (let* ((alist gnus-agent-article-alist) (prev (cons nil alist)) - (first prev)) + (first prev) + expired) (while (and alist (<= (caar alist) article)) (if (or (not (cdar alist)) @@ -1355,22 +1359,33 @@ (number-to-string (caar alist)) group)))) - (setcdr prev (setq alist (cdr alist))) + (progn + (push (caar alist) expired) + (setcdr prev (setq alist (cdr alist)))) (setq prev alist alist (cdr alist)))) (setq gnus-agent-article-alist (cdr first)) - ;;; Mark all articles up to the first article - ;;; in `gnus-article-alist' as read. + (gnus-agent-save-alist group) + ;; Mark all articles up to the first article + ;; in `gnus-article-alist' as read. (when (caar gnus-agent-article-alist) (setcar (nthcdr 2 info) (gnus-range-add (nth 2 info) (cons 1 (- (caar gnus-agent-article-alist) 1))))) + ;; Maybe everything has been expired from `gnus-article-alist' + ;; and so the above marking as read could not be conducted, + ;; or there are expired article within the range of the alist. + (when (or (not (caar gnus-agent-article-alist)) + (> (car expired) (caar gnus-agent-article-alist))) + (setcar (nthcdr 2 info) + (gnus-add-to-range + (nth 2 info) + (nreverse expired)))) (gnus-dribble-enter (concat "(gnus-group-set-info '" (gnus-prin1-to-string info) - ")")) - (gnus-agent-save-alist group))) + ")")))) expiry-hashtb) (set-buffer history) (setq histories (nreverse (sort histories '<)))